The impact of a commercial wearable tactile bass system (WTBS) on the emotions expressed by music: A comparative study of deaf and hearing individuals

Abstract

This study investigated the effectiveness of a commercial wearable tactile bass system (WTBS), device often used to enhance concert experiences for deaf and hard-of-hearing individuals, in conveying the emotions expressed by music to a mixed panel of participants (deaf and hearing individuals). The central hypothesis proposes that emotions conveyed through music are diminished during audio-tactile substitution (ATS) and that this diminution is reflected in how well both deaf and hearing individuals perceive the emotions. To test this, we assessed participants’ ability to recognize musical emotions from a set of film music excerpts. These excerpts were presented to both deaf and hearing participants under two different experimental conditions: auditory and tactile modality (experimental condition 1), and tactile modality with an auditory masking stimulus (experimental condition 2). To evaluate the emotional content of the music, we used a dimensional model, asking participants to rate their judgments on two dimensions: arousal and valence. The results revealed a significant difference in valence recognition between the two experimental conditions, particularly for music with positive valence. This suggests that the WTBS used was not able to reliably transmit the emotional dimensions of the music, particularly in terms of valence. These findings could enhance our understanding of how musical emotions are experienced through tactile means and inspire further efforts to create more inclusive musical experiences.

Keywords

musical haptics audio-tactile rendering vibrotactile sensory substitution emotion recognition

Get full access to this article

View all access options for this article.

References

Anderson

M. L.

Riker

Hakulin

Meehan

Gagne

Higgins

Stout

Pici-D’Ottavio

Cappetta

Wolf Craig

K. S.

(2020). Deaf ACCESS: Adapting consent through community engagement and state-of-the-art simulation. Journal of Deaf Studies and Deaf Education, 25(1), 115–125. https://doi.org/10.1093/deafed/enz035

Auvray

Lenay

O’Regan

K. J.

Lefèvre

(2005). Suppléance perceptive, immersion et informations proprioceptives [Perceptual substitution, immersive experience, and proprioceptive input]. Arob@se, 1, 94–113.

Auvray

Myin

(2009). Perception with compensatory devices: From sensory substitution to sensorimotor extension. Cognitive Science, 33(6), 1036–1058. https://doi.org/10.1111/j.1551-6709.2009.01040.x

Bach-y-Rita

Collins

C. C.

Saunders

F. A.

White

Scadden

(1969). Vision substitution by tactile image projection. Nature, 221, 963–964. https://doi.org/10.1038/221963a0

Bigand

Vieillard

Madurell

Marozeau

Dacquet

(2005). Multidimensional scaling of emotional responses to music: The effect of musical expertise and of the duration of the excerpts. Cognition and Emotion, 19(8), 1113–1139. https://doi.org/10.1080/02699930500204250

Bradley

M. M.

Lang

P. J.

(1994). Measuring emotion: The self-assessment manikin and the semantic differential. Journal of Behavior Therapy and Experimental Psychiatry, 25(1), 49–59. https://doi.org/10.1016/0005-7916(94)90063-9

Brétéché

(2021a). “Body ways”: The extra-ordinary music of the deaf. In Hoppe

Müller

S. A.

(Eds.), Body in music. Music in the body (pp. 229–253). Olms-Verlag.

Brétéché

(2021b). The deaf musical experience. Bodily and visual specificities: Corpaurality and vusicality. In Aramaki

Derrien

Kronland-Martinet

Ystad

(Eds.), Perception, representations, images, sound, music. 14th international symposium. CMMR 2019 (pp. 422–448). Springer-Verlag.

Cohen

A. J.

(2010). Chapter 31. Music as a source of emotion in film. In Juslin

P. N.

Sloboda

J. A.

(Eds.), Handbook of music and emotion. Theory, research, applications (pp. 879–908). Oxford University Press.

10.

Eerola

Vuoskoski

J. K.

(2011). A comparison of the discrete and dimensional models of emotion in music. Psychology of Music, 39(1), 18–49. https://doi.org/10.1177/0305735610362821

11.

Fourney

D. W.

Fels

D. I.

(2009). Creating access to music through visualization. In Proceedings of the 2009 IEEE Toronto International Conference on Science and Technology for Humanity (TIC-STH) (pp. 939–944). IEEE. https://doi.org/10.1109/TIC-STH.2009.5444364

12.

Haynes

Lawry

Kent

Rossiter

(2021). FeelMusic: Enriching our emotive experience of music through audio-tactile mappings. Multimodal Technologies and Interaction, 5(6), 29. https://doi.org/10.3390/mti5060029

13.

Hénault-Tessier

Christophe

Negrel

(2018). Sourds et malentendants comme publics de la musique. Le statut ambigu des technologies numériques dans une démarche d’accessibilité [Deaf and hard-of-hearing audiences in music. The ambiguous status of digital technologies in an accessibility approach]. Tic & Société, 12(2), 75–102. https://doi.org/10.4000/ticetsociete.2877

14.

Hopkins

Maté-Cid

Fulford

Seiffert

Ginsborg

(2023). Perception and learning of relative pitch by musicians using the vibrotactile mode. Musicæ Scientiae, 27(1), 3–26. https://doi.org/10.1177/10298649211015278

15.

Janata

Tomic

S. T.

Rakowski

S. K.

(2007). Characterization of music-evoked autobiographical memories. Memory, 15(8), 845–860. https://doi.org/10.1080/09658210701734593

16.

Karam

Russo

F. A.

Fels

D. I.

(2009). Designing the model human cochlea: An ambient crossmodal audio-tactile display. IEEE Transactions on Haptics, 2(3), 160–169. https://doi.org/10.1109/TOH.2009.32

17.

Mas-Herrero

Marco-Pallares

Lorenzo-Seva

Zatorre

R. J.

Rodriguez-Fornells

(2013). Individual differences in music reward experiences. Music Perception, 31(2), 118–138. https://doi.org/10.1525/mp.2013.31.2.118

18.

Mazzoni

Bryan-Kinns

(2016). Mood Glove: A haptic wearable prototype system to enhance mood music in film. Entertainment Computing, 17, 9–17. https://doi.org/10.1016/j.entcom.2016.06.002

19.

Merchel

Altinsoy

(2010). Chapter 7. Auditory-tactile experience of music. In Juslin

P. N.

Sloboda

J. A.

(Eds.), Handbook of music and emotion. Theory, research, applications (pp. 123–148). Oxford University Press.

20.

National Institutes of Health. (2019). Guidelines on communicating informed consent for individuals who are deaf or hard-of-hearing and scientists. https://www.nih.gov/health-information/nih-clinical-research-trials-you/researchers-trial-sites

21.

Nineuil

(2022). Neuropsychologie des émotions et de la mémoire musicale dans l’épilepsie du lobe temporal [Neuropsychology of emotions and musical memory in temporal lobe epilepsy] [Doctoral thesis, Université de Lille].

22.

Nineuil

Dellacherie

Samson

(2021). French adaptation of a film music stimulus set: Normative emotional ratings of valence and arousal prompted by music excerpts. Psychology of Music, 50(5), 1707–1715. https://doi.org/10.1177/03057356211050683

23.

Remache-Vinueza

Trujillo-León

Zapata

Sarmiento-Ortiz

Vidal-Verdú

(2021). Audio-tactile rendering: A review on technology and methods to convey musical information through the sense of touch. Sensor, 21(19), 6575. https://doi.org/10.3390/s21196575

24.

Rossi

(2021). L’Analyse de la musique de film: histoire, concepts et méthodes [Analysis of film music: History, concepts, and methods]. Symétrie.

25.

Sainz-de-Baranda Andujar

Gutiérrez-Martín

Miranda-Calero

J. Á.

Blanco-Ruiz

López-Ongil

. (2022). Gender biases in the training methods of affective computing: Redesign and validation of the Self-Assessment Manikin in measuring emotions via audiovisual clips. Frontiers in Psychology, 13, 955530. https://doi.org/10.3389/fpsyg.2022.955530

26.

Saliba

Lorenzo-Seva

Marco-Pallares

Tillmann

Zeitouni

Lehmann

(2016). French validation of the Barcelona Music Reward Questionnaire. PeerJ, 4, e1760. https://doi.org/10.7717/peerj.1760

27.

Schmitz

Holloway

Cho

(2020). Hearing through vibrations: Perception of musical emotions by profoundly deaf people. arXiv, 13265, 1–17. https://doi.org/10.48550/arXiv.2012.13265

28.

Sharp

Bacon

B. A.

Champoux

(2020). Enhanced tactile identification of musical emotion in the deaf. Experimental Brain Research, 238, 1229–1236. https://doi.org/10.1007/s00221-020-05789-9

29.

Sharp

Houde

M.-S.

Bacon

B.-A.

Champoux

(2019). Musicians show better auditory and tactile identification of emotions in music. Frontiers in Psychology, 10, 1976. https://doi.org/10.3389/fpsyg.2019.01976

30.

Song

Dixon

Pearce

M. T.

Halpern

A. R.

(2016). Perceived and induced emotion responses to popular music: Categorical and dimensional models. Music Perception, 33(4), 472–492. https://doi.org/10.1525/mp.2016.33.4.472

31.

SUBPAC. (2016). SUBPAC M2 owner’s manual.

32.

Verrillo

R. T.

(1992). Vibration sensation in humans. Music Perception, 9(3), 281–302. https://psycnet.apa.org/doi/10.2307/40285553

33.

World Medical Association. (2013). World Medical Association Declaration of Helsinki. Ethical principles for medical research involving human subjects. JAMA, 310(20), 2191–2194. https://doi.org/10.1001/jama.2013.281053

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.01 MB