Large sounds and loud numbers? Investigating the bidirectionality and automaticity of cross-modal loudness-number interactions

Abstract

Magnitude dimensions influence the processing of each other resulting in shorter reaction times in classification tasks when the magnitude information in both dimensions matches. These effects are often explained by a shared magnitude representation as proposed by A Theory of Magnitude (ATOM). Interactions between numbers and loudness indicate that loudness may also be represented as a magnitude. Three experiments were conducted to investigate loudness-number interactions with regard to cross-modality, automaticity, bidirectionality, and the influence of processing speed. In Experiment 1, participants classified the numerical value of visually presented numbers relative to a preceding standard number. Tones at different loudness levels were presented simultaneously with the target number. In Experiment 2, participants switched between a numerical classification task and a loudness classification task randomly between trials. Experiment 3 was similar to Experiment 1 but with reduced salience of the auditory dimension. Across all experiments, there was an interaction between loudness and number magnitude, with shorter reaction times for large (small) numbers when they were accompanied by loud (soft) tones compared to soft (loud) tones. In addition, Experiment 2 showed a bidirectional influence as the interaction occurred also in the loudness classification task. The effect of distance on the cross-modal loudness-number interaction only partially occurred, as only the loudness distance had an effect on the interaction, and this effect was mediated by task-relevance. This may reflect an asymmetry in the influence between numbers and loudness. Overall, the findings support the hypothesis that loudness is represented as a magnitude according to ATOM.

Keywords

Magnitude processing ATOM loudness number processing cross-modal

Get full access to this article

View all access options for this article.

References

Alards-Tomalin

Walker

A. C.

Shaw

J. D. M.

Leboe-McGowan

L. C.

(2015). Is 9 louder than 1? Audiovisual cross-modal interactions between number magnitude and judged sound loudness. Acta Psychologica, 160, 95–103. https://doi.org/10.1016/j.actpsy.2015.07.004

Algom

Dekel

Pansky

(1996). The perception of number from the separability of the stimulus: The Stroop effect revisited. Memory & Cognition, 24(5), 557–572. https://doi.org/10.3758/BF03201083

Arend

Henik

(2015). Choosing the larger versus choosing the smaller: Asymmetries in the size congruity effect. Journal of Experimental Psychology: Learning, Memory, and Cognition, 41(6), 1821–1830. https://doi.org/10.1037/xlm0000135

The Audacity Team. (2019). Audacity. https://www.audacityteam.org/

Aust

Barth

(2022). papaja: Prepare reproducible APA journal articles with R Markdown. https://github.com/crsh/papaja

Avitan

Wasserman

Henik

(2024). Endogenous attention modulates automaticity of number processing. Psychonomic Bulletin & Review. https://doi.org/10.3758/s13423-023-02438-4

Bonn

C. D.

Cantlon

J. F.

(2012). The origins and structure of quantitative concepts. Cognitive Neuropsychology, 29(1-2), 149–173. https://doi.org/10.1080/02643294.2012.707122

Bruzzi

Talamini

Priftis

Grassi

(2017). A SMARC effect for loudness. I-Perception, 8(6), Article 204166951774217. https://doi.org/10.1177/2041669517742175

Bueti

Walsh

(2009). The parietal cortex and the representation of time, space, number and other magnitudes. Philosophical Transactions of the Royal Society B: Biological Sciences, 364(1525), 1831–1840. https://doi.org/10.1098/rstb.2009.0028

10.

Cappelletti

Muggleton

Walsh

(2009). Quantity without numbers and numbers without quantity in the parietal cortex. NeuroImage, 46(2), 522–529. https://doi.org/10.1016/j.neuroimage.2009.02.016

11.

Chang

Cho

Y. S.

(2015). Polarity correspondence effect between loudness and lateralized response set. Frontiers in Psychology, 06, Article 683. https://doi.org/10.3389/fpsyg.2015.00683

12.

Cohen Kadosh

Henik

Rubinsten

. (2008). Are Arabic and verbal numbers processed in different ways? Journal of Experimental Psychology: Learning, Memory, and Cognition, 34(6), 1377–1391. https://doi.org/10.1037/a0013413

13.

Dadon

Henik

(2017). Adjustment of control in the numerical Stroop task. Memory & Cognition, 45(6), 891–902. https://doi.org/10.3758/s13421-017-0703-6

14.

Dehaene

(1996). The organization of brain activations in number comparison: Event-related potentials and the additive-factors method. Journal of Cognitive Neuroscience, 8(1), 47–68. https://doi.org/10.1162/jocn.1996.8.1.47

15.

Dehaene

Akhavein

(1995). Attention, automaticity, and levels of representation in number processing. Journal of Experimental Psychology: Learning, Memory, and Cognition, 21(2), 314–326. https://doi.org/10.1037/0278-7393.21.2.314

16.

Dehaene

Bossini

Giraux

(1993). The mental representation of parity and number magnitude. Journal of Experimental Psychology: General, 122(3), 371–396. https://doi.org/10.1037/0096-3445.122.3.371

17.

Ellermeier

Kattner

Raum

(2021). Cross-modal commutativity of magnitude productions. Attention, Perception, & Psychophysics, 83(7), 2955–2967. https://doi.org/10.3758/s13414-021-02385-z

18.

Fairhurst

M. T.

Deroy

(2017). Testing the shared spatial representation of magnitude of auditory and visual intensity. Journal of Experimental Psychology: Human Perception and Performance, 43(3), 629–637. https://doi.org/10.1037/xhp0000332

19.

Faulkenberry

T. J.

Cruise

Lavro

Shaki

(2016). Response trajectories capture the continuous dynamics of the size congruity effect. Acta Psychologica, 163, 114–123. https://doi.org/10.1016/j.actpsy.2015.11.010

20.

Fernandez-Prieto

Spence

Pons

Navarra

(2017). Does language influence the vertical representation of auditory pitch and loudness? I-Perception, 8(3), Article 204166951771618. https://doi.org/10.1177/2041669517716183

21.

Foltz

G. S.

Poltrock

S. E.

Potts

G. R.

(1984). Mental comparison of size and magnitude: Size congruity effects. Journal of Experimental Psychology: Learning, Memory, and Cognition, 10(3), 442–453. https://doi.org/10.1037/0278-7393.10.3.442

22.

Hartmann

Mast

F. W.

(2017). Loudness counts: Interactions between loudness, number magnitude, and space. Quarterly Journal of Experimental Psychology, 70(7), 1305–1322. https://doi.org/10.1080/17470218.2016.1182194

23.

Heinemann

Pfister

Janczyk

(2013). Manipulating number generation: Loud+long=large? Consciousness and Cognition, 22(4), 1332–1339. https://doi.org/10.1016/j.concog.2013.08.014

24.

Henik

Tzelgov

(1982). Is three greater than five: The relation between physical and semantic size in comparison tasks. Memory & Cognition, 10(4), 389–395. https://doi.org/10.3758/BF03202431

25.

Hope

R. M.

(2013). Rmisc: Ryan miscellaneous. https://CRAN.R-project.org/package=Rmisc

26.

Hübner

Druey

M. D.

(2008). Multiple response codes play specific roles in response selection and inhibition under task switching. Psychological Research, 72(4), 415–424. https://doi.org/10.1007/s00426-007-0118-2

27.

Hurewitz

Gelman

Schnitzer

(2006). Sometimes area counts more than number. Proceedings of the National Academy of Sciences, 103(51), 19599–19604. https://doi.org/10.1073/pnas.0609485103

28.

Ito

Hatta

(2004). Spatial structure of quantitative representation of numbers: Evidence from the SNARC effect. Memory & Cognition, 32(4), 662–673. https://doi.org/10.3758/BF03195857

29.

Kaufmann

Koppelstaetter

Delazer

Siedentopf

Rhomberg

Golaszewski

Felber

Ischebeck

(2005). Neural correlates of distance and congruity effects in a numerical Stroop task: An event-related fMRI study. NeuroImage, 25(3), 888–898. https://doi.org/10.1016/j.neuroimage.2004.12.041

30.

Kiesel

Steinhauser

Wendt

Falkenstein

Jost

Philipp

A. M.

Koch

(2010). Control and interference in task switching: A review. Psychological Bulletin, 136(5), 849–874. https://doi.org/10.1037/a0019842

31.

Koch

Schubert

Blankenberger

(2023). The spatial representation of loudness in a timbre discrimination task. I-Perception, 14(6), Article 20416695231213212. https://doi.org/10.1177/20416695231213213

32.

Kochari

A. R.

(2019). Conducting web-based experiments for numerical cognition research. Journal of Cognition, 2(1), 39. https://doi.org/10.5334/joc.85

33.

Kochari

A. R.

Schriefers

(2021). Processing symbolic magnitude information conveyed by number words and by scalar adjectives. Quarterly Journal of Experimental Psychology, 174702182110311. https://doi.org/10.1177/17470218211031158

34.

Lenth

R. V.

(2022). Emmeans: Estimated marginal means, aka least-squares means. https://CRAN.R-project.org/package=emmeans

35.

Lourenco

S. F.

Ayzenberg

Lyu

(2016). A general magnitude system in human adults: Evidence from a subliminal priming paradigm. Cortex, 81, 93–103. https://doi.org/10.1016/j.cortex.2016.04.013

36.

Marks

L. E.

(1987). On cross-modal similarity: Auditory-visual interactions in speeded discrimination. Journal of Experimental Psychology: Human Perception and Performance, 13(3), 384–394. https://doi.org/10.1037/0096-1523.13.3.384

37.

Meiran

(1996). Reconfiguration of processing mode prior to task performance. Journal of Experimental Psychology: Learning, Memory, and Cognition, 22(6), 1423–1442. https://doi.org/10.1037/0278-7393.22.6.1423

38.

Morey

R. D.

(2008). Confidence intervals from normalized data: A correction to Cousineau (2005). Tutorials in Quantitative Methods for Psychology, 4(2), 61–64. https://doi.org/10.20982/tqmp.04.2.p061

39.

Moyer

R. S.

Landauer

T. K.

(1967). Time required for judgements of numerical inequality. Nature, 215(5109), 1519–1520. https://doi.org/10.1038/2151519a0

40.

Oriet

Tombu

Jolicoeur

(2005). Symbolic distance affects two processing loci in the number comparison task. Memory & Cognition, 33(5), 913–926. https://doi.org/10.3758/BF03193085

41.

Pansky

Algom

(1999). Stroop and Garner effects in comparative judgment of numerals: The role of attention. Journal of Experimental Psychology: Human Perception and Performance, 25(1), 39–58. https://doi.org/10.1037/0096-1523.25.1.39

42.

Peirce

Gray

J. R.

Simpson

MacAskill

Höchenberger

Sogo

Kastman

Lindeløv

J. K.

(2019). PsychoPy2: Experiments in behavior made easy. Behavior Research Methods, 51(1), 195–203. https://doi.org/10.3758/s13428-018-01193-y

43.

Petrizzo

Pellegrino

Anobile

Doricchi

Arrighi

(2023). Top-down determinants of the numerosity–time interaction. Scientific Reports, 13(1), 21098. https://doi.org/10.1038/s41598-023-47507-9

44.

Pinel

Piazza

Le Bihan

Dehaene

(2004). Distributed and overlapping cerebral representations of number, size, and luminance during comparative judgments. Neuron, 41(6), 983–993. https://doi.org/10.1016/S0896-6273(04)00107-2

45.

Proctor

R. W.

Cho

Y. S.

(2006). Polarity correspondence: A general principle for performance of speeded binary classification tasks. Psychological Bulletin, 132(3), 416–442. https://doi.org/10.1037/0033-2909.132.3.416

46.

Proctor

R. W.

Xiong

(2015). Polarity correspondence as a general compatibility principle. Current Directions in Psychological Science, 24(6), 446–451. https://doi.org/10.1177/0963721415607305

47.

R Core Team. (2021). R: A language and environment for statistical computing. R Foundation for Statistical Computing. https://www.R-project.org/

48.

Reike

Schwarz

(2017). Exploring the origin of the number-size congruency effect: Sensitivity or response bias? Attention, Perception, & Psychophysics, 79(2), 383–388. https://doi.org/10.3758/s13414-016-1267-4

49.

Rosenberger

J. L.

Gasko

(1983). Comparing location estimators: Trimmed means, medians, and trimmean. In Hoaglin

D. C.

Mosteller

Tukey

J. W.

(Eds.), Understanding robust and exploratory data analysis (pp. 297–338). Wiley.

50.

Santens

Verguts

(2011). The size congruity effect: Is bigger always more? Cognition, 118(1), 94–110. https://doi.org/10.1016/j.cognition.2010.10.014

51.

Schliephake

Bahnmueller

Willmes

Koch

Moeller

(2023). Influences of cognitive control on number processing: New evidence from switching between two numerical tasks. Quarterly Journal of Experimental Psychology, 76(11), 2514–2523. https://doi.org/10.1177/17470218231154155

52.

Schwarz

Ischebeck

(2003). On the relative speed account of number-size interference in comparative judgments of numerals. Journal of Experimental Psychology: Human Perception and Performance, 29(3), 507–522. https://doi.org/10.1037/0096-1523.29.3.507

53.

Schwarz

Stein

(1998). On the temporal dynamics of digit comparison processes. Journal of Experimental Psychology: Learning, Memory, and Cognition, 24(5), 1275–1293. https://doi.org/10.1037/0278-7393.24.5.1275

54.

Sekuler

Rubin

Armstrong

(1971). Processing numerical information: A choice time analysis. Journal of Experimental Psychology, 90(1), 75–80. https://doi.org/10.1037/h0031366

55.

Singmann

Bolker

Westfall

Aust

Ben-Shachar

M. S.

(2022). Afex: Analysis of factorial experiments. https://CRAN.R-project.org/package=afex

56.

Smith

L. B.

Sera

M. D.

(1992). A developmental analysis of the polar structure of dimensions. Cognitive Psychology, 24(1), 99–142. https://doi.org/10.1016/0010-0285(92)90004-L

57.

Spence

(2011). Crossmodal correspondences: A tutorial review. Attention, Perception, & Psychophysics, 73(4), 971–995. https://doi.org/10.3758/s13414-010-0073-7

58.

Spence

Di Stefano

(2024). What, if anything, can be considered an amodal sensory dimension? Psychonomic Bulletin & Review. https://doi.org/10.3758/s13423-023-02447-3

59.

Stevens

S. S.

(1957). On the psychophysical law. Psychological Review, 64(3), 153–181. https://doi.org/10.1037/h0046162

60.

Stevens

S. S.

(1966). Matching functions between loudness and ten other continua. Perception & Psychophysics, 1(1), 5–8. https://doi.org/10.3758/BF03207813

61.

Sudevan

Taylor

D. A.

(1987). The cuing and priming of cognitive operations. Journal of Experimental Psychology: Human Perception and Performance, 13(1), 89–103. https://doi.org/10.1037/0096-1523.13.1.89

62.

Sutherland

C. A. M.

Thut

Romei

(2014). Hearing brighter: Changing in-depth visual perception through looming sounds. Cognition, 132(3), 312–323. https://doi.org/10.1016/j.cognition.2014.04.011

63.

Takeshima

Gyoba

(2013). High-intensity sound increases the size of visually perceived objects. Attention, Perception, & Psychophysics, 75(3), 501–507. https://doi.org/10.3758/s13414-012-0403-z

64.

Tzelgov

Meyer

Henik

(1992). Automatic and intentional processing of numerical information. Journal of Experimental Psychology: Learning, Memory, and Cognition, 18(1), 166–179. https://doi.org/10.1037/0278-7393.18.1.166

65.

van Opstal

Verguts

. (2011). The origins of the numerical distance effect: The same–different task. Journal of Cognitive Psychology, 23(1), 112–120. https://doi.org/10.1080/20445911.2011.466796

66.

Walsh

(2003). A Theory of Magnitude: Common cortical metrics of time, space and quantity. Trends in Cognitive Sciences, 7(11), 483–488. https://doi.org/10.1016/j.tics.2003.09.002

67.

Walsh

(2015). A Theory of Magnitude: The parts that sum to number. In Cohen Kadosh

Dowker

(Eds.), The Oxford Handbook of Numerical Cognition (Vol. 1, pp. 552–565). Oxford University Press. https://doi.org/10.1093/oxfordhb/9780199642342.013.64

68.

Wickham

Averick

Bryan

Chang

McGowan

L. D.

François

Grolemund

Hayes

Henry

Hester

Kuhn

Pedersen

T. L.

Miller

Bache

S. M.

Müller

Ooms

Robinson

Seidel

D. P.

Spinu

. . .Yutani

(2019). Welcome to the tidyverse. Journal of Open Source Software, 4(43), 1686. https://doi.org/10.21105/joss.01686

69.

Wilke

C. O.

(2020). Cowplot: Streamlined plot theme and plot annotations for ‘ggplot2.’ https://CRAN.R-project.org/package=cowplot

70.

Winter

Marghetis

Matlock

(2015). Of magnitudes and metaphors: Explaining cognitive interactions between space, time, and number. Cortex, 64, 209–224. https://doi.org/10.1016/j.cortex.2014.10.015

71.

Yates

M. J.

Loetscher

Nicholls

M. E. R.

(2012). A generalized magnitude system for space, time, and quantity? A cautionary note. Journal of Vision, 12(7), 9. https://doi.org/10.1167/12.7.9

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.05 MB