Abstract
The Registered Replication Report (RRR) by Colling et al. (2020; this issue) is a welcome opportunity not only to assess the reliability of attentional shifts induced by viewing numbers, but also to examine influences of potentially moderating factors that were discovered since the original report (Fischer, Castel, Dodd, & Pratt, 2003) of an attentional spatial-numerical association of response codes (Att-SNARC) effect (which already included a replication). The lack of an Att-SNARC effect in this replication project, at odds with other replications using similar methods (e.g., Dodd, Van der Stigchel, Leghari, Fung, & Kingstone, 2008; Galfano, Rusconi, & Umiltà, 2006; Ristic, Wright, & Kingstone, 2006), converges with another recent summary report with different statistical analyses (Pellegrino et al., 2019). Furthermore, no moderation through vividness of mental imagery or either verbal or visual learning styles of participants was found in that latter study.
Colling et al. report that camera use (for eye tracking) was not a crucial moderator in their replication project, contrary to findings concerning the replicability of another foundational study in the cognitive sciences (see Noah, Schul, & Mayo, 2018). Yet other potentially useful detail is missing from the report: First and foremost, depth of number processing is a likely moderator of the Att-SNARC effect, which is stronger when participants compute and retain number meaning (e.g., when they report the number, as in Casarotti, Michielin, Zorzi, & Umiltá, 2007; when they classify its magnitude, as in Zanolie & Pecher, 2014; or when they imagine it, as in Fattorini, Pinto, Merola, D’Onofrio, & Doricchi, 2016, and Pinto et al., 2018). This important aspect was left uncontrolled in our original procedure, and the issue was also deliberately excluded in Colling et al.’s replication project. However, the use of ambiguous or spatial terminology in task descriptions or instructions can induce such processing differences (Pinto, Pellegrino, Marson, Lasaponara, & Doricchi, 2019).
Second, average detection speeds are not reported but would be informative regarding the alleged homogeneity of results across labs. Moreover, spatial associations tend to be stronger in slower responses (Gevers, Verguts, Reynvoet, Caessens, & Fias, 2006). Average response speed also addresses the potential involvement of top-down strategies, such as number contextualization on a mental number line for further cognitive manipulation (cf. Pellegrino et al., 2019).
Third, a limitation related to such strategy use is that neither the RRR itself nor the accompanying materials in the Open Science Framework repository report whether the potential Att-SNARC moderators examined in this project (finger-counting habits, experienced direction of reading and writing, handedness, math skills and math anxiety) were measured before testing, between blocks, or only after the experimentation (as in Pellegrino et al., 2019), or whether participants were explicitly informed about the noninformativeness of numbers (as in our original work) to discourage (or perhaps induce?) mapping strategies.
Finally, Colling et al. do not report the proportion of female participants in each sample (nor did we in the original study); females are now known to have weaker spatial-numerical associations than males (Bull, Cleland, & Mitchell, 2013). In addition, whether participants were recruited within an academic program in the natural sciences (as in the original study) or an academic program in the arts might have indirectly contributed to the extent of number-space association in the sample (e.g., Cipora et al., 2016).
A final consideration is that gaming experience, which has become ever more prevalent, might attenuate attentional signatures (e.g., Castel, Pratt, & Drummond, 2005; Feng, Spence, & Pratt, 2007) but was not examined as a moderator in this replication project. Although we acknowledge the clear results of the project, these many potential moderators both present a need for additional investigation and can inform such future studies.
Despite the replication project’s objective of adjudicating the existence of the Att-SNARC effect, we argue that the impressive absence of evidence for the effect does not constitute terminal evidence of absence. The Att-SNARC effect was discovered with underpowered experiments (N = 14 and 10, respectively), but as Brysbaert (2019) recently pointed out, “true effects that are detected tend to have inflated effect sizes (i.e., a true effect is only significant in an underpowered study when the effect obtained in the study is larger than the effect at the population level)” (p. 1). Moreover, the effect has been conceptually replicated with several other methods (e.g., Casarotti et al., 2007; Myachykov, Ellis, Cangelosi, & Fischer, 2015, 2016; Schuller, Hoffmann, Goffaux, & Schiltz, 2014). Thus, the Att-SNARC effect remains a viable and theoretically important finding because it suggests conceptually driven spatial associations without response selection (Fischer & Knops, 2014; but see Aiello et al., 2012, as well as Shaki & Fischer, 2018, and Pinto et al., 2019, for further theoretical refinement). The Att-SNARC effect is also practically relevant insofar as it likely underpins well-documented attentional effects in mental arithmetic (e.g., Liu, Cai, Verguts, & Chen, 2017; Masson & Pesenti, 2014; Mathieu et al., 2018; Mathieu, Gourjon, Couderc, Thevenot, & Prado, 2016; see also the review in Shaki, Pinhas, & Fischer, 2018). What the RRR formally confirms is that the effect is experimentally finicky and that the classic Posnerian cuing paradigm is almost certainly not the best method with which to measure it.
