People tend to categorize the valence of a target stimulus more quickly and accurately if the target appears after a prime stimulus of the same (than of the opposite) valence. The evaluative priming task (EPT) utilizes this priming effect for indirect evaluation measurement but suffers from low reliability. The present research compared the reliability and validity of 2,160 EPT scoring algorithms across 12 datasets. In contrast to current norms to delete trials with error responses and to rely solely on differences in response latency between different task conditions, superior performance was found when incorporating latency and accuracy data, and when basing the score on differences in mean ranking of each condition’s trials by performance (a scale-invariant non-parametric dominance score named G score). We recommend seven new scoring algorithms that, in comparison to current scoring norms, increase internal consistency by a mean of 41% and correlations with other measures by 17%.
AxtJ. R.Bar-AnanY.VianelloM. (2020). The relation between evaluation and racial categorization of emotional faces. Social Psychological and Personality Science, 11, 196–206.
2.
AxtJ.ButtrickN.FengR. Y. (2023). A Comparative Investigation of the Predictive Validity of Four Indirect Measures of Bias and Prejudice. Personality and Social Psychology Bulletin, 01461672221150229.
3.
Bar-AnanY.NosekB. A. (2012). Reporting intentional rating of the primes predicts priming effects in the affective misattribution procedure. Personality and Social Psychology Bulletin, 38(9), 1194–1208.
4.
Bar-AnanY.NosekB. A. (2014). A comparative investigation of seven indirect attitude measures. Behavior Research Methods, 46, 668–688.
5.
CameronC. D.Brown-IannuzziJ. L.PayneB. K. (2012). Sequential priming measures of implicit social cognition: A meta-analysis of associations with behavior and explicit attitudes. Personality and Social Psychology Review, 16(4), 330–350.
6.
CunninghamW. A.PreacherK. J.BanajiM. R. (2001). Implicit attitude measures: Consistency, stability, and convergent validity. Psychological Science, 12(2), 163–170.
7.
FazioR. H.JacksonJ. R.DuntonB. C.WilliamsC. J. (1995). Variability in automatic activation as an unobtrusive measure of racial attitudes: A bona fide pipeline?Journal of Personality and Social Psychology, 69(6), 1013–1027.
8.
FazioR. H.SanbonmatsuD. M.PowellM. C.KardesF. R. (1986). On the automatic activation of attitudes. Journal of Personality and Social Psychology, 50(2), 229–238.
9.
GawronskiB.De HouwerJ. (2014). Implicit measures in social and personality psychology. In ReisH. T.JuddC. M. (Eds.), Handbook of research methods in social and personality psychology (2nd ed., pp. 283–310). Cambridge University Press.
10.
GawronskiB.De HouwerJ.ShermanJ. W. (2020). Twenty-five years of research using implicit measures. Social Cognition, 38(Supplement), s1–s25.
11.
GlashouwerK. A.SmuldersF. T.de JongP. J.RoefsA.WiersR. W. (2013). Measuring automatic associations: Validation of algorithms for the Implicit Association Test (IAT) in a laboratory setting. Journal of Behavior Therapy and Experimental Psychiatry, 44(1), 105–113.
12.
GreenwaldA. G.McGheeD. E.SchwartzJ. L. (1998). Measuring individual differences in implicit cognition: The implicit association test. Journal of Personality and Social Psychology, 74(6), 1464.
13.
GreenwaldA. G.NosekB. A.BanajiM. R. (2003). Understanding and using the implicit association test: I. An improved scoring algorithm. Journal of Personality and Social Psychology, 85(2), 197–216.
14.
GreenwaldA. G.PoehlmanT. A.UhlmannE. L.BanajiM. R. (2009). Understanding and using the Implicit Association Test: III. Meta-analysis of predictive validity. Journal of Personality and Social Psychology, 97(1), 17–41.
15.
HettsJ. J.SakumaM.PelhamB. W. (1999). Two roads to positive regard: Implicit and explicit self-evaluation and culture. Journal of Experimental Social Psychology, 35(6), 512–559.
16.
HuX.GawronskiB.BalasR. (2017). Propositional versus dual-process accounts of evaluative conditioning: I. The effects of co-occurrence and relational information on implicit and explicit evaluations. Personality and Social Psychology Bulletin, 43(1), 17–32.
17.
HughesS.CumminsJ.HusseyI. (2023). Effects on the affect misattribution procedure are strongly moderated by influence awareness. Behavior Research Methods, 55(4), 1558–1586.
18.
Koppehele-GosselJ.HoffmannL.BanseR.GawronskiB. (2020). Evaluative priming as an implicit measure of evaluation: An examination of outlier-treatments for evaluative priming scores. Journal of Experimental Social Psychology, 87, 103905.
19.
McFarlandS. G.CrouchZ. (2002). A cognitive skill confound on the Implicit Association Test. Social Cognition, 20(6), 483–510.
20.
MierkeJ.KlauerK. C. (2003). Method-specific variance in the implicit association test. Journal of Personality and Social Psychology, 85(6), 1180–1192.
21.
NavonM.ShechterA.Bar-AnanY. (2021). The effect of group membership and individuating information on automatic and deliberate evaluation of well-known people. Journal of Personality and Social Psychology, 121(3), 498–523.
22.
NavonM.Bar-AnanY. (2023). Is the automatic evaluation of individual group members inherently biased by their group membership?. Journal of Experimental Social Psychology, 108, 104491.
23.
NosekB. A.Bar-AnanY.SriramN.AxtJ.GreenwaldA. G. (2014). Understanding and using the brief implicit association test: Recommended scoring procedures. PLOS ONE, 9(12), e110938.
24.
OlsonM. A.FazioR. H. (2004). Reducing the influence of extrapersonal associations on the Implicit Association Test: Personalizing the IAT. Journal of Personality and Social Psychology, 86(5), 653–667.
25.
PayneB. K.ChengC. M.GovorunO.StewartB. D. (2005). An inkblot for attitudes: Affect misattribution as implicit measurement. Journal of Personality and Social Psychology, 89(3), 277–293.
26.
RichetinJ.CostantiniG.PeruginiM.SchönbrodtF. (2015). Should we stop looking for a better scoring algorithm for handling Implicit Association Test data? Test of the role of errors, extreme latencies treatment, scoring formula, and practice trials on reliability and validity. PLOS ONE, 10(6), e0129601.
27.
SriramN.NosekB. A.GreenwaldA. (2006). Scale invariant contrasts of response latency distributions.
28.
StahlC.UnkelbachC. (2009). Evaluative learning with single versus multiple unconditioned stimuli: The role of contingency awareness. Journal of Experimental Psychology: Animal Behavior Processes, 35(2), 286–291.
29.
SteffensM. C. (2004). Is the implicit association test immune to faking?Experimental Psychology, 51(3), 165–179.
30.
Teige-MocigembaS.BeckerM.ShermanJ. W.ReichardtR.KlauerK. C. (2017). The affect misattribution procedure: In search of prejudice effects. Experimental Psychology, 64, 215–230.
31.
Teige-MocigembaS.PenzlB.BeckerM.HennL.KlauerK. C. (2016). Controlling the “uncontrollable”: Faking effects on the affect misattribution procedure. Cognition and Emotion, 30(8), 1470–1484.
32.
UnkelbachC.FiedlerK. (2020). The challenge of diagnostic inferences from implicit measures: The case of non-evaluative influences in the evaluative priming paradigm. Social Cognition, 38(Supplement), s208–s222.
33.
Van DesselP.RatliffK.BrannonS. M.GawronskiB.De HouwerJ. (2021). Illusory-correlation effects on implicit and explicit evaluation. Personality and Social Psychology Bulletin, 47, 1480–1494.
34.
VianelloM.Bar-AnanY. (2021). Can the Implicit Association Test measure automatic judgment? The validation continues. Perspectives on Psychological Science, 16, 415–421. https://doi.org/10.1177/174569161989796
35.
ZayasV.WangA. M.McCallaJ. D. (2022). Me as good and me as bad: Priming the self triggers positive and negative implicit evaluations. Journal of Personality and Social Psychology, 122(1), 106–134.
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
