Correlations among implicit measures of racial bias tend to be weak, raising questions about their validity. Weak individual-level correlations, however, could reflect a temporally unstable construct. According to the Bias of Crowds model, implicit bias can be fleeting for individuals but, when aggregated, reveals which concepts are widely accessible for a population. Using this framework, we hypothesized that correlations between two tests—the Implicit Association Test (IAT) and Affect Misattribution Procedure (AMP)—would be weak across individuals but stronger across counties. We also predicted that county-level racial bias should relate to meaningful contextual factors, like structural inequality. In support of these hypotheses (N = 2,137 people, 103 counties), the IAT and AMP were correlated more strongly at the level of counties (r = .47) than individuals (r = .24). Furthermore, county-level bias was significantly related to structural inequality. This study shows that implicit measures show substantially more convergent validity at the context than the person level.
Bar-AnanY.NosekB. A. (2014). A comparative investigation of seven indirect attitude measures. Behavior Research Methods, 46(3), 668–688. https://doi.org/10.3758/s13428-013-0410-6
2.
Bar-AnanY.VianelloM. (2018). A multi-method multi-trait test of the dual-attitude perspective. Journal of Experimental Psychology: General, 147(8), 1264–1272. https://doi.org/10.1037/XGE0000383
3.
BossonJ. K.SwannW. B.PennebakerJ. W. (2000). Stalking the perfect measure of implicit self-esteem: The blind men and the elephant revisited?Journal of Personality and Social Psychology, 79(4), 631–643. https://doi.org/10.1037/0022-3514.79.4.631
4.
CalanchiniJ.ShermanJ. W. (2013). Implicit attitudes reflect associative, non-associative, and non-attitudinal processes. Social and Personality Psychology Compass, 7(9), 654–667. https://doi.org/10.1111/spc3.12053
5.
ChettyR.HendrenN.JonesM. R.PorterS. R. (2020). Race and economic opportunity in the United States: An intergenerational perspective. The Quarterly Journal of Economics, 135(2), 711–783. https://doi.org/10.1093/QJE/QJZ042
6.
ChettyR.HendrenN.KlineP.SaezE. (2014). Where is the land of opportunity? The geography of intergenerational mobility in the United States. The Quarterly Journal of Economics, 129(4), 1553–1623. https://doi.org/10.1093/qje/qju022
7.
ChinM. J.QuinnD. M.DhaliwalT. K.LovisonV. S. (2020). Bias in the air: A nationwide exploration of teachers’ implicit racial attitudes, aggregate bias, and student outcomes. Educational Researcher, 49(8), 566–578. https://doi.org/10.3102/0013189X20937240
8.
ClaytonK.HorrilloJ.SnidermanP. M. (2023). The BIAT and the AMP as measures of racial prejudice in political science: A methodological assessment. Political Science Research and Methods, 11(2), 363–373. https://doi.org/10.1017/psrm.2022.56
9.
CohenJ. (1988). The effect size. Statistical power analysis for the behavioral sciences. Abingdon: Routledge, 77–83.
10.
ConnorP.EversE. R. K. (2020). The bias of individuals (in crowds): Why implicit bias is probably a noisily measured individual-level construct. Perspectives on Psychological Science, 15, 1329–1345. https://doi.org/10.1177/1745691620931492
11.
CunninghamW. A.PreacherK. J.BanajiM. R. (2001). Implicit attitude measures: Consistency, stability, and convergent validity. Psychological Science, 12(2), 163–170. https://doi.org/10.1111/1467-9280.00328
12.
EkstromP. D.Le ForestierJ. M.LaiC. K. (2022). Racial demographics explain the link between racial disparities in traffic stops and county-level racial attitudes. Psychological Science, 33(4), 497–509. https://doi.org/10.1177/09567976211053573
13.
ForscherP. S.LaiC. K.AxtJ. R.EbersoleC. R.HermanM.DevineP. G.NosekB. A. (2019). A meta-analysis of procedures to change implicit measures. Journal of Personality and Social Psychology, 117(3), 522–559. https://doi.org/10.1037/pspa0000160
14.
GalvanM. J.PayneB. K. (2024). Implicit bias as a cognitive manifestation of systemic racism. Daedalus, 153(1), 106–122. https://doi.org/10.1162/daed_a_02051
15.
GawronskiB. (2019). Six lessons for a cogent science of implicit bias and its criticism. Perspectives on Psychological Science, 14(4), 574–595. https://doi.org/10.1177/1745691619826015
16.
GawronskiB.MorrisonM.PhillsC. E.GaldiS. (2017). Temporal stability of implicit and explicit measures: A longitudinal analysis. Personality and Social Psychology Bulletin, 43(3), 300–312. https://doi.org/10.1177/0146167216684131
GreenwaldA. G.NosekB. A.BanajiM. R. (2003). Understanding and using the Implicit Association Test: I. An improved scoring algorithm. Journal of Personality and Social Psychology, 85(2), 197–216. https://doi.org/10.1037/0022-3514.85.2.197
19.
HannayJ. W.PayneB. K. (2022). Effects of aggregation on implicit bias measurement. Journal of Experimental Social Psychology, 101, 104331. https://doi.org/10.1016/j.jesp.2022.104331
20.
HehmanE.CalanchiniJ.FlakeJ. K.LeitnerJ. B. (2019). Establishing construct validity evidence for regional measures of explicit and implicit racial bias. Journal of Experimental Psychology: General, 148(6), 1022–1040. https://doi.org/10.1037/xge0000623
21.
HehmanE.FlakeJ. K.CalanchiniJ. (2018). Disproportionate use of lethal force in policing is associated with regional racial biases of residents. Social Psychological and Personality Science, 9(4), 393–401. https://doi.org/10.1177/1948550617711229
22.
JohnsonD. J.ChopikW. J. (2019). Geographic variation in the black-violence stereotype. Social Psychological and Personality Science, 10(3), 287–294. https://doi.org/10.1177/1948550617753522
23.
KurdiB.MannT. C.CharlesworthT. E. S.BanajiM. R. (2019). The relationship between implicit intergroup attitudes and beliefs. Proceedings of the National Academy of Sciences of the United States of America, 116(13), 5862–5871. https://doi.org/10.1073/pnas.1820240116
24.
LeitnerJ. B.HehmanE.AydukO.Mendoza-DentonR. (2016). Racial bias is associated with ingroup death rate for Blacks and Whites: Insights from Project Implicit. Social Science & Medicine, 170, 220–227. https://doi.org/10.1016/j.socscimed.2016.10.007
25.
LeitnerJ. B.HehmanE.SnowdenL. R. (2018). States higher in racial bias spend less on disabled Medicaid enrollees. Social Science & Medicine, 208, 150–157. https://doi.org/10.1016/j.socscimed.2018.01.013
26.
MacheryE. (2016). De-Freuding implicit attitudes. In BrownsteinM.SaulJ. (Eds.), Implicit bias and philosophy, volume 1: Metaphysics and epistemology (1st ed., Vol. 1, pp. 104–129). Oxford Academic.
MonteithM. J.VoilsC. I. (1998). Proneness to prejudiced responses: Toward understanding the authenticity of self-reported discrepancies. Journal of Personality and Social Psychology, 75(4), 901–916. https://doi.org/10.1037/0022-3514.75.4.901
29.
OrchardJ.PriceJ. (2017). County-level racial prejudice and the Black-White gap in infant health outcomes. Social Science & Medicine, 181, 191–198. https://doi.org/10.1016/j.socscimed.2017.03.036
30.
OswaldF. L.MitchellG.BlantonH.JaccardJ.TetlockP. E. (2013). Predicting ethnic and racial discrimination: A meta-analysis of IAT criterion studies. Journal of Personality and Social Psychology, 105(2), 171–192. https://doi.org/10.1037/a0032734
31.
PayneB. K. (2005). Conceptualizing control in social cognition: How executive functioning modulates the expression of automatic stereotyping. Journal of Personality and Social Psychology, 89(4), 488–503. https://doi.org/10.1037/0022-3514.89.4.488
32.
PayneB. K.BurkleyM. A.StokesM. B. (2008). Why do implicit and explicit attitude tests diverge? The role of structural fit. Journal of Personality and Social Psychology, 94(1), 16–31. https://doi.org/10.1037/0022-3514.94.1.16
PayneB. K.VuletichH.LundbergK. B. (2022). Critique of the bias-of-crowds model simply restates the model: Reply to Connor and Evers (2020). Perspectives on Psychological Science, 17(2), 606–610. https://doi.org/10.1177/1745691621997394
35.
PayneB. K.VuletichH. A.Brown-IannuzziJ. L. (2019). Historical roots of implicit bias in slavery. Proceedings of the National Academy of Sciences of the United States of America, 116(24), 11693–11698. https://doi.org/10.1073/pnas.1818816116
36.
PayneB. K.VuletichH. A.LundbergK. B. (2017). The bias of crowds: How implicit bias bridges personal and systemic prejudice. Psychological Inquiry, 28(4), 233–248. https://doi.org/10.1080/1047840X.2017.1335568
37.
RiddleT.SinclairS. (2019). Racial disparities in school-based disciplinary actions are associated with county-level rates of racial bias. Proceedings of the National Academy of Sciences of the United States of America, 116(17), 8255–8260. https://doi.org/10.1073/pnas.1808307116
38.
ShermanS. J.RoseJ. S.KochK.PressonC. C.ChassinL.PressonC. (2003). Implicit and explicit attitudes toward cigarette smoking: The effects of context and motivation. Journal of Social and Clinical Psychology, 22(1), 13–39. https://doi.org/10.1521/jscp.22.1.13.22766
39.
StelterM.EssienI.SanderC.DegnerJ. (2022). Racial bias in police traffic stops: White residents’ county-level prejudice and stereotypes are related to disproportionate stopping of Black drivers. Psychological Science, 33(4), 483–496. https://doi.org/10.1177/09567976211051272
VuletichH. A.SommetN.PayneB. K. (2023). The Great Migration and implicit bias in the Northern United States. Social Psychological and Personality Science, 15, 498–508. https://doi.org/10.1177/19485506231181718
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
