Processing social information from faces is difficult for individuals with autism spectrum disorder (ASD). However, it remains unclear whether individuals with ASD make high-level social trait judgments from faces in the same way as neurotypical individuals. Here, we comprehensively addressed this question using naturalistic face images and representatively sampled traits. Despite similar underlying dimensional structures across traits, online adult participants with self-reported ASD showed different judgments and reduced specificity within each trait compared with neurotypical individuals. Deep neural networks revealed that these group differences were driven by specific types of faces and differential utilization of features within a face. Our results were replicated in well-characterized in-lab participants and partially generalized to more controlled face images (a preregistered study). By investigating social trait judgments in a broader population, including individuals with neurodevelopmental variations, we found important theoretical implications for the fundamental dimensions, variations, and potential behavioral consequences of social cognition.
American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders (5th ed.).
4.
Baron-CohenS.WheelwrightS.SkinnerR.MartinJ.ClubleyE. (2001). The Autism-Spectrum Quotient (AQ): Evidence from Asperger syndrome/high-functioning autism, males and females, scientists and mathematicians. Journal of Autism and Developmental Disorders, 31, 5–17.
5.
BenjaminiY.HochbergY. (1995). Controlling the false discovery rate: A practical and powerful approach to multiple testing. Journal of the Royal Statistical Society B: Methodological, 57, 289–300.
6.
BonnefonJ.-F.HopfensitzA.De NeysW. (2015). Face-ism and kernels of truth in facial inferences. Trends in Cognitive Sciences, 19, 421–422.
7.
BosakJ.SczesnyS.EaglyA. H. (2011). The impact of social roles on trait judgments: A critical reexamination. Personality and Social Psychology Bulletin, 38, 429–440.
8.
CaoR.LiX.BrandmeirN. J.WangS. (2021). Encoding of facial features by single neurons in the human amygdala and hippocampus. Communications Biology, 4, Article 1394. https://doi.org/10.1038/s42003-021-02917-1
9.
CaoR.LinC.HodgeJ.LiX.TodorovA.BrandmeirN. J.WangS. (2022). A neuronal social trait space for first impressions in the human amygdala and hippocampus. Molecular Psychiatry, 27, 3501–3509.
10.
CaoR.WangJ.LinC.De FalcoE.PeterA.ReyH. G.DiCarloJ.TodorovA.RutishauserU.LiX.BrandmeirN. J.WangS. (2020). Feature-based encoding of face identity by single neurons in the human medial temporal lobe. bioRxiv. https://doi.org/10.1101/2020.09.01.278283
ChawarskaK.MacariS.ShicF. (2012). Context modulates attention to social scenes in toddlers with autism. Journal of Child Psychology and Psychiatry, 53, 903–913.
13.
ChelnokovaO.LaengB.EikemoM.RiegelsJ.LøsethG.MaurudH.WillochF.LeknesS. (2014). Rewards of beauty: The opioid system mediates social motivation in humans. Molecular Psychiatry, 19, 746–747.
14.
CogsdillE. J.TodorovA. T.SpelkeE. S.BanajiM. R. (2014). Inferring character from faces: A developmental study. Psychological Science, 25, 1132–1139.
15.
ConstantinoJ. N.GruberC. P. (2012). Social Responsiveness Scale: SRS-2. Western Psychological Services.
DengJ.DongW.SocherR.LiL.-J.LiK.Fei-FeiL. (2009). ImageNet: A large-scale hierarchical image database. In IEEE Conference on Computer Vision and Pattern Recognition (pp. 248–255). IEEE.
18.
de WitT. C. J.Falck-YtterT.von HofstenC. (2008). Young children with autism spectrum disorder look differently at positive versus negative emotional faces. Research in Autism Spectrum Disorders, 2, 651–659.
19.
DobsK.MartinezJ.KellA. J. E.KanwisherN. (2022). Brain-like functional specialization emerges spontaneously in deep neural networks. Science Advances, 8, Article eabl8913. https://doi.org/10.1126/sciadv.abl8913
20.
FiskeS. T.CuddyA. J. C.GlickP. (2007). Universal dimensions of social cognition: Warmth and competence. Trends in Cognitive Sciences, 11, 77–83.
21.
Forgeot d’ArcB.RamusF.LefebvreA.BrottierD.ZallaT.MoukawaneS.AmsellemF.LetellierL.PeyreH.MourenM.-C.LeboyerM.DelormeR. (2016). Atypical social judgment and sensitivity to perceptual cues in autism spectrum disorders. Journal of Autism and Developmental Disorders, 46, 1574–1581.
22.
GrossmanS.GazivG.YeagleE. M.HarelM.MégevandP.GroppeD. M.KhuvisS.HerreroJ. L.IraniM.MehtaA. D.MalachR. (2019). Convergent evolution of face spaces across human face-selective neuronal groups and deep convolutional networks. Nature Communications, 10, Article 4934. https://doi.org/10.1038/s41467-019-12623-6
23.
HadadB.-S.YasharA. (2022). Sensory perception in autism: What can we learn?Annual Review of Vision Science, 8, 239–264.
24.
HamermeshD. S. (2011). Beauty pays. Princeton University Press.
25.
HappeF.RonaldA.PlominR. (2006). Time to give up on a single explanation for autism. Nature Neuroscience, 9, 1218–1220.
26.
HehmanE.SutherlandC. A. M.FlakeJ. K.SlepianM. L. (2017). The unique contributions of perceiver and target characteristics in person perception. Journal of Personality and Social Psychology, 113, 513–529.
27.
HesterN.XieS. Y.HehmanE. (2021). Little between-region and between-country variance when people form impressions of others. Psychological Science, 32, 1907–1917.
28.
HintonG. E.RoweisS. T. (2002). Stochastic neighbor embedding. In Proceedings of the 15th International Conference on Neural Information Processing Systems (pp. 857–864). MIT Press. https://dl.acm.org/doi/10.5555/2968618.2968725
29.
HookerS.ErhanD.KindermansP.-J.KimB. (2019). A benchmark for interpretability methods in deep neural networks. In Proceedings of the 33rd International Conference on Neural Information Processing Systems (pp. 9737–9748). Curran Associates Inc. https://dl.acm.org/doi/10.5555/3454287.3455160
30.
HugenbergK.YoungS. G.SaccoD. F.BernsteinM. J. (2011). Social categorization influences face perception and face memory. In CalderA. J.RhodesG.JohnsonM. H.HaxbyJ. V. (Eds.), Oxford Handbook of Face Perception (pp. 245–262). Oxford University Press.
31.
HusV.LordC. (2014). The Autism Diagnostic Observation Schedule, Module 4: Revised algorithm and standardized severity scores. Journal of Autism and Developmental Disorders, 44, 1996–2012.
32.
JahrE.EikesethS.EldevikS.AaseH. (2007). Frequency and latency of social interaction in an inclusive kindergarten setting: A comparison between typical children and children with autism. Autism, 11, 349–363.
33.
KarK. (2022). A computational probe into the behavioral and neural markers of atypical facial emotion processing in autism. The Journal of Neuroscience, 42, 5115–5126.
34.
KarK.KornblithS.FedorenkoE. (2022). Interpretability of artificial neural network models in artificial intelligence versus neuroscience. Nature Machine Intelligence, 4, 1065–1067.
35.
KelesU.LinC.AdolphsR. (2021). A cautionary note on predicting social judgments from faces with deep neural networks. Affective Science, 2, 438–454.
36.
KliemannD.DziobekI.HatriA.SteimkeR.HeekerenH. R. (2010). Atypical reflexive gaze patterns on emotional faces in autism spectrum disorders. The Journal of Neuroscience, 30, 12281–12287.
37.
KriegeskorteN.MurM.BandettiniP. (2008). Representational similarity analysis—connecting the branches of systems neuroscience. Frontiers in Systems Neuroscience, 2, Article 4. https://doi.org/10.3389/neuro.06.004.2008
38.
LatimierA.KovarskiK.PeyreH.FernandezL. G.GrasD.LeboyerM.ZallaT. (2019). Trustworthiness and dominance personality traits’ judgments in adults with autism spectrum disorder. Journal of Autism and Developmental Disorders, 49, 4535–4546.
39.
LenzG. S.LawsonC. (2011). Looking the part: Television leads less informed citizens to vote based on candidates’ appearance. American Journal of Political Science, 55, 574–589.
40.
LinC.KelesU.AdolphsR. (2021). Four dimensions characterize attributions from faces using a representative set of English trait words. Nature Communications, 12, Article 5168. https://doi.org/10.1038/s41467-021-25500-y
LiuZ.LuoP.WangX.TangX. (2015). Deep learning face attributes in the wild. In Proceedings of the IEEE International Conference on Computer Vision (ICCV) (pp. 3730–3738).
43.
LordC.RutterM.GoodeS.HeemsbergenJ.JordanH.MawhoodL.SchoplerE. (1989). Autism Diagnostic Observation Schedule: A standardized observation of communicative and social behavior. Journal of Autism and Developmental Disorders, 19, 185–212.
44.
MaD. S.CorrellJ.WittenbrinkB. (2015). The Chicago Face Database: A free stimulus set of faces and norming data. Behavior Research Methods, 47, 1122–1135.
45.
MaennerM. J.ShawK. A.BaioJ.WashingtonA.PatrickM.DiRienzoM.ChristensenD. L.WigginsL. D.PettygroveS.AndrewsJ. G.LopezM.HudsonA.BaroudT.SchwenkY.WhiteT.RosenbergC. R.LeeL.-C.HarringtonR. A.HustonM.. . . DietzP. M. (2020). Prevalence of autism spectrum disorder among children aged 8 years—Autism and Developmental Disabilities Monitoring Network, 11 sites, United States, 2016. MMWR Surveillance Summaries, 69, 1–12.
46.
McGrawK. O.WongS. P. (1996). Forming inferences about some intraclass correlation coefficients. Psychological Methods, 1, 30–46.
47.
NeumannD.SpezioM. L.PivenJ.AdolphsR. (2006). Looking you in the mouth: Abnormal gaze in autism resulting from impaired top-down modulation of visual attention. Social Cognitive and Affective Neuroscience, 1, 194–202.
48.
OhD.MartinJ. D.FreemanJ. B. (2022). Personality across world regions predicts variability in the structure of face impressions. Psychological Science, 23, 1240–1256.
49.
Oliveras-RentasR. E.KenworthyL.RobersonR. B.MartinA.WallaceG. L. (2012). WISC-IV profile in high-functioning autism spectrum disorders: Impaired processing speed is associated with increased autism communication symptoms and decreased adaptive communication abilities. Journal of Autism and Developmental Disorders, 42, 655–664.
50.
PardeC. J.HuY.CastilloC.SankaranarayananS.O’TooleA. J. (2019). Social trait information in deep convolutional neural networks trained for face identification. Cognitive Science, 43, Article e12729. https://doi.org/10.1111/cogs.12729
51.
ParkhiO.VedaldiA.ZissermanA. (2019). Deep face recognition. In Proceedings of the British Machine Vision Conference (BMVC) (pp. 1–12). British Machine Vision Association.
52.
PelphreyK.SassonN.ReznickJ. S.PaulG.GoldmanB.PivenJ. (2002). Visual scanning of faces in autism. Journal of Autism and Developmental Disorders, 32, 249–261.
53.
ShicF.WangQ.MacariS. L.ChawarskaK. (2020). The role of limited salience of speech in selective attention to faces in toddlers with autism spectrum disorders. Journal of Child Psychology and Psychiatry, 61, 459–469.
54.
SimonyanK.ZissermanA. (2015). Very deep convolutional networks for large-scale image recognition. In International Conference on Learning Representations.
55.
SpekA.SchatorjéT.ScholteE.van Berckelaer-OnnesI. (2009). Verbal fluency in adults with high functioning autism or Asperger syndrome. Neuropsychologia, 47, 652–656.
56.
SpezioM. L.AdolphsR.HurleyR. S. E.PivenJ. (2007). Analysis of face gaze in autism using “Bubbles.” Neuropsychologia, 45, 144–151.
57.
StolierR. M.HehmanE.FreemanJ. B. (2018). A dynamic structure of social trait space. Trends in Cognitive Sciences, 22, 197–200.
58.
SutherlandC. A. M.BurtonN. S.WilmerJ. B.BloklandG. A. M.GermineL.PalermoR.CollovaJ. R.RhodesG. (2020). Individual differences in trust evaluations are shaped mostly by environments, not genes. Proceedings of the National Academy of Sciences, USA, 117, 10218–10224.
59.
SutherlandC. A. M.YoungA. W. (2022). Understanding trait impressions from faces. British Journal of Psychology, 113, 1056–1078.
60.
TamirD. I.ThorntonM. A.ContrerasJ. M.MitchellJ. P. (2016). Neural evidence that three dimensions organize mental state representation: Rationality, social impact, and valence. Proceedings of the National Academy of Sciences, USA, 113, 194–199.
61.
TodorovA. (2017). Face value: The irresistible influence of first impressions. Princeton University Press.
62.
TodorovA.OlivolaC. Y.DotschR.Mende-SiedleckiP. (2015). Social attributions from faces: Determinants, consequences, accuracy, and functional significance. Annual Review of Psychology, 66, 519–545.
63.
van der MaatenL.HintonG. (2008). Visualizing data using t-SNE. Journal of Machine Learning Research, 9, 2579–2605.
64.
WalkerM.JiangF.VetterT.SczesnyS. (2011). Universals and cultural differences in forming personality trait judgments from faces. Social Psychological and Personality Science, 2, 609–617.
65.
WallachM. A.KoganN. (1959). Sex differences and judgment processes. Journal of Personality, 27, 555–564.
66.
WangS.AdolphsR. (2017a). Reduced specificity in emotion judgment in people with autism spectrum disorder. Neuropsychologia, 99, 286–295.
67.
WangS.AdolphsR. (2017b). Social saliency. In ZhaoQ. (Ed.), Computational and cognitive neuroscience of vision (pp. 171–193). Springer.
68.
WangS.JiangM.DuchesneX. M.LaugesonE. A.KennedyD. P.AdolphsR.ZhaoQ. (2015). Atypical visual saliency in autism spectrum disorder quantified through model-based eye tracking. Neuron, 88, 604–616.
XieS. Y.ThaiS.HehmanE. (2022). Everyday perceiver-context influences on impression formation: No evidence of consistent effects. Personality and Social Psychology Bulletin, 49, 955–968.
71.
ZebrowitzL. A.KikuchiM.FellousJ.-M. (2010). Facial resemblance to emotions: Group differences, impression effects, and race stereotypes. Journal of Personality and Social Psychology, 98, 175–189.
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
