Explicit versus implicit social cognition testing in autism spectrum disorder

Abstract

Although autism spectrum disorder is defined by reciprocal social-communication impairments, several studies have found no evidence for altered social cognition test performance. This study examined explicit (i.e. prompted) and implicit (i.e. spontaneous) variants of social cognition testing in autism spectrum disorder. A sample of 19 adolescents with autism spectrum disorder and 19 carefully matched typically developing controls completed the Dewey Story Test. ‘Explicit’ (multiple-choice answering format) and ‘implicit’ (free interview) measures of social cognition were obtained. Autism spectrum disorder participants did not differ from controls regarding explicit social cognition performance. However, the autism spectrum disorder group performed more poorly than controls on implicit social cognition performance in terms of spontaneous perspective taking and social awareness. Findings suggest that social cognition alterations in autism spectrum disorder are primarily implicit in nature and that an apparent absence of social cognition difficulties on certain tests using rather explicit testing formats does not necessarily mean social cognition typicality in autism spectrum disorder.

Keywords

Asperger’s syndrome assessment mentalizing neurodevelopmental disorders psychometrics theory of mind

Introduction

Qualitative impairments in reciprocal social interaction and communication form two elements of the autism triad. Robust evidence indicates that atypical social cognition accounts for a substantial proportion of such behaviour difficulties in autism spectrum disorder (ASD) (Frith and Happé, 2005; Happé et al., 2006). The term ‘social cognition’ generally refers to the perception, processing and interpretation of information related to social interaction with regard to conspecifics (Brothers, 1990). In ASD, the process of acquisition of inherent social cognition is derailed early on, likely as a result of reduced salience of social stimuli and concomitant enactment of socially irrelevant aspects of the environment (Klin et al., 2003). Social cognition is composed of dual processes: explicit (controlled, slow and increasingly conscious; Frith and Frith, 2008; Satpute and Lieberman, 2006) and implicit cognition (spontaneous, fast and increasingly unconscious; Spunt and Lieberman, 2013), with the implicit system preceding the explicit system (Apperly and Butterfill, 2009; Low and Perner, 2012; Satpute and Lieberman, 2006). Language and executive functions, such as the ability to generalize (to encode and compare abstraction across contexts), are important to combine and harmonize implicit and explicit reasoning in social cognition (Low and Perner, 2012). It is presumed that individuals on the autism spectrum lack implicit social cognition but may acquire explicit skills through learning and experience (Frith, 2004). Normally, they do not automatically attend to socially relevant information but might be able to process the social information when their attention is navigated towards it (Senju, 2012a). In controlled assessment settings including explicit instructions, individuals with ASD often perform better than in spontaneously occurring natural situations (Paul and Cohen, 1985). Dewey (1991) postulated that social cognition in ASD is rather based on trying to follow static rules and logical reasoning than on social awareness and intuition. Frith and Happé (1999) described the social cognition strategies used by individuals with ASD as slow and consciously calculating.

Although social cognition alterations are a hallmark of ASD in a substantial minority of studies, individuals with ASD show a performance in social cognition largely comparable to typically developing (TD) or psychiatric controls, including affect processing tasks, indicating no atypicality or deficit (e.g. Bowler, 1992; Buitelaar et al., 1999; Castelli, 2005; Ozonoff et al., 1990). This seeming paradox might be related to how much social cognition is operationalized explicitly or implicitly, with an increasing explicitness being associated with an increasing likelihood of passing the tests. As Klin et al. (2004) and Ponnet et al. (2004) have pointed out, social cognition tests in the form of social situations often contain in narrative form all necessary facts for making a socially adequate decision. In addition, Dewrang (2011) argued that some of the stories used in social cognition tests are logical in the composition of the plot and that the actions in the stories happen one at a time. In controlled assessment conditions with explicit instructions and limited behavioural options, individuals with ASD may be able to perform better than in unstructured situations in which they must act rather freely and spontaneously (Paul and Cohen, 1985).

Differentiating between spontaneous (i.e. implicit) and elicited (i.e. explicit) social cognition thus might be important when testing individuals with ASD. Based on the evidence described above, we hypothesized that individuals with ASD, when explicitly prompted (by using a multiple-choice format), would reason around descriptions of social situations no differently from TD individuals. In contrast, we hypothesized that individuals with ASD, when judging social situations in a more free, spontaneous unstructured fashion, would give explanations differing from those of TD controls in terms of spontaneous perspective taking and implicit social awareness.

Methods

Participants

The sample comprised 19 adolescents with ASD (6 females, 13 males) with a mean age of 15.1 years (standard deviation (SD) = 1.6 years, range = 13–18 years) and a mean vocabulary scaled score on the Swedish versions of the Wechsler Intelligence Scale for Children–Third Edition (WISC-III; Wechsler, 1999), WISC–Fourth Edition (WISC-IV, Wechsler, 2007) or Wechsler Adult Intelligence Scale–Third Edition (WAIS-III; Wechsler, 2003) of 8.8 (SD = 2.6, range = 4–14). This group was carefully matched pairwise with 19 TD participants for sex (6 females, 13 males), age (mean = 15.3 years, SD = 1.7 years, range = 13–18 years) and vocabulary (mean = 8.8, SD = 2.6, range = 4–14). These highly comparable parallel cohorts were selected from a larger sample of 20 individuals with ASD and 73 with typical development. Informed consent was collected from all participants and their parents. ASD participants were recruited from three child and adolescent neuropsychiatric departments in Örebro County (central Sweden). ASD diagnoses were clinical Diagnostic and Statistical Manual of Mental Disorders (4th ed., text rev.; DSM-IV-TR) consensus diagnoses by child psychiatrists, child psychologists and paediatricians corroborated by results (cut-off met for ASD) from the Swedish version of the Autism Diagnostic Observation Schedule (ADOS) (Lord et al., 1999); exceptions were three participants who were diagnosed clinically only, without the availability of ADOS scores. Single DSM-IV-TR diagnoses in the ASD sample were autistic disorder in 8, Asperger’s disorder in 5 and pervasive developmental disorder–not otherwise specified in 7. Control participants were recruited from two secondary schools and one high school in the city of Örebro.

Instrument and procedure

To examine explicit versus implicit social cognition in ASD, we used the Dewey Story Test (1991). It is a vignette-based test requiring that participants depict violations of social norms using eight sample situations. The test is widely used in Scandinavian countries and beyond in both clinical practice and research (e.g. Blair and Cipolotti, 2000; Dewrang, 2011; Ellis et al., 1994; Söderstrand and Almkvist, 2012; Vermeulen, 2002). To measure explicit social cognition, we used the standard scores of the Dewey Story Test. Here, the test-takers are explicitly asked to rate how they thought most people would judge the described behaviour in the stories if they witnessed it, according to a multiple-choice answering format. To measure implicit social cognition processing, we introduced the concepts of spontaneous perspective taking and implicit social awareness to the Dewey administration and scoring. Spontaneous perspective taking is the ability to explain behaviour using other people’s mental states such as thoughts, feelings and desires, without being explicitly prompted to (Senju, 2012a, 2012b). Implicit social awareness refers to the internalization of social rules and norms that create a cognitive shortcut when predicting a person’s behaviour in a certain situation (Baird and Baldwin, 2001; Low and Perner, 2012; Povinelli and Vonk, 2004).

A Swedish translation of the Dewey Story Test was used (Dewey, 1998). Slight changes were introduced compared to the English original to better fit Swedish culture and language: for example, the names of people appearing in the situations were modified to names commonly given in Sweden; in vignette 4, the currency was changed to Swedish Crowns; and in vignette 3, the probable cause of a screaming child was changed from an open safety pin in the diaper to a wet diaper. The eight Dewey Story Test stories are divided into 24 subsections (1.1, 1.2, etc. to 8.6). For each section, the participant has to decide using a multiple-choice format how most people would perceive the behaviour of a protagonist, using the following categories: (a) fairly normal, (b) rather strange, (c) very eccentric and (d) shocking. The Dewey Story Test and the test instructions are presented in Appendix 1. The test was administered individually in a quiet room, with the stories read aloud to the participants who could follow the story in their own copy of the text.

Explicit social cognition was measured using the Dewey Story Test multiple-choice (4 choices) total score for the 24 subsections, following the scoring principles first applied by Ellis et al. (1994) and later used by Blair and Cipolotti (2000), Vermeulen (2002), Dewrang (2011) and Söderstrand and Almkvist (2012). Here, deviance scores are generated, with increasing scores indicating decreasing performance. Categories are first ranked depending on answer patterns. The responses preferred by a majority of the TD participants are viewed as correct, equalling a score of 0. The cut-off criterion for qualifying as a substantially supported answer was set to a response ratio of 30% (Söderstrand and Almkvist, 2012). The category with second highest support received a deviation score of 1, and the other two categories received deviation scores of 2 and 3 points, respectively. For the Dewey’s total deviance score, the maximum is 57. The test–retest reliability (Pearson r) for this total score within the Swedish version of the Dewey Story Test was examined using responses from 10 participants in the TD group, reaching r_tt = .53 after an average interval of 5.4 months.

To assess implicit social cognition in terms of spontaneous perspective taking and social awareness on the Dewey Story Test, a method derived from Vermeulen (2002) was used. After the explicit assessment using the multiple-choice format, the participants were freely interviewed for 5 min about why they thought most people would perceive the protagonist’s behaviour in a certain way (e.g. ‘Why do you think this behaviour is fairly normal/rather strange/very eccentric/shocking?’). These responses were tape-recorded and transcribed verbatim. Originally, Vermeulen (2002) categorized the answers in six different ways: no motivation, references to general rule, references to self or own experience, references to physical reality in the story, references to the main character’s perspective and references to a minor character’s perspective. From this scheme, we extracted two new categories to simplify the scoring. First, ‘references to general rule’ and ‘references to physical reality in the story’ were collapsed to create a new category labelled ‘implicit social awareness’, containing references about social norms and rules, but no perspective taking. It was defined by comments such as ‘it is not normal to’ or ‘you cannot do that’. Second, ‘references to the main character’s perspective’ and ‘references to a minor character’s perspective’ were collapsed to create a new category labelled ‘spontaneous perspective taking’, containing different forms of perspective taking and how people were feeling and thinking, but no references to social norms and rules. It was defined by comments such as ‘he wants to know her name’ or ‘it is embarrassing for the other person’. A more complete list of common answers operationalizing the categories for each of the 24 subsections of the Dewey’s test is provided in Appendix 2.

The total score for each category was defined by the number of times the participant used the category. When overlapping references that fit both categories were given, a score was assigned for each of the categories separately. For spontaneous perspective taking, the maximum score was 48 (if a participant used both a main and a minor character perspective taking in all sections), and for implicit social awareness, the maximum score was 24 (if a participant gave at least one indication of using an implicit social norm in all sections).

The inter-rater reliability for rating spontaneous perspective taking and implicit social awareness was examined using intra-class correlation between ratings. This correlation was made by an independent layperson blinded to the data, who rated five interview transcripts each from the ASD group and the TD group and compared those ratings with the examiner’s ratings on these cases (720 vs 720 ratings in total). The agreement between the two raters was substantial (r_ic = .70, confidence interval = 0.66–0.74). The test–retest reliability (Pearson r) for spontaneous perspective taking and implicit social awareness scores for 10 TD participants, after an average interval of 5.4 months, reached r_tt = .68 (p = .01) and r_tt = .59 (p = .04), respectively.

Data analyses

To examine explicit versus implicit social cognition performance on the Dewey Story Test between the one-by-one matched ASD and TD groups, t-tests for paired samples were run. Primary assumptions of parametric testing were fulfilled (Gaussian distribution, homogeneity of (error) variances). An alpha of 5% was adopted, but because of the rather small sample sizes and the associated risk for type 2 errors with regard to small effects, trends (p < .10) were also interpreted. Between-group differences were not expected for explicit social cognition (multiple-choice total score) but were predicted for implicit social cognition (spontaneous perspective taking and implicit social awareness scores based on free interview). Given the sample size of this study and the alpha (.05), the test power (1-beta) for detecting significant group mean differences between the samples using a dependent t-statistic was .33 for a small (d = .20), .91 for a medium (d = .50) and .99 for a large (d = 80) effect. Correlations, partialled out for verbal abilities (vocabulary) between the explicit and implicit scores and ADOS scores in the ASD sample, were calculated if informative for understanding the Dewey Story Test findings.

Results

In line with our hypotheses, explicit social cognition (Dewey’s multiple-choice total score) did not differ between the ASD (mean = 6.9, SD = 5.2, range = 1–22) and TD groups (mean = 6.2, SD = 4.1, range = 0–15) (t = .694, p = .50) (Figure 1). In addition, as expected, the ASD group differed from the control group in the implicit social cognition measures. Spontaneous perspective-taking scores were significantly lower and on average 5.1 (SD = 3.6, range = 0–15) in ASD and 7.4 in controls (SD = 3.3, range = 1–13) (t = −2.2, p = .04). For implicit social awareness, there was a trend (t = −1.92; p = .07) for decreased performance in the ASD group (mean = 16.4, SD = 4.4, range = 8–22) compared to the control group (mean = 18.8, SD = 2.5, range = 15–24). In the ASD sample (n = 16, three ADOS missing), ADOS scores for social interaction and communication correlated positively with the Dewey’s multiple-choice total score explicit social cognition measure (r =.53 and .35, p < .03), but negatively with spontaneous perspective taking (r =.−32 and −.40, p < .03) and implicit social cognition measure.

Figure 1.

Box plot for Dewey’s Test performance in explicit social cognition (Dewey’s total multiple-choice score) and implicit social cognition (spontaneous perspective taking and social awareness) between the autism spectrum disorder (ASD) and typically developing (TD) control groups.

Discussion

Consistent with the hypotheses of this study, ASD and TD individuals did not differ in explicit social cognition as operationalized by the multiple-choice prompting on the Dewey’s Story Test. This finding is consistent with the results of Ellis et al. (1994), Söderstrand (2004) and Dewrang (2011), who also found no differences between controls and ASD participants on the Dewey’s Story Test for the multiple-choice total score. Moreover, in line with our expectations, differences emerged with regard to implicit social cognition tasks, operationalized as free verbal judgement of the Dewey’s social situations in terms of spontaneous perspective taking and implicit social awareness. Although individuals with ASD gave indications of spontaneous perspective taking and implicit social awareness, both were present more consistently in the TD group. Because samples were carefully matched, these differences are likely not attributable to differences in language abilities, age or sex.

Similar to our findings, Nah and Poon (2010) also observed differences between explicit and implicit social cognition using closed and open item formats when testing for social cognition skills. They examined children ages 9 to 13 years with ASD and a group of age/sex-matched controls using Dewey Story Test–like vignettes. The children with ASD overall rated the vignettes no differently from their TD peers when using closed categorical item response formats. However, in the interviews of the children regarding how they reached their decisions, the ASD group generated many inappropriate responses, including failure to respond at all.

Interestingly, in the current ASD group, ADOS scores for social interaction and communication problems correlated positively with the explicit social cognition measure on the Dewey Story Test with a closed answering format while implicit social processing in terms of spontaneous perspective taking showed a negative association with ADOS scores. A possible inference is that the explicit social cognition measures not only show limited sensitivity to social cognition atypicalities in ASD but also themselves measure some form of autistic behaviour. The latter is consistent with a multitude of findings indicating that explicit instructions and demands are beneficial for the performance of individuals with ASD, compared to more naturalistic situations requiring active flexible responses (Paul and Cohen, 1985; Senju, 2012a).

Our findings support the notion that explicit, prompted answer formats may decrease the complexity of social cognition demands and facilitate the use of compensatory strategies (Dewrang, 2011; Klin et al., 2004; Ponnet et al., 2004). Such formats surely have strengths with regard to objectivity, economy and standardization but probably at the price of decreased sensitivity to social cognition alterations in ASD. Open formats, such as those used in this study, probably create a more naturalistic way of evaluating social cognition in verbal individuals with ASD. Inter-rater and retest reliability analyses showed good interpersonal fidelity and stability for scoring these unstructured assessments.

This study has several limitations, including a rather small sample size and a lack of inclusion of a more established social cognition measure to explore convergent validity of the Dewey Story Test measures (e.g. Reading Mind in the Eyes (Baron-Cohen et al., 2001) or Strange Stories (Happé, 1994)). Perhaps most notable is the immature stage of our classification of the Dewey’s Story Test multiple-choice performance embodying explicit social cognition skills, and the interview-based perspective taking and social awareness measures of implicit social cognition. Although labelling the multiple-choice measure as explicit is quite straightforward because of its demand for an active and conscious process of selecting answers from a given choice, the labelling of the free interview-based measures as implicit is surely arguable. A general definition of implicit social cognition is unconscious, automatic, fast and not requiring attention or verbal report at the level of psychological test performance. Usually, to assess implicit social cognition, participants perform an apparently irrelevant task (e.g. judging the gender or attractiveness of a person) during collection of an indirect measure such as eye movement, reaction times or neural responses. This design does not apply for the implicit measure used in this study. Thus, ‘implicit’ here is defined rather by an in-depth capacity to demonstrate theory of mind skills, convey comprehension of social life and reason about and spontaneously elaborate on a given multiple-choice reply.

In conclusion, when explicit social cognition was assessed using the Dewey Story Test and a multiple-choice answering format, ASD and TD individuals did not differ in how they thought most people would judge social behaviour. However, ASD and controls did differ when tested with a more complex implicit social cognition measure assessing spontaneous perspective taking and implicit social awareness. The use of categorical, prompted answering formats is likely to decrease the complexity of social cognition demands and might facilitate the use of compensatory strategies, leading to unremarkable social cognition performance in ASD. Rating a free verbal report might create a more naturalistic way of evaluating implicit social understanding.

Footnotes

Appendix 1

Appendix 2 Acknowledgements

We thank all participants, clinical units and schools who participated in this study.

Funding

Sven Bölte was supported by the Swedish Research Council (grant no. 523-2009-7054).

References

Apperly

Butterfill

(2009) Do humans have two systems to track beliefs and belief-like states? Psychological Review 116: 953–970.

Baird

Baldwin

(2001) Making sense of human behavior: action parsing and intentional inference. In: Malle

Moses

Baldwin

(eds) Intentions and Intentionality: Foundations of Social Cognition. Cambridge, MA: The MIT Press, pp. 193–206.

Baron-Cohen

Wheelwright

Hill

. (2001) The ‘Reading the Mind in the Eyes’ test revised version: a study with normal adults, and adults with Asperger syndrome or high-functioning autism. Journal of Child Psychology and Psychiatry 42: 241–251.

Blair

RJR

Cipolotti

(2000) Impaired social response reversal. A case of ‘acquired sociopathy’. Brain 123: 1122–1141.

Bowler

(1992) ‘Theory of mind’ in Asperger’s syndrome. Journal of Child Psychology and Psychiatry 33: 877–893.

Brothers

(1990) The social brain: a project for integrating primate behavior and neurophysiology in a new domain. Concepts in Neuroscience 1: 27–51.

Buitelaar

Van der Gaag

Klin

. (1999) Exploring the boundaries of pervasive developmental disorder not otherwise specified: analyses of data from the DSM-IV Autistic Disorder Field Trial. Journal of Autism and Developmental Disorders 29: 33–43.

Castelli

(2005) Understanding emotions from standardized facial expressions in autism and normal development. Autism 9: 428–449.

Dewey

(1991) Living with Asperger’s syndrome. In: Frith

(ed.) Autism and Asperger Syndrome. Cambridge: University Press, pp. 184–206.

10.

Dewey

(1998) Att leva med Aspergers syndrom. In: Frith

(ed.) Autism och Asperger syndrom. Stockholm: Liber, pp. 234–263.

11.

Dewrang

(2011) Asperger Syndrome: Social Functioning in Relation to Behavioral and Cognitive Traits from Infancy to Young Adulthood: Early Signs, Self-Perception, Ritualistic Traits and Cognitive Ability in Adolescents and Young Adults. Gothenburg: Department of Psychology, University of Gothenburg.

12.

Ellis

Fraser

. (1994) A preliminary study of right hemisphere cognitive deficits and impaired social judgments among young people with Asperger syndrome. European Child & Adolescent Psychiatry 3: 255–266.

13.

Frith

(2008) Implicit and explicit processing in social cognition. Neuron 60: 503–510.

14.

Frith

(2004) Emanuel Miller lecture: confusions and controversies about Asperger syndrome. Journal of Child Psychology and Psychiatry 45: 672–686.

15.

Frith

Happé

(1999) Theory of mind and self-consciousness: what is it like to be autistic? Mind & Language 14: 1–22.

16.

Frith

Happé

(2005) Autism spectrum disorder. Current Biology 11: 786–790.

17.

Happé

(1994) An advanced test of theory of mind: understanding of story characters’ thoughts and feelings by able autistic, mentally handicapped, and normal children and adults. Journal of Autism and Developmental Disorders 24: 129–154.

18.

Happé

Ronald

Plomin

(2006) Time to give up on a single explanation for autism. Nature Neuroscience 9: 1218–1220.

19.

Klin

Jones

Schultz

. (2003) The enactive mind, or from actions to cognition: lessons from autism. Philosophical Transactions of the Royal Society of London Series B: Biological Sciences 358: 345–360.

20.

Klin

Jones

Schultz

. (2004) The enactive mind, or from actions to cognitions: lessons from autism. In: Frith

Hill

(eds) Autism: Mind and Brain. New York: Oxford University Press, pp. 127–160.

21.

Lord

Rutter

DiLavore

. (1999) Autism Diagnostic Observation Schedule-WPS (ADOS-WPS). Los Angeles, CA: Western Psychological Services.

22.

Low

Perner

(2012) Implicit and explicit theory of mind: state of the art. British Journal of Developmental Psychology 30: 1–13.

23.

Nah

Poon

(2010) The perception of social situations by children with autism spectrum disorders. Autism 14: 1–19.

24.

Ozonoff

Pennington

Rogers

(1990) Are there specific emotion perception deficits in young autistic children? Journal of Child Psychology and Psychiatry 31: 343–361.

25.

Paul

Cohen

(1985) Comprehension of indirect requests in adults with autistic disorders and mental retardation. Journal of Speech and Hearing Research 28: 475–479.

26.

Ponnet

Roeyers

Buysse

. (2004) Advanced mind-reading in adults with Asperger syndrome. Autism 8: 249–266.

27.

Povinelli

Vonk

(2004) We don’t need a microscope to explore the chimpanzee mind. Mind & Language 19: 1–28.

28.

Satpute

Lieberman

(2006) Integrating automatic and controlled processes into neurocognitive models of social cognition. Brain Research 1079: 86–97.

29.

Senju

(2012a) Atypical development of spontaneous social cognition in autism spectrum disorders. Brain & Development 35: 96–101.

30.

Senju

(2012b) Spontaneous theory of mind and its absence in autism spectrum disorders. Neuroscientist 18: 108–113.

31.

Söderstrand

(2004) Intuitionens neuropsykologi – neuropsykologisk utredning av social kognition vid ADHD och Aspergers syndrom. Lund: Department of Psychology, University of Lund.

32.

Söderstrand

Almkvist

(2012) Psychometric data on the Eyes Test, the Faux Pas Test, and the Dewey Social Stories Test in a population-based Swedish adult sample. Nordic Psychology 64: 30–43.

33.

Spunt

Lieberman

(2013) The busy social brain: evidence for automaticity and control in the neural systems supporting social cognition and action understanding. Psychological Science 24: 80–86.

34.

Vermeulen

(2002) Beter vroeg dan laat en beter laat dan nooit. Berchem: Epo Uitgeverij.

35.

Wechsler

(1999) Wechsler Intelligence Scale for Children–Third Edition (WISC-III). Stockholm: Pearson Inc.

36.

Wechsler

(2003) Wechsler Adult Intelligence Scale–Third Edition (WAIS-III). Stockholm: Pearson Inc.

37.

Wechsler

(2007) Wechsler Intelligence Scale for Children–Fourth Edition (WISC-IV). Stockholm: Pearson Inc.