Identifying cognitive profiles in children with neurodevelopmental disorders using online cognitive testing

Abstract

Children with neurodevelopmental disorders (NDDs) such as autism spectrum disorder (ASD), and attention deficit hyperactivity disorder (ADHD) tend to exhibit similar deficits in attention and memory ability. Early screening of cognitive deficits in children with NDDs, particularly in preschool children, is fundamental to improving cognitive and academic outcomes. In order to determine cognitive profiles in children with ASD and ADHD, we developed accessible audiovisual instructions for an online battery of 13 cognitive tests. Children ages 4–16 who were diagnosed with ADHD (n = 83), or ASD (n = 37), or who were typically developing children (TD) (n = 86) were recruited. Data were analyzed using a stepwise Discriminant Analysis to determine which cognitive tasks were the strongest discriminators between the diagnostic groups. Results revealed four tasks reflective of working memory, reasoning, and attentional processes, which correctly classified approximately 53–60% of each group. The ADHD group had lower scores on attentional tasks compared to TD, while ASD group had lower scores on reasoning tasks compared to the TD children, and made more attempts across all four tasks. The results from this study stress the need for cognitive screening assessments that include domain-specific items to improve the characterization of executive function deficits and promote academic achievement in all children with NDDs.

Plain language summary

Commonly diagnosed Neurodevelopmental disorders (NDDs) include autism spectrum disorder (ASD), and attention deficit hyperactivity disorder (ADHD). Children with NDDs often experience a wide range of cognitive difficulties which can seriously impact their academic, emotional and behavioural outcomes at school. In this study, we used online cognitive tests that were developed for adults. These ‘gamified’ tasks assess a number of cognitive abilities including working memory, attention, verbal skills, and reasoning. We developed audiovisual instructions to make these tasks more suitable to children with and without NDDs. These tasks were then used in an online sample of children with ASD, ADHD, and typically developing children. We wanted to see how each group of children performed on the tasks, to assess their relative cognitive strengths and difficulties. We found that the tasks could successfully categorize each group of children based on their task performance. The ADHD group had lower scores on attentional tasks compared to TD children. The ASD group had lower scores on reasoning compared to TD children. The cognitive task battery may eventually be used to help identify cognitive difficulties and improve outcomes in children with NDDs.

Keywords

Working memory executive functioning autism spectrum disorder attention deficit hyperactivity disorder

Introduction

Children with neurodevelopmental disorders (NDDs), including those with autism spectrum disorder (ASD) and attention deficit hyperactivity disorder (ADHD), often face academic challenges due to difficulties with executive functioning (EF) skills, which are essential for adaptive behavior and learning (e.g., working memory, planning, inhibition, attention) (Dajani et al., 2016; Powell & Voeller, 2004; Sun & Buys, 2012). Both children with ASD and ADHD have a high prevalence in the general child population and show significant variability in their cognitive functioning” (Tomlinson et al., 2014). Up to 40% of children with ASD (Long et al., 2011) and 30%–60% of children with ADHD have cognitive difficulties (Coghill et al., 2014; Lipszyc & Schachar, 2010). Children with ASD and ADHD similarly present with impairments in processing speed, working memory, and response inhibition, irrespective of comorbid symptoms (Corbett et al., 2009; Karalunas et al., 2018). However, other evidence suggests that these children may have more distinct cognitive phenotypes (Bal et al., 2022; Rosello et al., 2023). Learning difficulties are also highly prevalent and often attributed to deficits in EF in both groups of children, with an incidence of about 44% of children with ADHD and of 65%–85% of children with ASD (Gillberg & Coleman, 2000; Pastor & Reuben, 2008).

Cognitive difficulties may influence the age at which children are diagnosed in addition to exacerbating social impairments. Early screening for cognitive deficits may assist in directing referrals for further assessment, early access to therapies, and improving outcomes for children with ASD and ADHD (Hirota et al., 2021; Landa & Kalb, 2012). Without early screening, children’s deficits may go undetected, leading to significant challenges at school and placing the child at risk for behavioral and emotional difficulties (van Steensel et al., 2011). Early screening can help tailor these intervention programs by identifying concerns in children’s EF domains (Cioni et al., 2016). Cognitive interventions have demonstrated positive effects on attention, working memory, inhibitory skills, and may even promote the use of metacognitive strategies (i.e., monitoring one’s own thinking) (Kerns et al., 2017; Loomes et al., 2008; Nash et al., 2015). However, better characterization of cognitive deficits in children with ASD and ADHD are needed to tailor future target cognitive interventions.

Online cognitive screening in adults and adolescents has advanced significantly in the last two decades. Online administration has advanced the ease and accessibility to cognitive tests that can be used as screeners for deficits in EF. Further, cognitive difficulties in children with ADHD and ASD can vary widely. Cognitive difficulties in children with ADHD and ASD can also be influenced by other factors, such as comorbidities (e.g., learning disorders, anxiety, depression) and environmental factors (e.g., home environment, school support). In turn, online cognitive screeners can aid with individual assessments and appropriate support tailored to specific EF skills in the child (e.g., planning, organization, self-regulation at school).

The primary aim for the current study was to determine whether an established online battery of tests can successfully discriminate targeted cognitive abilities in children with ASD, ADHD and TD children. This battery includes a total of 13 subtests that target working memory, attention, fluid reasoning, and verbal abilities that are “gamified” to maintain interest. Audiovisual instructions were developed to facilitate at-home and online testing in children with ASD and ADHD. Based on the literature (Corbett et al., 2009; Karalunas et al., 2018), we predicted that distinct cognitive patterns would emerge among the groups. ASD and ADHD groups are expected to perform worse than TD children on tasks that target attention/inhibition and working memory, which are reported as core cognitive deficits in both diagnoses. As such, there may be significant overlap in cognitive patterns between the diagnostic groups (Corbett et al., 2009; Karalunas et al., 2018). Children with ADHD may perform particularly poorly on attentional tasks, as difficulty maintaining concentration is a hallmark symptom of this disorder (American Psychiatric Association, 2013). Results may inform the development of accessible screening measures that can be administered to children with ASD and AHDHD, with emphasis on informing appropriate school-based interventions to promote cognitive ability and academic achievement in this population.

Methods

Participants

Participants were English-speaking (verbal) Canadian school-aged children. A broad recruitment strategy was implemented through social media advertisements, and the online research platform, Prolific (https://www.prolific.co/). Potential participants first completed screening questions to determine their eligibility for participation. If participants were deemed eligible to participate, they were then invited to participate in the study via a new study invitation sent through Prolific. Inclusion criteria were as follows: must be able to speak English, Canadian, between the ages of 4–16 years, with or without a diagnosed neurodevelopmental disorder (ASD or ADHD). Exclusion criteria included diagnosed global developmental delay (GDD), and motor difficulties. The study protocol was approved by the Health Sciences Research Ethics Board. Parents provided informed consent and children provided assent.

Online test battery

The online battery (40 minutes) that encompass 13 “gamified” tests that measure (1) working memory, (2) reasoning, (3) verbal abilities, and (4) attention. The 13 tests are: Spatial Span (SS), Grammatical Reasoning (GR), Double Trouble (DT), Odd One Out (OOO), Monkey Ladder (ML), Rotations (R), Feature Match (FM), Digit Span (DS), Spatial Planning (SP), Paired Associates (PA), Polygons (P), Token Search (TS), and the sustained attention to a response task (SART). The tests are on-demand through the research platform provided by Creyos (https://www.creyos.com).

The Creyos platform is used to administer online cognitive tests to children, adolescents, and adults. Creyos has a database of roughly 4.5 million scores from ∼400,000 users, with 75,000 of these scores being used to establish associations between task performance and IQ (Hampshire et al., 2012). The cognitive tasks have been validated in several large-scale studies examining healthy controls and patient populations (Brenkel et al., 2017; Levine et al., 2013). However, these tasks have not yet been validated in children. Cognitive assessments in young adults on the Creyos battery of tests were comparable to that of standardized tests to assess cognitive function such as the Wechsler Adult Intelligence Scale Revised (WAIS-R) (Levine et al., 2013; Wechsler, 1981), and the Montreal Cognitive Assessment (MoCA) (Brenkel et al., 2017). Descriptions of each cognitive task used in the assessment are found in the supplemental information (Wild et al., 2018).

Development of enhanced instructions

The original task instructions consisted of written paragraphs displayed to users, before they were prompted to complete an interactive tutorial. An enhanced set of instructions was developed for children with limited reading abilities that we based on the original instructions. The instructions were created using screen captures of the interactive tutorial, and voiceover stimuli of written instructions were recorded. The voice recordings were developed by one of the authors (A.H.) and were developed in a soundproof booth at the Instructional Technology Resource Centre (ITRC) at Western University. The recordings were made using a standard microphone and the files were saved as MP3 files. The files were uploaded to the Creyos platform and played at the same time the written instructions were displayed to the participants. Participants were then prompted to play the interactive tutorial to practice the tasks while receiving written feedback (e.g., “Oops! That wasn’t the right sequence” or “Correct! You’ve completed this level.”).

To determine whether the audiovisual instructions were appropriate for online testing, field testing was conducted in a sample of children (4–11 years) with NDDs and psychiatric disorders (7 with ADHD, 1 with co-morbid ASD/ADHD, 1 with co-morbid ASD/OCD, and 1 with learning disorder) (n = 10), as well as TD children (n = 6) and their caregivers (n = 12). Participants were administered the audiovisual instructions and the tasks on the Creyos platform during in-person testing sessions. Feedback was gathered from participants on the task’s new audiovisual instructions using an interview guide with set questions regarding the comprehension of each task (see supplemental information). For each task the participants were asked whether they understood the instructions (yes/no/maybe), whether they believe they could perform the test (yes/no/maybe), and specific details about the instructions (open ended responses). Parents were asked whether the instructions are at the ability level of their children as well as specific details on the instructions (open ended responses). Interviews were analyzed using NVivo software to categorize responses and identify themes. Feedback was grouped according to the task’s targeted cognitive domain (short-term memory, attention, verbal abilities, and reasoning) (Hampshire et al., 2012).

Across the short-term memory tasks, 90% of parent responses indicated the instructions were appropriate for their child’s ability. 92% of children indicated that they understood how to play the game after watching the instructions. Across the attention tasks, 90% of parent responses indicated the instructions were appropriate for their child’s ability. 95% of responses from children indicated that they understood how to play the games after watching instructions. Across the verbal tasks, 80% of parent responses indicated the instructions were appropriate for their child’s ability. 100% of responses from children indicated that they understood how to play the games after watching instructions. Across the reasoning tasks, 90% of the parent responses indicated the instructions were appropriate for their child’s ability. 96% of the children indicated that they understood how to play the games after watching instructions.

All children were able to complete the 13 tasks. Note that the youngest participant (4 years) did require some parental assistance (e.g., clicking buttons on laptop, remembering rules). Over half of interview responses for all tasks (children and parents) suggested that the participants could have understood the task rules with fewer examples. Thus, the instructions could be abbreviated in subsequent trials. Overall, the field testing indicated that audiovisual instructions that were subsequently used for the main research study were suitable for administration in these populations (ASD, ADHD and TD children).

Online cognitive testing

The primary component of the study was conducted online through Prolific (https://www.prolific.com), an online participant testing platform used to collect data from large samples of pre-screened participants. The task battery was launched online, with the new audiovisual instructions embedded before each task. The battery is self-guided, and participants watched the audiovisual instructions before commencing the task. After consenting to participate in the study, parents of the participants answered a short demographic questionnaire which took about 15 minutes to complete. Questions were asked about the child’s age, sex, and diagnosis. At the end of the demographic questionnaire, there was a link to the online cognitive tests that the children completed. The entire study participation took approximately 60 min to complete.

Statistical analysis

Statistical analyses were conducted using Statistical Package for the Social Sciences (SPSS, v30, Chicago, IL). For each task, the reaction time, number of errors and correct responses, max scores, and final or average scores were recorded. Missing values in the dataset (less than 3% of the data) were addressed using the mean substitution method. Prior to analysis, all scores were transformed to z-scores and were adjusted for participant age and sex.

In order to identify which cognitive tasks were the strongest discriminators between the diagnostic (Dx) groups, a single stepwise discriminant analysis (DA) and a leave one out cross validation was performed. Stepwise DA is a multivariate method used to estimate group membership using continuous variables (Beggiato et al., 2017). A threshold of the probability of the F value (p < .05) was used to enter the variables into the model, and a probability value (p = .1) was used to remove variables from the analysis. Variables that were entered into the model included participant’s final or average score (depending on the task) for each of the 13 tasks. Final scores were provided in the performance output if the task measured both speed and accuracy (timed tasks, e.g., rotations), while average scores were given from tasks with no time limit (e.g., token search). Upon entering the variables into the model, the probability of group membership was fixed at .5. Standardized canonical function coefficients (SCFC) were given to each continuous variable in the model, which indicates the variable’s relative contribution to group discrimination (larger values indicate a greater contribution). Estimated effect sizes were represented by the percentage of participants who are correctly classified into each Dx group. Significance of observed differences between Dx groups is estimated by the value of Wilk’s Lambda.

Results

Participants

A total of 206 participants were recruited for the study, ages 4 to 16 (M = 7.9, SD = 2.7). Participants included 86 TD children (44 males, 42 females), 83 children with ADHD (40 males, 42 females, 1 other/unspecified), and 37 children with ASD (27 males, 10 females. In order to maintain the focus of the research question, participants with comorbidities and other neurological or behavioral disorders were excluded (e.g., conduct disorder, obsessive compulsive disorder, oppositional defiant disorder, sensory disorder) (Table 1).

Table 1.

Participant demographics.

	Demographics
	N	Age	Sex (M/F/O)
TD	86	8.1 (2.9)	44/42
ASD	37	7.3 (2.4)	27/10
ADHD	83	7.9 (2.6)	40/42/1

Note. Count (n), mean age and standard deviation, and sex (male, female, unspecified) for each group of participants.

Stepwise DA

In order to determine which Creyos tasks were the strongest discriminators between ASD, ADHD, and TD participants, a stepwise DA was performed. With all 13 performance measures on the tasks (i.e., final score, average score) included in the analysis, the stepwise DA identified 55.3% of the original groups correctly (with chance level being 33%). A leave-one-out cross-validation resulted in a comparable classification scheme, whereby 53.4% of the original groups were correctly classified. Based on the stepwise DA, overall, 4 variables that classified 53% of controls, 57% of participants with ADHD and 60% of participants with ASD (Figure 1).

Figure 1.

Predicted group membership based on stepwise discriminant analysis. Left: Proportion of TD (n = 59, 53%) participants correctly classified by the stepwise DA. Middle: Proportion of ADHD participants (n = 46, 57%) correctly classified. Right: Proportion of ASD participants (n = 9, 60%) correctly classified.

The two discriminant functions in the analysis were found to be statistically significant (Wilks’s Λ = 0.85, χ² (8) = 32.79, p = .001 for discriminant function 1 through 2; Wilks’s Λ = 0.93, χ² (3) = 14.20, p = .003 for discriminant function 2, Figure 2.). The first discriminant function explained 57% of the variance and the second discriminant function explained 43%. Canonical correlations are .30 and .26 for both discriminant functions, indicating that 30% and 26% of variances were explained by the relationship between predictors (tasks) and group membership by discriminant function 1 (DF1) and discriminant function 2 (DF2) respectively.

Figure 2.

Plot of canonical function coefficients of the stepwise discriminant analysis for TD (blue stars), ADHD (green triangles) and ASD (red circles) from the participants assessed on the online cognitive tests. The black squares represent group centroids.

The stepwise DA identified 4 tasks (Double Trouble (DT), Odd One Out (OOO), Digit Span (DS), Sustained Attention to Response Task (SART)) reflective of working memory and attentional processes. Canonical discriminant function coefficients revealed that for DF1, DS had the largest contribution to group discrimination followed by OOO, DT, and SART. In DF2, SART had the largest contribution to group discrimination followed by DT, DS, and OOO (See Table 2).

Table 2.

Creyos tasks identified by stepwise DA.

Task	Task description	Task domain	Standardized canonical function coefficients
Task	Task description	Task domain	Function 1	Function 2
DT	Colour-word remapping task	Attention	.147	.590
OOO	Deductive reasoning task	Reasoning	−.678	−.197
DS	Digit sequence recall	Short-term memory	.727	−.494
SART	Go/no go response task for attention and inhibition	Attention	−.069	.744

Note. Descriptions of the Creyos tasks identified as significant discriminators for the cognitive profiles for the 3 different diagnostic groups (TD, ADHD, ASD), with their associated cognitive domain, and discriminant function coefficients for both functions.

Abbreviations: double trouble (DT), odd one out (OOO), digit span (DS), sustained attention to response task (SART).

Multivariate GLMs

To investigate the group differences amongst the task performance measures (i.e., average/final score, number of errors, number correct, attempts, and reaction time), four multivariate GLMs was performed for each of the four tasks identified by the stepwise DA (DT, OOO, DS, and SART).

Results revealed a statistically significant difference in task performance based on participant diagnosis, for DT (F (10, 398) = 7.91, p < .001; Wilk’s Λ = 0.696, partial η2 = .17), OOO (F (10, 398) = 6.74, p < .001; Wilk’s Λ = 0.731, partial η2 = .14), and DS, (F (10, 398) = 7.34, p < .001; Wilk’s Λ = 0.713, partial η2 = .16), and SART (F (10, 398) = 6.79, p < .001; Wilk’s Λ = 0.730, partial η2 = .15) (Table 3)

Table 3.

Tests of Between Subjects Effects.

Variable	DT	OOO	DS	SART
Final score	(F (2, 203) = 28.94; p < .001; partial η² = .22)	(F (2, 203) = 30.88; p < .001; partial η² = .23	(F (2, 203) = 29.96; p < .001; partial η² = .23)	(F (2, 203) = 34.76; p < .001; partial η² = .25)
No. attempts	(F (2, 203) = 8.37; p < .001; partial η² = .08)	(F (2, 203) = 8.69; p < .001; partial η² = .08)	(F (2, 203) = 18.34; p < .001; partial η² = .15)	N/A
No. correct	(F (2, 203) = 14.72; p < .001; partial η² = .13)	(F (2, 203) = .76; p = .47; partial η² = .01)	(F (2, 203) = 18.34; p < .001; partial η² = .15)	(F (2, 203) = 34.76; p < .001; partial η² = .25)
No. errors	(F (2, 203) = 14.24; p < .001; partial η² = .12)	(F (2, 203) = 29.66; p < .001; partial η² = .23)	N/A	(F (2, 203) = 34.76; p < .001; partial η² = .25)
Duration (ms)	(F (2, 203) = 0.67; p = .51; partial η² = .01)	(F (2, 203) = .74; p = .48; partial η² = .01)	(F (2, 203) = 15.32; p < .001; partial η² = .13)	(F (2, 203) = 1.87; p = .16; partial η² = .02)
Ms correct	(F (2, 203) = 3.10, p = .05; partial η2 = .03)	(F (2, 203) = 5.46; p = .005; partial η² = .05)	(F (2, 203) = 6.13; p = .003; partial η² = .06)	(F (2, 203) = 10.12; p < .001; partial η² = .09)

Note. Between subjects effects for each task’s multivariate GLM. Dependent variables include final scores, number of attempts, number correct, number of errors, total task duration, and milliseconds (ms) per correct item. Due to the nature of the tasks, errors and attempts were not available for DS and SART, respectively (N/A).

Multiple comparisons

Double trouble (DT)

Tukey’s HSD Test for multiple comparisons revealed that the mean value of DT final score was significantly higher for controls than ADHD (p < .001, 95% C.I. = [6.92, 14.97]), and the ASD group had higher scores than the ADHD group (p < .001, 95% C.I. = [10.38, 25.83], see Table 4). The ASD group had more attempts than controls (p < .001, 95% C.I. = [7.72, 30.86]) and the ADHD group (p = .007, 95% C.I. = [3.34, 27.00]). The ASD group had more correct responses than controls (p < .001, 95% C.I. = [5.96, 20.50] and ADHD (p < .001, 95% C.I. = [9.20, 24.07]. Lastly, the ADHD group had more errors than controls (p < .001, 95% C.I. = [4.05, 11.01].

Table 4.

Task final scores.

Task	Dx
Task	TD	ADHD	ASD
DT	3.93 (1.08)	−7.02 (1.27)	11.09 (2.94)
SART	23.24 (4.27)	−31.34 (5.03)	13.13 (11.62)
OOO	.47 (.57)	1.18 (.68)	−11.93 (1.56)
DS	−.35 (.09)	.13 (.11)	1.60 (.24)

Note. Estimated marginal means and standard errors (M (SE)) of the final scores for each task and diagnostic group (Dx). Double trouble (DT), odd one out (OOO), digit span (DS), Sustained attention to Response task (SART).

Odd one out (OOO)

Tukey’s HSD Test for multiple comparisons revealed that controls had higher OOO final scores than the ASD group (p < .001, 95% C.I. = [8.38, 16.43], and the ADHD group had higher scores than the ASD group (p < .001, 95% C.I. = [8.99, 17.22]. ASD made more errors than controls (p < .001, 95% C.I. = [7.76, 15.65] and ADHD (p < .001, 95% C.I. = [8.68, 16.75]. ASD also made more attempts than controls (p < .001, 95% C.I. = [6.09, 15.61] and ADHD (p < .001, 95% C.I. = [7.35, 17.08]). Lastly, controls had longer correct item RTs than ASD (p = .024, 95% C.I. = [170.24, 3687.34])

Digit span (DS)

Tukey’s HSD Test for multiple comparisons revealed that the ASD group had higher final DS scores than controls (p = .005, 95% C.I. = [1.34, 2.58]), the ADHD group had higher final scores than controls (p < .001, 95% C.I. = [0.15, 0.82], and the ASD group had higher final scores than the ADHD group (p < .001, 95% C.I. = [0.83, 2.11]. The ASD group made more correct responses than controls (p < .001, 95% C.I. = [1.77, 4.13]) and the ADHD group (p < .001, 95% C.I. = [1.47, 3.88]). The ASD group also made more attempts than controls (p < .001, 95% C.I. = [1.77, 4.13]) and the ADHD group (p < .001, 95% C.I. = [1.47, 3.88]. The ASD group had a longer total task duration (ms) than controls (p < .001, 95% C.I. = [61732.26, 162463.57]) and ADHD (p < .001, 95% C.I. = [32757.48, 135786.02]). Lastly, the ASD group had a longer RT for each correct response than controls (p = .01, 95% C.I. = [823.85, 7343.00])

Sustained attention to response task (SART)

Tukey’s HSD Test for multiple comparisons revealed that controls had significantly higher final scores and correct responses than the ADHD group (p < .001, 95% C.I. = [38.65, 70.51]), and the ASD group had higher final scores and correct responses than the ADHD group (p = .002, 95% C.I. = [13.90, 75.03]). The ADHD group made more errors than controls (p < .001, 95% C.I. = [38.65, 70.51]), and the ASD group (p = .002, 95% C.I. = [13.90, 75.03]). Lastly, controls had a longer RT(ms) for correct trials than the ADHD group (p < .001, 95% C.I. = [77.83, 258.05]).

Discussion

Children with ASD and ADHD are known to have cognitive difficulties impacting a broad range of executive functions. Critically, screening for cognitive difficulties, particularly in young preschool children with ASD and ADHD is essential to identify even subtle deficits, which may become more apparent when tested at school age. In order to address a need to identify cognitive profiles in children with ASD and ADHD, a heterogenous population of young preschool children, children and adolescents along with typically developing peers were recruited and tested on an adapted online battery of cognitive tests that assessed working memory, reasoning, verbal ability and attention. We identified distinct profiles of children with ADHD and with ASD. Overall, children with ADHD performed more poorly on tasks requiring attention compared to typically-developing children. Conversevely, children with ASD performed more poorly on reasoning tasks compared to typically developing children and were more likely to make multiple attempts, indicative of impulsivity. The identification of distinct cognitive profiles can provide the foundation for improved and accessible screening measures that can be administered longitudinally to children with ASD and ADHD.

Cognitive profiles

Of the thirteen tasks administered, two tasks targeting attentional processes (DT, SART) were discriminating the diagnostic groups. While ADHD is particularly associated with attention-regulation impairments, this has been attributed to core inhibitory or working memory deficits (Barkley, 1997; Kofler et al., 2018). Other cognitive domains that emerged as relevant discriminators from the stepwise DA include reasoning and verbal short-term memory, targeted by OOO and DS, respectively. Impairments in spatial reasoning and short-term memory are commonly found in children with ASD (Banker et al., 2021; Cantio et al., 2018).

Results shed light on group differences with respect to each task’s performance measures. For attentional processes (DT), typically developing children and children with ASD had higher final scores compared to children with ADHD. DT is a challenging version of the Stroop task, which demands focused attention and inhibition in order to successfully navigate the incongruent and doubly incongruent trials (Wild et al., 2018). Response inhibition deficits are commonly reported in studies of EF performance in children with ADHD, compared to children without ADHD (Corbett et al., 2009; Willcutt et al., 2005). In the current study, the ADHD group had significantly more errors than the typically-developing children. The children with ASD made more attempts, but had more correct responses than the typically-developing children and ADHD group. This may suggest that the ASD group was more persistent after successful and unsuccessful trials during this task, compared to the other groups. Children with ASD have been known to hyperfocus on a challenging task or activity of interest, which may have contributed to more attempts and correct responses on DT (Dupuis et al., 2022).

TD children and children with ASD also performed better than children with ADHD on the task of sustained attention, the SART (higher scores, more correct items, and fewer errors). Typically-developing children also had a longer RTs for each correct response (no-gos for targets) compared to the ADHD group. The SART is intended to be a sensitive measure of participant’s vigilance to a non-engaging yet challenging task. Thus, controlled processing must be used when they encounter the rare target (number 3) in order to overcome automatic responses to the non-targets (Robertson et al., 1997). These current results are in line with previous work on sustained attention performance of ADHD children which indicates a reduced sensitivity to discriminating between targets and non-targets (Huand-Pollock et al., 2020). Such challenges may also arise due to slow processing speed (Kofler et al., 2018; Leitner et al., 2007).

In the deductive reasoning task, Odd One Out, ADHD and typically-developing children had higher final scores compared to the children with ASD. The ASD group also had more errors and attempts than the typically-developing children and ADHD group. Odd One Out requires participants to keep track of various pattern features (colour, shape, number of items) and they must also be able to quickly deduce which set of rules relates all the patterns together on each trial (Wild et al., 2018). Research has suggested that individuals with ASD may show difficulty with making rapid decisions and prefer a more deliberative approach to reasoning, which is not necessarily available during a brief timed task (Brosnan et al., 2016; Luke et al., 2012). Some research suggests that spatial reasoning and planning are skills which may also be impacted in ASD (Banker et al., 2021; Cantio et al., 2018). Further, children with ASD may show more difficulty than children without ASD in maintaining multiple rules, or shifting these rules due to working memory deficits or cognitive inflexibility, which may be reflected in higher error rates (Landry & Mitchell, 2021).

Lastly, in the working memory task, DS, the children with ASD performed better overall in terms of their final scores (i.e., more correct answers) compared to the typically-developing children and the ADHD group. The children with ADHD also had higher scores compared to the typically-developing children. This is contrary to what would be expected based on working memory impairments typically reported in both ASD and ADHD populations (Habib et al., 2019; Ramos et al., 2020). However, some research has reported no impairment on verbal working memory using digit span tasks in children with ASD, and mixed findings in ADHD (Faja & Dawson, 2014; Rosenthal et al., 2006). The present results suggest that these groups of children were relatively high-functioning as well, with little overall impairment in their verbal working-memory. As with the other tasks, the ASD group also had more attempts than controls and the ADHD group. Lastly, the ASD group had a longer total task duration (ms) than controls and ADHD, and a longer RT for each correct response than controls. Thus, it may be the case that children with ASD were using a more cautious approach to responding on this task, compared to the other groups.

Overall, results demonstrated that an online battery of cognitive tests that target attention, reasoning, and working memory could categorize over half of participants from each diagnostic group, based on performance variables. Although some unique cognitive patterns emerged, it is evident that there is also overlap in cognitive strengths and impairments across diagnostic groups (typically-developing, ASD, and ADHD). Indeed, significant heterogeneity in overall EF (based on attention, inhibition, and working memory performance) has been reported within ASD, ADHD and typically developing children, revealing subgroups according to impairment levels (Dajani et al., 2016). However, results show that the ADHD children performed significantly worse than children without ADHD on the attention tasks. Although similar deficits are reported in ASD, performance measures revealed the greatest attentional/inhibitory impairment in the ADHD group. This is an important finding in the context of dissociating cognitive profiles among children with and without ADHD.

Limitations

A few notable limitations of the current study should be mentioned. First, this study had a relatively small sample size, particularly with respect to ASD participants. Thus, it is difficult to interpret some of the unexpected findings in this group. In future investigations, larger sample sizes may reveal more distinct cognitive patterns among children with different NDDs compared to typically-developing children and may permit more concrete conclusions when comparing ADHD and ASD groups. Further, participants with ADHD or ASD were not assessed for symptom severity, which may have impacted cognitive performance differences among the groups. Indeed, children with more severe ADHD symptomology have demonstrated weaker performance on tasks measuring attention, memory, and response inhibition (Mansour et al., 2021). A second main limitation of the study relates to the online nature of testing. Although the use of the online tasks have been validated in adults, they have not been validated extensively in school-age children. Further, there are some inherent limitations to online assessment, including a loss of researcher control. For example, it is unclear whether participants completed the tasks independently with no outside influence, or assistance from caregivers. Younger children may have been particularly likely to seek assistance; however, based on the field testing we believe that the participants were able to complete the tasks without assistance from their caregivers. Indeed, higher scores have been reported on online cognitive assessments in children when compared to their in-person counterpart (Ashworth et al., 2021). Since an aspect of this study was feasibility in relation to the use of the audiovisual instructions, conclusions are limited until the task battery has been validated in these populations of children. An additional consideration for online cognitive screening is that the participants may have come from backgrounds with higher socioeconomic statuses in general as only families with access to devices and the internet would be eligible to participate.

Conclusions

Children with ASD and ADHD can experience diverse cognitive difficulties, which may affect multiple cognitive domains (e.g., attention, working memory, verbal abilities, and reasoning). When left unidentified, these deficits can seriously impact children’s academic, emotional, and behavioural outcomes (Dajani et al., 2016; Gray et al., 2017). Readily accessible online screeners for cognitive functioning may be a feasible option for early identification of these cognitive difficulties. The online battery of tests identified unique cognitive profiles in children with ADHD, ASD and in TD children, and revealed cognitive patterns across diagnoses. However, results of the present study were mixed, and warrant the need for subsequent work which addresses the above limitations. Future work should examine task performance in a larger sample, in relation to standardized cognitive assessment measures. Overlapping EF impairments underscore the importance of cognitive screening for individuals regardless of their diagnosis. Future work examining whether children with an ASD or ADHD diagnosis have distinct subgroups with varying cognitive profiles should be examined in larger and more heterogenous populations.

Supplemental Material

Supplemental Material - Identifying cognitive profiles in children with neurodevelopmental disorders using online cognitive testing

Supplemental Material for Identifying cognitive profiles in children with neurodevelopmental disorders using online cognitive testing by Abagail Hennessy, Emily S. Nichols, Sarah Al-Saoud, Marie Brossard-Racine, and Emma G. Duerden in Clinical Child Psychology and Psychiatry

Footnotes

Acknowledgements

The authors would like to sincerely thank the families who participated in this research. We thank Ella Christaans and Brian Korviuk for their help with administrative aspects of the study.

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: Funding for this research was provided by the New Frontiers in Research Fund and Mitacs, Canada Research Chairs

Ethical statement

Consent to participate

For participants younger than 18 years of age, consent was obtained from their guardians and assent obtained from the participant.

ORCID iD

Emma G Duerden

Data availability statement

The datasets generated and/or analysed during the current study are available upon reasonable request.

Supplemental Material

Supplemental material for this article is available online.

Author biographies

Abagail Hennessy is a PhD student in the field of School and Applied Child Psychology in the faculty of Education at Western University.

Emily S. Nichols is a neuroscientist and is a Research Scientist and Adjunct Research Professor in the Faculty of Education at Western University.

Sarah A. Al-Saoud is a Master's student in the field of School and Applied Child Psychology in the faculty of Education at Western University.

Marie Brossard-Racine is a neuroscientist and an occupational therapist. She is an Associate Professor in the School of Physical and Occupational Therapy at McGill University. She holds a Canada Research Chair in Brain and Child Development.

Emma G. Duerden is a neuroscientist and is an Assistant Professor in the Faculty of Education at Western University. She holds a Canada Research Chair in Neuroscience and Learning Disorders.

References

American Psychiatric Association . (2013). Diagnostic and statistical manual of mental disorders: DSM-5 (5th ed.). https://doi.org/10.1176/appi.books.9780890425596

Ashworth

Palikara

Burchell

Purser

Nikolla

Van Herwegen

(2021). Online and face-to-face performance on two cognitive tasks in children with Williams Syndrome. Frontiers in Psychology, 11, 594465. https://doi.org/10.3389/fpsyg.2020.594465

Baddeley

A. D.

(1968). A 3 min reasoning test based on grammatical transformation. Psychonomic Science, 10(10), 341–342. https://doi.org/10.3758/BF03331551

Bal

V. H.

Wilkinson

Fok

(2022). Cognitive profiles of children with autism spectrum disorder with parent-reported extraordinary talents and personal strengths. Autism: The International Journal of Research and Practice, 26(1), 62–74. https://doi.org/10.1177/13623613211020618

Banker

S. M.

Schiller

Foss-Feig

(2021). Hippocampal contributions to social and cognitive deficits in autism spectrum disorder. Trends in Neurosciences, 44(10), 793–807. https://doi.org/10.1016/j.tins.2021.08.005

Barkley

R. A.

(1997). Behavioral inhibition, sustained attention, and executive functions: Constructing a unifying theory of ADHD. Psychological Bulletin, 121(1), 65–94. https://doi.org/10.1037/0033-2909.121.1.65

Beggiato

Peyre

Maruani

Scheid

Rastam

Amsellem

Gillberg

C. I.

Leboyer

Bourgeron

Gillberg

Delorme

(2017). Gender differences in autism spectrum disorders: Divergence among specific core symptoms. Autism Research: Official Journal of the International Society for Autism Research, 10(4), 680–689. https://doi-org.proxy1.lib.uwo.ca/10.1002/aur.1715

Brenkel

Shulman

Hazan

Herrmann

Owen

A. M.

(2017). Assessing capacity in the elderly: Comparing the MoCA with a novel computerized battery of executive function. Dementia and Geriatric Cognitive Disorders Extra, 7(2), 249–256. https://doi.org/10.1159/000478008

Brosnan

Lewton

Ashwin

(2016). Reasoning on the autism spectrum: A dual process theory account. Journal of Autism and Developmental Disorders, 46(6), 2115–2125. https://doi.org/10.1007/s10803-016-2742-4

10.

Cantio

White

Madsen

G. F.

Bilenberg

Jepsen

J. R. M.

(2018). Do cognitive deficits persist into adolescence in autism? Autism Research: Official Journal of the International Society for Autism Research, 11(9), 1229–1238. https://doi.org/10.1002/aur.1976

11.

Cattell

(1949). Culture free intelligence test, scale 1, handbook. Institute of Personality and Ability Testing.

12.

Cioni

Inguaggiato

Sgandurra

(2016). Early intervention in neurodevelopmental disorders: Underlying neural mechanisms. Developmental Medicine and Child Neurology, 58(Suppl 4), 61–66. https://doi.org/10.1111/dmcn.13050

13.

Coghill

D. R.

Seth

Matthews

(2014). A comprehensive assessment of memory, delay aversion, timing, inhibition, decision making and variability in attention deficit hyperactivity disorder: Advancing beyond the three-pathway models. Psychological Medicine, 44(9), 1989–2001. https://doi.org/10.1017/S0033291713002547

14.

Collins

Roberts

A. C.

Dias

Everitt

B. J.

Robbins

T. W.

(1998). Perseveration and strategy in a novel spatial self-ordered sequencing task for nonhuman primates: Effects of excitotoxic lesions and dopamine depletions of the prefrontal cortex. Journal of Cognitive Neuroscience, 10(3), 332–354. https://doi.org/10.1162/089892998562771

15.

Corbett

B. A.

Constantine

L. J.

Hendren

Rocke

Ozonoff

(2009). Examining executive functioning in children with autism spectrum disorder, attention deficit hyperactivity disorder and typical development. Psychiatry Research, 166(2-3), 210–222. https://doi.org/10.1016/j.psychres.2008.02.005

16.

Corsi

P. M.

(1972). Human memory and the medial temporal region of the brain. Doctoral Thesis at McGill University (Canada).

17.

Dajani

Llabre

Nebel

Mostofsky

S. H.

Uddin

L. Q.

(2016). Heterogeneity of executive functions among comorbid neurodevelopmental disorders. Scientific Reports, 6(36566), Article 36566. https://doi.org/10.1038/srep36566

18.

Dupuis

Mudiyanselage

Burton

C. L.

Arnold

P. D.

Crosbie

Schachar

R. J.

(2022). Hyperfocus or flow? Attentional strengths in autism spectrum disorder. Frontiers in Psychiatry, 13, 886692. https://doi.org/10.3389/fpsyt.2022.886692

19.

Faja

Dawson

(2014). Performance on the dimensional change card sort and backward digit span by young children with autism without intellectual disability. Child Neuropsychology: A Journal on Normal and Abnormal Development in Childhood and Adolescence, 20(6), 692–699. https://doi.org/10.1080/09297049.2013.856395

20.

Folstein

M. F.

Folstein

S. E.

McHugh

P. R.

(1975). “Mini-mental state”. A practical method for grading the cognitive state of patients for the clinician. Journal of Psychiatric Research, 12(3), 189–198. https://doi.org/10.1016/0022-3956(75)90026-6

21.

Gillberg

Coleman

(2000). The biology of the autistic syndromes. Cambridge University Press.

22.

Gould

R. L.

Brown

R. G.

Owen

A. M.

Bullmore

E. T.

Williams

S. C. R.

Howard

R. J.

(2005). Functional neuroanatomy of successful paired associate learning in Alzheimer’s disease. American Journal of Psychiatry, 162(11), 2049–2060. https://doi.org/10.1176/appi.ajp.162.11.2049

23.

Gray

S. A.

Dueck

Rogers

Tannock

(2017). Qualitative review synthesis: The relationship between inattention and academic achievement. Educational Research, 59(1), 17–35. https://doi.org/10.1080/00131881.2016.1274235

24.

Habib

Harris

Pollick

Melville

(2019). A meta-analysis of working memory in individuals with autism spectrum disorders. PLoS One, 14(4), Article e0216198. https://doi.org/10.1371/journal.pone.0216198

25.

Hampshire

Highfield

Parkin

Owen

(2012). Fractionating human intelligence. Neuron, 76(6), 1225–1237. https://doi.org/10.1016/j.neuron.2012.06.022

26.

Hirota

Bishop

Adachi

Shui

Takahashi

Mori

Nakamura

(2021). Utilization of the maternal and child health handbook in early identification of autism spectrum disorder and other neurodevelopmental disorders. Autism Research: Official Journal of the International Society for Autism Research, 14(3), 551–559. https://doi.org/10.1002/aur.2442

27.

Huang-Pollock

Ratcliff

McKoon

Roule

Warner

Feldman

Wise

(2020). A diffusion model analysis of sustained attention in children with attention deficit hyperactivity disorder. Neuropsychology, 34(6), 641–653. https://doi.org/10.1037/neu0000636

28.

Inoue

Matsuzawa

(2007). Working memory of numerals in chimpanzees. Current Biology: CB, 17(23), R1004–R1005. https://doi.org/10.1016/j.cub.2007.10.027

29.

Karalunas

S. L.

Hawkey

Gustafsson

Miller

Langhorst

Cordova

Fair

Nigg

J. T.

(2018). Overlapping and distinct cognitive impairments in attention-deficit/hyperactivity and autism spectrum disorder without intellectual disability. Journal of Abnormal Child Psychology, 46(8), 1705–1716. https://doi.org/10.1007/s10802-017-0394-2

30.

Kerns

K. A.

Macoun

MacSween

Pei

Hutchison

(2017). Attention and working memory training: A feasibility study in children with neurodevelopmental disorders. Applied Neuropsychology: Child, 6(2), 120–137. https://doi.org/10.1080/21622965.2015.1109513

31.

Kofler

M. J.

Sarver

D. E.

Harmon

S. L.

Moltisanti

Aduen

P. A.

Soto

E. F.

Ferretti

(2018). Working memory and organizational skills problems in ADHD. The Journal of Child Psychology and Psychiatry and Allied Disciplines, 59(1), 57–67. https://doi.org/10.1111/jcpp.12773

32.

Landa

R. J.

Kalb

L. G.

(2012). Long-term outcomes of toddlers with autism spectrum disorders exposed to short-term intervention. Pediatrics, 130(Suppl 2), S186–S190. https://doi.org/10.1542/peds.2012-0900Q

33.

Landry

Mitchell

(2021). An examination of perseverative errors and cognitive flexibility in autism. PLoS ONE, 16(1), 19. https://doi.org/10.1371/journal.pone.0223160

34.

Leitner

Doniger

G. M.

Barak

Simon

E. S.

Hausdorff

J. M.

(2007). A novel multidomain computerized cognitive assessment for attention-deficit hyperactivity disorder: Evidence for widespread and circumscribed cognitive deficits. Journal of Child Neurology, 22(3), 264–276. https://doi.org/10.1177/0883073807299859

35.

Levine

Bacopulous

Anderson

Black

Davidson

Fitneva

McAndrews

Spaniol

Jeyakumar

Abdi

Beaton

Owen

Hampshire

(2013). Validation of a novel computerized test battery for automated testing [Paper Presentation]. In Stroke. Lippincott Williams & Wilkins.

36.

Lipszyc

Schachar

(2010). Inhibitory control and psychopathology: A meta-analysis of studies using the stop signal task. Journal of the International Neuropsychological Society: JINS, 16(6), 1064–1076. https://doi.org/10.1017/S1355617710000895

37.

Long

Gurka

M. J.

Blackman

(2011). Cognitive skills of young children with and without autism spectrum disorder using the BSID-III. Autism Research and Treatment, 2011(3), Article 759289. https://doi.org/10.1155/2011/759289

38.

Loomes

Rasmussen

Pei

Manji

Andrew

(2008). The effect of rehearsal training on working memory span of children with fetal alcohol spectrum disorder. Research in Developmental Disabilities, 29(2), 113–124. https://doi.org/10.1016/j.ridd.2007.01.001

39.

Luke

Clare

I. C. H.

Ring

Redley

Watson

(2012). Decision-making difficulties experienced by adults with autism spectrum conditions. Autism: The International Journal of Research and Practice, 16(6), 612–621. https://doi.org/10.1177/1362361311415876

40.

Mansour

Ward

A. R.

Lane

D. M.

Loveland

K. A.

Aman

M. G.

Jerger

Schachar

R. J.

Pearson

D. A.

(2021). ADHD severity as a predictor of cognitive task performance in children with Autism Spectrum Disorder (ASD). Research in Developmental Disabilities, 111, Article 103882. https://doi.org/10.1016/j.ridd.2021.103882

41.

Nash

Stevens

Greenbaum

Weiner

Koren

Rovet

(2015). Improving executive functioning in children with fetal alcohol spectrum disorders. Child Neuropsychology: A Journal on Normal and Abnormal Development in Childhood and Adolescence, 21(2), 191–209. https://doi.org/10.1080/09297049.2014.889110

42.

Pastor

P. N.

Reuben

C. A.

(2008). Diagnosed attention deficit hyperactivity disorder and learning disability: United States, 2004–2006. Vital and Health Statistics. Series 10, Data from the National Health Survey, 10(237), 1–14. https://www.lib.uwo.ca/cgi-bin/ezpauthn.cgi?url=http://search.proquest.com/reports/diagnosed-attention-deficit-hyperactivity/docview/61998418/se-2

43.

Powell

K. B.

Voeller

K. K. S.

(2004). Prefrontal executive function syndromes in children. Journal of Child Neurology, 19(10), 785–797. https://www.lib.uwo.ca/cgi-bin/ezpauthn.cgi?url=https://search.proquest.com/scholarly-journals/prefrontal-executive-function-syndromes-children/docview/620565430/se-2?accountid=15115

44.

Ramos

A. A.

Hamdan

A. C.

Machado

(2020). A meta-analysis on verbal working memory in children and adolescents with ADHD. The Clinical Neuropsychologist, 34(5), 873–898. https://doi.org/10.1080/13854046.2019.1604998

45.

Robertson

I. H.

Manly

Andrade

Baddeley

B. T.

Yiend

(1997). “Oops!”: Performance correlates of everyday attentional failures in traumatic brain injured and normal subjects. Neuropsychologia, 35(6), 747–758. https://doi.org/10.1016/S0028-3932(97)00015-8

46.

Rosello

Martinez-Raga

Tomas

J. M.

Mira

Cortese

(2023). Cognitive and behavioral profiles in children with autism spectrum disorder with and without Attention-Deficit/hyperactivity disorder. Child and Adolescent Mental Health, 28(2), 269–276. https://doi.org/10.1111/camh.12562

47.

Rosenthal

E. N.

Riccio

C. A.

Gsanger

K. M.

Jarratt

K. P.

(2006). Digit Span components as predictors of attention problems and executive functioning in children. Archives of Clinical Neuropsychology: The Official Journal of the National Academy of Neuropsychologists, 21(2), 131–139. https://doi.org/10.1016/j.acn.2005.08.004

48.

Shallice

(1982). Specific impairments of planning. Philosophical transactions of the Royal Society of London. Series B, Biological sciences, 298(1089), 199–209. https://doi.org/10.1098/rstb.1982.0082

49.

Silverman

Choi

Mackewn

Fisher

Moro

Olshansky

(2000). Evolved mechanisms underlying wayfinding: Further studies on the hunter-gatherer theory of spatial sex differences. Evolution and Human Behavior: Official Journal of the Human Behavior and Evolution Society, 21(3), 201–213. https://doi.org/10.1016/S1090-5138(00)00036-2

50.

Stroop

J. R.

(1935). Studies of interference in serial verbal reactions. Journal of Experimental Psychology, 18(6), 643–662. https://doi.org/10.1037/h0054651

51.

Sun

Buys

(2012). Early executive function deficit in preterm children and its association with neurodevelopmental disorders in childhood: A literature review. International Journal of Adolescent Medicine and Health, 24(4), 291–299. https://doi.org/10.1515/ijamh.2012.042

52.

Tomlinson

Yasamy

M. T.

Emerson

Officer

Richler

Saxena

(2014). Setting global research priorities for developmental disabilities, including intellectual disabilities and autism. Journal of Intellectual Disability Research: Journal of Intellectual Disability Research, 58(12), 1121–1130. https://doi.org/10.1111/jir.12106

53.

Treisman

A. M.

Gelade

(1980). A feature-integration theory of attention. Cognitive Psychology, 12(1), 97–136. https://doi.org/10.1016/0010-0285(80)90005-5

54.

van Steensel

F. J.

Bögels

S. M.

Perrin

(2011). Anxiety disorders in children and adolescents with Autistic spectrum disorders: A meta-analysis. Clinical Child and Family Psychology Review, 14(3), 302–317. https://doi.org/10.1007/s10567-011-0097-0

55.

Wechsler

(1981). Wechsler adult intelligence scale (WAIS-R) manual. The Psychological Coro.

56.

Wild

C. J.

Nichols

E. S.

Battista

M. E.

Stojanoski

Owen

A. M.

(2018). Dissociable effects of self-reported daily sleep duration on high-level cognitive abilities. Sleep, 41(12), 1–11. https://doi.org/10.1093/sleep/zsy182

57.

Willcutt

E. G.

Doyle

A. E.

Nigg

J. T.

Faraone

S. V.

Pennington

B. F.

(2005). Validity of the Executive Function Theory of Attention-Deficit/Hyperactivity Disorder: A Meta-Analytic Review. Biological Psychiatry, 57(11), 1336–1346. https://doi.org/10.1016/j.biopsych.2005.02.006

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.26 MB