Abstract
Objective:
The National Institute of Mental Health has initiated the Research Domain Criteria (RDoC) project. Instead of using disorder categories as the basis for grouping individuals, the RDoC suggests finding relevant dimensions that can cut across traditional disorders. Our aim was to use the RDoC’s framework to study patterns of attention deficit based on results of Conners’ Continuous Performance Test (CPT II) in youths diagnosed with bipolar disorder (BD), attention-deficit/hyperactivity disorder (ADHD), BD+ADHD and controls.
Method:
Eighteen healthy controls, 23 patients with ADHD, 10 with BD and 33 BD+ADHD aged 12–17 years old were assessed. Pattern recognition was used to partition subjects into clusters based simultaneously on their performance in all CPT II variables. A Fisher’s linear discriminant analysis was used to build a classifier.
Results:
Using cluster analysis, the entire sample set was best clustered into two new groups, A and B, independently of the original diagnoses. ADHD and BD+ADHD were divided almost 50% in each subgroup, and there was an agglomeration of controls and BD in group B. Group A presented a greater impairment with higher means in all CPT II variables and lower Children’s Global Assessment Scale. We found a high cross-validated classification accuracy for groups A and B: 95.2%. Variability of response time was the strongest CPT II measure in the discriminative pattern between groups A and B.
Conclusion:
Our classificatory exercise supports the concept behind new approaches, such as the RDoC framework, for child and adolescent psychiatry. Our approach was able to define clinical subgroups that could be used in future pathophysiological and treatment studies.
Keywords
Introduction
Psychiatric research has faced challenging moments in the past few years discussing categorical and dimensional diagnostic paradigms. The contribution of the Diagnostic and Statistical Manual of Mental Disorders (DSM) and the International Classification of Diseases (ICD) in research and clinical progress is unquestionable. Nevertheless, their categories and criteria were formulated before our current knowledge of neuroscience and perhaps do not reflect the organization of brain circuits and their associated behaviours (Morris and Cuthbert, 2012). Categorical approach analysis implicitly involves the notion of a unitary entity; thus, not taking into account the acknowledged heterogeneity present in all clinical diagnoses and possibly control subjects (Fair et al., 2012; Morris and Cuthbert, 2012). High comorbidity rates also contribute to this debate, raising questions about the core features of a specific diagnosis (Sanislow et al., 2010). The categorical approach has not led to consistent biomarker findings. For this purpose, the National Institute of Mental Health (NIMH) has initiated the Research Domain Criteria (RDoC) project. The RDoC encourages a new research framework on mental disorders. Instead of using traditional disorder categories as the basis for grouping individuals, the RDoC suggests finding relevant dimensions that can cut across traditional disorders (NIMH, 2008). Thus, the approach changes the starting point of research from patients’ categorical diagnoses to other possibly different units of analysis (e.g. negative valence systems, cognitive systems), as independent variables (Morris and Cuthbert, 2012). The suggested starting point for studying comorbid disorders should be shared symptoms and behaviours, as opposed to two distinct diagnostic groups (Morris and Cuthbert, 2012).
This shift of paradigm may shed light on one of the strongest controversies in child and adolescent psychiatry, the high comorbidity rate between bipolar disorder (BD) and attention-deficit/hyperactivity disorder (ADHD), from 22% to 61% (Birmaher et al., 2006; Singh et al., 2006). The co-occurrence of these disorders may bewilder the process of diagnosis, treatment and prognosis of these children (Singh et al., 2006). This controversy is mainly due to several overlapping symptoms (Geller et al., 2002). Kowatch et al. (2005b) conducted a literature review and meta-analysis to study the phenomenology and clinical characteristics of BD in youths and found that their most common symptoms are also present in ADHD (Geller et al., 2002). ‘Distractibility’ is one of the most common symptoms and is used as criteria in categorical diagnosis for both disorders, making it a good candidate for an RDoC unit of analysis. Moreover, its presence can worsen the prognosis of BD children. The use of the gold standard treatment for inattention, stimulants, can induce switches to mania/hypomania in BD patients (Kowatch et al., 2005a). Therefore, for a better understanding of this symptom’s similarities and dissimilarities in patients with BD and ADHD, a different approach can be of great value in clinical and research settings.
Attention is not a single entity but a finite set of brain processes (Parasuraman, 2000). At least three components of attention can be described: selection, vigilance or sustained attention and control (Parasuraman, 2000). The Continuous Performance Test is a well-known test for this purpose, especially Conners’ Continuous Performance Test (CPT II) (Conners and Staff, 2000, 2004; Strauss et al., 2006). The paradigm used in CPT II when compared to other continuous performance tests yielded higher average weighted effect sizes for both diagnostic groups (Walshaw et al., 2010). Our aim was to study the patterns of attention deficit by means of the results obtained by the CPT II in children and adolescents with an original categorical diagnosis of BD, ADHD, BD+ADHD and controls, and to build a classifier to compare the classification accuracy of these newly formed groups and the original diagnostic ones.
Method
Subjects
Children and adolescents aged between 12 and 17 years old were assessed when enrolled at the BD Program or the ADHD Disorder Outpatient Clinic at the Institute of Psychiatry of the University of São Paulo Medical School, in São Paulo, Brazil and at the Juvenile BD Outpatient Program (ProCAB) or the ADHD Outpatient Program (Prodah), Division of Child and Adolescent Psychiatry, Hospital de Clínicas de Porto Alegre (HCPA), Federal University of Rio Grande do Sul, Brazil. Controls with no personal or family history of psychiatric illness were recruited via advertisements within the local community. The final sample consisted of: 18 healthy controls, 23 individuals with ADHD, 10 with BD and 33 with BD+ADHD. Exclusion criteria for all the participants were an intelligence quotient (IQ) lower than 70, substance use in the previous 6 months and diseases that affected the central nervous system. Both sites had approval from their Research Ethics Committee. The parents or legal guardians of all subjects gave written informed consent and assent was obtained from the children.
Measures
All clinical measures were administered by our research team and reviewed by Brazilian Board Certified Child Psychiatrists (AK, SCC, CPZ, ST).
Psychiatric disorders
Current and lifetime psychiatric disorders were assigned using the Schedule for Affective Disorders and Schizophrenia for School-Age Children – Present and Lifetime version (K-SADS-PL) (Kaufman et al., 1997) or the Schedule for Affective Disorders and Schizophrenia for School-Age Children – Epidemiological version (K-SADS-E) (Orvaschel, 1985) for all participants (patients and controls). Patients were diagnosed with BD-NOS if they presented episodes that were clearly manic or hypomanic yet lacked the necessary duration to meet the criteria for diagnosis of BD I or BD II (Birmaher et al., 2006). Comorbid diagnoses with ADHD were assigned based on symptoms present during euthymia. IQs were assessed with the Wechsler Abbreviated Scale of Intelligence (Wechsler, 1999). Socioeconomic status was assessed with the Brazilian Association of Market Research Institutes Demographic and Socioeconomic Class Scale (Januzzi and Baeninger, 1996).
Symptom severity
The Children’s Global Assessment Scale (CGAS) was used to evaluate global severity based on psychiatric symptoms and functional impairment (Shaffer et al., 1983). To assess the severity of depression and mania, we applied the Children’s Depression Rating Scale – Revised (CDRS-R) (Poznanski et al., 1985) and the Young Mania Rating Scale (YMRS) (Young et al., 1978), respectively. The Multimodal Treatment of Attention Deficit Hyperactivity Disorder (MTA) Swanson, Nolan and Pelham scale, version IV (SNAP-IV) was completed by the parent. This is a 26-item SNAP-IV version that assesses ADHD core symptoms of hyperactivity/impulsivity and inattention, along with symptoms of oppositional defiant disorder (Swanson et al., 2001).
Attention performance
Conners’ Continuous Performance Test Computer Program for Windows (CPT II; Conners and Staff, 2000, 2004) was used to assess attention characteristics. The CPT II is the most popular commercial version (Conners and Staff, 2000, 2004; Strauss et al., 2006). From 15 studies that evaluated attention in children and adolescents with BD, 14 used a continuous performance test (Biederman et al., 2011; Brotman et al., 2009; Doyle et al., 2009; Mattis et al., 2011; Pavuluri et al., 2006a; Udal et al., 2013; Walshaw et al., 2010). This test is also widely used in ADHD research (Egeland and Kovalik-Gran, 2010; Miranda et al., 2012). Patients were off stimulants at least 24 hours prior to the assessment. Participants were required to depress the computer’s space bar when any letter, except the letter ‘X’, appeared on the screen. It is a14-minute task divided into six consecutive time blocks with three sub-blocks each. Each sub-block has a different interstimulus interval (ISI) (1, 2 or 4s). The CPT II generates 12 multiple dependent measures, described in Table 1. Measures are T-scores that compare a subject’s responses with those in the normative group who are of the same sex and age group. A T-score of 50 represents the average of the comparison group, and each 10 points denotes one standard deviation. For more details refer to the CPT II software manual (Conners and Staff, 2000, 2004).
The 12 CPT II variables: description and definition.
Data analysis
All statistical analyses were performed using the Statistical Package for Social Sciences, version 14.0 (SPSS Inc., Chicago, IL, USA). Analysis of variance (ANOVA) with Tukey’s post-hoc analysis was used for quantitative variables. Fisher’s exact test was used in order to analyse categorical variables. The normality of continuous data was verified.
K-means clustering was used to partition subjects into clusters based simultaneously on their performance in the 12 CPT II variables independent of each individual’s original diagnosis. The K-means method simultaneously used the 12 CPT II variables subjects’ means to cluster them into two to 10 different groups. All 12 CPT II results were available for all participants. We then used the silhouette method in order to assess how many new groups the individuals were best classified in, based on the homogeneity of those groups found (Rousseeuw, 1987).
A Fisher’s linear discriminant analysis was used to build a classifier based on the CPT II variables. The linear function took into account all 12 variables. The best CPT II variable predictors were found by correlation among CPT II measures and the discriminant functions. The accuracy was assessed in these new clusters and also in the original diagnostic groups. For quality purposes the accuracy was measured with the leave-one-out method to assess the generalization of our findings. The level of significance was set at 5%.
Results
Sociodemographic and clinical characteristics
The sociodemographic and clinical characteristics are described in Tables 2 and 3. There were no statistically significant differences in any of the clinical or sociodemographic variables when comparing individuals from both sites.
Sociodemographic characteristics of the sample groups.
ANOVA.
Fisher’s exact test.
BD: bipolar disorder; ADHD: attention deficit/hyperactivity disorder; SD: standard deviation.
Clinical variables of the sample groups.
ANOVA.
Fisher’s exact test.
BD vs BD+ADHD.
ADHD vs BD+ADHD.
Patients were off stimulants 24 hours prior to assessment.
BD vs BD+ADHD and BD vs ADHD.
BD: bipolar disorder; ADHD: attention deficit/hyperactivity disorder; NOS: not otherwise specified; CGAS: Children’s Global Assessment Scale; SD: standard deviation; SNAP-IV: MTA Swanson, Nolan and Pelham scale – version IV; CDRS-R: Children’s Depression Rating Scale – Revised; YMRS: Young Mania Rating Scale.
K-means clustering
All individuals, independent of their original diagnosis, were clustered into new groups based on the 12 CPT II variables’ performance. The silhouette method showed that these subjects were best clustered into two new groups: A and B (silhouette was verified for two to 10 groups). Table 4 describes the diagnoses distributions of these new groups. Table 5 describes the sociodemographic and clinical data of the new groups.
Subjects’ distribution in groups A and B based on performance of the CPT II measures.
BD: bipolar disorder; ADHD: attention deficit/hyperactivity disorder.
Sociodemographic and clinical variables of the new groups A and B.
t-test.
Fisher’s exact test.
Patients were off stimulants 24 hours prior to assessment.
SD: standard deviation; BD: bipolar disorder; ADHD: attention deficit/hyperactivity disorder; NOS: not otherwise specified; CGAS: Children’s Global Assessment Scale; SNAP-IV: MTA Swanson, Nolan and Pelham scale – version IV; CDRS-R: Children’s Depression Rating Scale – Revised; YMRS: Young Mania Rating Scale.
CPT II variable results
There were no statistically significant differences in any of the CPT II measures when comparing controls, BD, ADHD and BD+ADHD (Figure 1). However, when we compared the CPT II variables’ performance between the newly clustered groups A and B we found that, after correcting for multiple comparisons, eight out of the 12 CPT II measures were statistically significant: omission (p=0.0003), commission (p=0.00000002), hit reaction time (RT) standard error (SE) (p=1.7×10−20), variability of SE (p=4.3×10−22), detectability (p=0.000008), perseveration (p=0.0000001), hit RT by interstimulus interval (ISI) (p=4.7×10−10) and hit RT SE ISI (p=1.5×10−13). The mean score of CPT II measures are illustrated in Figure 1 for the four originally diagnosed groups, and in Figure 2 for groups A and B. CPT II performance was not associated with sex, mood state or presence of psychosis using a t-test or with CGAS scores and SNAP means using a Pearson correlation.
Mean T-score of CPT II measures for controls, BD, ADHD and BD+ADHD. The red line is set at a T-score of 50, which represents the average of the comparison normative group. (For abbreviations, see Table 1.)
Mean T-score of CPT II measures for groups A and B. Asterisks indicate significant group differences after correction for multiple comparisons and the red line is set at a T-score of 50, which represents the average of the comparison normative group. (For abbreviations, see Table 1.)
Linear discriminant analysis
In order to generalize our data and verify the quality of our findings, we built a classifier based on the CPT II performance. We found a high cross-validated classification accuracy for groups A and B (95.2%), confirming the consistency of these new clusters. The strongest CPT II variables in the discriminative pattern were: variability of standard error ranking first (0.776), followed by hit RT SE (0.726), hit RT SE ISI (0.607), hit RT ISI (0.486), perseveration (0.460) and commission (0.366); numbers indicate correlations between the variable and the discriminant function. The cross-validated classification accuracy based on the CPT II measures’ performance when used to classify subjects in the four original groups was much lower (23.8%).
Discussion
The entire sample set was best clustered into two new groups based on the 12 CPT II variables’ performance, independently of their original DSM-IV diagnoses. Of note, the four original categorical groups (BD, ADHD, BD+ADHD, controls) were present in both new assemblies. There was an agglomeration of control and BD subjects in group B. In contrast, ADHD and BD+ADHD subjects were divided almost 50% in each subgroup. This finding is consistent with the ADHD literature, since only 50% of ADHD subjects presented neurocognitive deficits measured by neuropsychological tasks (Nigg et al., 2005). In this sense, one should not conclude that good performance on neuropsychological tests means that there are no real attention problems. Laboratory assessments are less sensitive than evaluating the individual’s performance in the various domains of daily life (Brown, 2006).
We found no significant differences when we compared CPT II performance between BD, ADHD, BD+ADHD and controls; however, when we compared the newly clustered groups A and B, we found that eight out of 12 of the CPT II measures were statistically significant. Group A presented deficits in sustained attention and inhibitory control, higher levels of impulsiveness and poor vigilance. Moreover, participants in group A were alerted and prepared for immediate response when a stimulus was presented at a high rate, but did not sustain this alertness with longer rates. The newly clustered subjects in group A also presented a greater impairment exhibited by higher means in all CPT II variables and SNAP-IV, and lower CGAS scores. That is, deficits measured by this neuropsychological test were associated with functional impairment regarding the diagnoses. This is in agreement with Biederman et al., who reported worse overall functioning in ADHD children with executive function impairments when compared to ADHD children without these neuropsychological deficits (Biederman et al., 2004). Functional impairment, independent of categorical diagnoses, should be considered an important clinical manifestation and treatment target of attentional difficulties in adolescents.
These results also highlight the heterogeneity of CPT II responses among each diagnostic group. This heterogeneity may explain in part the controversy found in literature when comparing the neuropsychological performance between individuals with BD, ADHD, BD+ADHD and controls. Only four other research studies assessed attention performance concomitantly in BD and ADHD patients and the results were non-conclusive. Two studies compared the same four groups as this study did. Rucklidge et al. (2006) assessed youths aged 14 to 17 years old and showed results that were similar to our findings in that BD subjects performed similar to controls, and patients with BD+ADHD and ADHD performed worse. Udal et al. (2013) assessed children and adolescents aged 6 to 19 years old. They described only one CPT II variable (commission), and found no significant differences among the four groups (same result as Rucklidge’s and ours). Henin et al. (2007) compared subjects aged 6 to 17 years old with BD, BD+ADHD and controls and found no differences between the groups. Mattis et al. (2011) studied medicated patients, aged 8 to 17 years old, with BD, ADHD and BD+ADHD. They found that individuals with BD and BD+ADHD exhibited worse performance than patients with ADHD, suggesting that medication could improve impairment in ADHD but not in children with affective disorders with or without ADHD (Mattis et al., 2011). Some studies did secondary analysis to assess the impact of comorbid ADHD in attentional performance in BD patients. But the results are also conflicting. Pavuluri et al. (2006b) found a worse performance in patients with comorbid ADHD, while Biederman et al. (2011) did not find this difference. These result disparities could in part also be explained by the methodological difference between these studies regarding age inclusion criteria, since robust age-related effects have been found in epidemiologic studies when assessing attention symptomatology (Conners et al., 2003; Miranda et al., 2008, 2013).
The high cross-validated accuracy found in the newly clustered groups when compared to the low accuracy found in original DSM-IV diagnostic groups, based on the CPT II variables’ results, also emphasizes diagnostic group heterogeneity, at least regarding their attentional deficits (95.2% and 23.8%, respectively). Our results are consistent with Fair et al. (2012) who questioned the potential heterogeneity of patients clustered by categorical classification and also the less tangible heterogeneity in typically developing children. They assessed children with ADHD and controls through an extensive neuropsychological battery, and, using a graph theory and community detection approach, they found that ADHD subjects could be divided into six new subgroups based on the test results, whereas controls were into four new subgroups (Fair et al., 2012). These results are consistent with ours, highlighting the heterogeneity found not only within the ADHD group but also within the typically developing children group.
Interestingly, the three variables that most influenced the newly clustered groups were the ones that measure variability of response time (variability of SE ranking first, hit RT SE and hit RT SE ISI). Response time variability is one of the most replicated deficits and is considered a core feature of ADHD (Castellanos and Tannock, 2002; Epstein et al., 2011; Klein et al., 2006). This characteristic has been replicated across a variety of reaction time tasks, laboratories and cultures (Castellanos and Tannock, 2002; Klein et al., 2006). This characteristic has been observed in ADHD relatives (Bidwell et al., 2007; Kuntsi et al., 2010, 2013), persisted even in clinically remitted young adults (Halperin et al., 2008) and is associated with significant impairment (Flory et al., 2006). An understanding of the cause of RT variability in children with ADHD is still uncertain. Several theories have been proposed such as insufficient suppression of the default mode network (Sonuga-Barke and Castellanos, 2007), subcortical mediated problems in state regulation (Sergeant et al., 2003), deficit in motor timing (Rubia et al., 2001), among others. Increased intra-subject variability in RT was also associated with BD (Bora et al., 2006; Brotman et al., 2009; Mattis et al., 2011). Brotman et al. (2009) suggested that intra-subject variability could be a non-specific endophenotypic marker for psychopathology.
Our results are also consistent with Miranda et al. (2012). They used the CPT II to compare ADHD patients with a CPT II Brazilian standardized sample and found high effect sizes in the same variables we did to build this classifier (Miranda et al., 2012).
This new research approach highlights the heterogeneity within groups divided by categorical diagnoses and healthy subjects. It contributes to a better understanding of why it has been so difficult to identify unique biological mechanisms for each psychiatric disorder.
Categorical classification has persisted as a descriptive taxonomy based on expressed feelings and observed behaviour and probably does not encompass pathophysiological mechanisms. Brain circuits possibly do not respect the arbitrary limits imposed by categorical classification. Kapur et al. (2012) emphasized this challenging moment for psychiatry. While the current diagnostic classification was not intended to assist pathophysiological mechanisms, biological studies, to date, have not been able to propose a clinically viable alternative system (Kapur et al., 2012). The suggestion is to find cognitive tests and biological markers that could subdivide individuals from the same categorical diagnostic group to tailor the treatment (Kapur et al., 2012).
Our newly clustered groups could contribute to build a more homogenous group facilitating the study of pathophysiology and tailoring treatment. In our study individuals in group A presented a more severe functional impairment translated by lower CGAS and higher SNAP IV means, and were best differentiated by higher intra-subject variability in response time. This approach could enable a customized psychiatric approach to treat specific patients like the ones clustered in group A. First, we would have to define useful markers as cognitive tests that could improve our understanding of the individual’s difficulties, regardless of his/her diagnosis. After defining these homogenous groups, independent of categorical diagnosis, we could not only study pathophysiology, but also research useful treatments specific for that group’s difficulties.
The limitations of our study include the use of only one test to assess such a complex construct as attention and a relatively small sample size. Additional restraints specific from the CPT II are the presence of some clinical cases in the normative sample (Conners and Staff, 2000, 2004). In addition, our results cannot be generalized for children under 12 years old. Only youths aged 12 or older were included to minimize possible confounders, since robust age-related effects have been found in epidemiologic attention studies (Conners et al., 2003; Miranda et al., 2008). The 12-year-old cut-off was based on attention (Conners et al., 2003; Miranda et al., 2008) and structural and functional brain development studies (Vaidya, 2012). Medication use by youths with BD and BD+ADHD, and the CPT II assessments in different mood-states are also limitations that need to be considered.
In summary, our findings highlight the heterogeneity of patients clustered by categorical diagnostic classification. In addition, our classificatory exercise supports the concept behind new approaches such as the RDoC framework for children and adolescent psychiatry. It can define meaningful clinical subgroups for the purpose of pathophysiological studies and treatment selection, and provide a pathway by which research findings can be translated into changes in clinical decision-making (Morris and Cuthbert, 2012).
Footnotes
Acknowledgements
The authors would like to thank Ramiro Borges Rodrigues, from the Hospital de Clínicas de Porto Alegre, Federal University of Rio Grande do Sul, Brasil; and Silvia Leticia dos Santos and Alessandra Bizeli Oliveira Sartori from the Bipolar Research Program, Department of Psychiatry, University of São Paulo Medical School, São Paulo, Brazil for data collection and scoring.
Funding
This research was supported in part by the Fundação de Amparo a Pesquisa de São Paulo, Brazil (FAPESP); the Conselho Nacional de Desenvolvimento Cientifico e Tecnológico, Brazil (CNPq); the National Alliance for Research on Schizophrenia and Depression (NARSAD), Young Investigator; the American Psychiatric Association/AstraZeneca Young Minds in Psychiatry International Awards and the L’Oréal, ABC and Unesco – Para Mulheres na Ciência, L’Oréal, ABC e Unesco, Brazil.
Declaration of interest
Ana Kleinman has received scholarship from CNPq. Dr Zeni received travel support for scientific meetings from Shire Pharmaceuticals. Dr Rohde has been on the speakers’ bureau/advisory board and/or has acted as a consultant for Eli Lilly, Janssen-Cilag, Novartis and Shire Pharmaceuticals in the last 3 years. He receives authorship royalties from Oxford University Press and ArtMed. He also received travel awards from Shire Pharmaceuticals for taking part in the 2014 APA meeting. The ADHD and Juvenile Bipolar Disorder Outpatient Programs chaired by him received unrestricted educational and research support from the following pharmaceutical companies in the last 3 years: Eli Lilly, Janssen-Cilag, Novartis, and Shire Pharmaceuticals. The other authors have no conflict of interest to report.