Traditional diagnostic systems for neurodevelopmental disorders define diagnostic categories that are heterogeneous in behavior and underlying neurobiological alterations. The goal of this study was to parse heterogeneity in a core executive function (EF), cognitive flexibility, in children with a range of abilities (N = 132; children with autism spectrum disorder, attention-deficit/hyperactivity disorder [ADHD], and typically developing children) using directed functional connectivity profiles derived from resting-state functional magnetic resonance imaging data. Brain regions activated in response to a cognitive flexibility task in adults were used to guide region-of-interest selection to estimate individual connectivity profiles in this study. We expected to find subgroups of children who differed in their network connectivity metrics and symptom measures. Unexpectedly, we did not find a stable or valid subgrouping solution, which suggests that categorical models of the neural substrates of cognitive flexibility in children may be invalid. Exploratory analyses revealed dimensional associations between network connectivity metrics and ADHD symptomatology and EF ability across the entire sample. Results shed light on the validity of conceptualizing the neural substrates of cognitive flexibility categorically in children. Ultimately, this work may provide a foundation for the development of a revised nosology focused on neurobiological substrates as an alternative to traditional symptom-based classification systems.
Get full access to this article
View all access options for this article.
References
1.
AchenbachT.1991. Integrative guide for the 1991 CBCL/4–18, YSR, and TRF profiles. Burlington, VT: Department of Psychiatry, University of Vermont.
2.
AchenbachT, RescorlaL. 2001. Manual for the ASEBA school-age forms & profiles: an integrated system of multi-informant assessment. Burlington, VT: University of Vermont. Research Center for Children, Youth, & Families.
3.
American Psychiatric Association. 2013. DSM 5. Washington, DC: American Psychiatric Association.
4.
AndersonJS, DruzgalTJ, FroehlichA, DuBrayMB, LangeN, AlexanderAL, et al.2011. Decreased interhemispheric functional connectivity in autism. Cereb Cortex, 21:1134–1146.
5.
AndersonV, JacobsR, AndersonPJ. 2008. Executive Functions and the Frontal Lobes: A Lifespan Perspective: Philadelphia, PA: Taylor & Francis.
6.
BassettDS, XiaCH, SatterthwaiteTD. 2018. Understanding the emergence of neuropsychiatric disorders with network neuroscience. Biol Psychiatry Cogn Neurosci Neuroimaging, 3:742–753.
7.
BodfishJW, SymonsFJ, ParkerDE, LewisMH. 2000. Varieties of repetitive behavior in autism: comparisons to mental retardation. J Autism Dev Disord, 30:237–243.
8.
BrownTA.2006. Confirmatory Factor Analysis for Applied Research. New York, NY: Guilford Press.
9.
BullR, EspyKA, WiebeSA. 2008. Short-term memory, working memory, and executive functioning in preschoolers: longitudinal predictors of mathematical achievement at age 7 years. Dev Neuropsychol, 33:205–228.
10.
ChabernaudC, MennesM, KellyC, NoonerK, Di MartinoA, CastellanosFX, MilhamMP. 2012. Dimensional brain-behavior relationships in children with attention-deficit/hyperactivity disorder. Biol Psychiatry, 71:434–442.
11.
CoghillD, Sonuga-BarkeEJ. 2012. Annual research review: categories versus dimensions in the classification and conceptualisation of child and adolescent mental disorders—implications of recent empirical study. J Child Psychol Psychiatry, 53:469–489.
ConnersCK.2008. Conners 3rd Edition. Ontario, Canada: Multi-Health Systems Toronto.
14.
Costa DiasTG, IyerSP, CarpenterSD, CaryRP, WilsonVB, MitchellSH, et al.2015. Characterizing heterogeneity in children with and without ADHD based on reward system connectivity. Dev Cogn Neurosci, 11:155–174.
15.
CsardiG, NepuszT. 2006. The igraph software package for complex network research. http://igraph.org Last accessed Nov.7, 2019.
16.
CuthbertBN.2014. The RDoC framework: facilitating transition from ICD/DSM to dimensional approaches that integrate neuroscience and psychopathology. World Psychiatry, 13:28–35.
17.
CuthbertBN, InselTR. 2013. Toward the future of psychiatric diagnosis: the seven pillars of RDoC. BMC Med, 11:126.
DajaniDR, BurrowsCA, OdriozolaP, BaezA, NebelMB, MostofskySH, UddinL.Q. 2019. Investigating functional brain network integrity using a traditional and novel categorical scheme for neurodevelopmental disorders. Neuroimage Clin, 21:101678.
20.
DajaniDR, LlabreMM, NebelMB, MostofskySH, UddinLQ. 2016. Heterogeneity of executive functions among comorbid neurodevelopmental disorders. Sci Rep, 6:36566.
21.
DajaniDR, UddinLQ. 2015. Demystifying cognitive flexibility: implications for clinical and developmental neuroscience. Trends Neurosci, 38:571–578.
22.
DennisEL, JahanshadN, McMahonKL, de ZubicarayGI, MartinNG, HickieIB, et al.2014. Development of insula connectivity between ages 12 and 30 revealed by high angular resolution diffusion imaging. Hum Brain Mapp, 35:1790–1800.
23.
Di MartinoA, ZuoXN, KellyC, GrzadzinskiR, MennesM, SchvarczA, et al.2013. Shared and distinct intrinsic functional network centrality in autism and attention-deficit/hyperactivity disorder. Biol Psychiatry, 74:623–632.
24.
DiamondA, LeeK. 2011. Interventions shown to aid executive function development in children 4 to 12 years old. Science, 333:959–964.
25.
DickAS.2014. The development of cognitive flexibility beyond the preschool period: an investigation using a modified flexible item selection task. J Exp Child Psychol, 125:13–34.
26.
DickieEW, AmeisSH, ShahabS, CalarcoN, SmithDE, MirandaD, et al.2018. Personalized intrinsic network topography mapping and functional connectivity deficits in autism spectrum disorder. Biol Psychiatry, 84:278–286.
27.
DuPaulGJ, PowerTJ, AnastopoulosAD, ReidR. 1998. ADHD Rating Scale-IV: Checklists, Norms, and Clinical Interpretation, Vol. 25. New York: Guilford Press.
28.
EltonA, AlcauterS, GaoW. 2014. Network connectivity abnormality profile supports a categorical-dimensional hybrid model of ADHD. Hum Brain Mapp, 35:4531–4543.
29.
EltonA, Di MartinoA, HazlettHC, GaoW. 2016. Neural connectivity evidence for a categorical-dimensional hybrid model of autism spectrum disorder. Biol Psychiatry, 80:120–128.
30.
FairDA, BathulaD, NikolasMA, NiggJT. 2012a. Distinct neuropsychological subgroups in typically developing youth inform heterogeneity in children with ADHD. Proc Natl Acad Sci U S A, 109:6769–6774.
31.
FairDA, NiggJT, IyerS, BathulaD, MillsKL, DosenbachNU, et al.2012b. Distinct neural signatures detected for ADHD subtypes after controlling for micro-movements in resting state functional connectivity MRI data. Front Syst Neurosci, 6:80.
32.
FarrantBM, MayberyMT, FletcherJ. 2012. Language, cognitive flexibility, and explicit false belief understanding: longitudinal analysis in typical development and specific language impairment. Child Dev, 83:223–235.
33.
FeczkoE, BalbaN, Miranda-DominguezO, CordovaM, KaralunasSL, IrwinL, et al.2018. Subtyping cognitive profiles in autism spectrum disorder using a functional random forest algorithm. Neuroimage, 172:674–688.
34.
FischerDB, BoesAD, DemertziA, EvrardHC, LaureysS, EdlowBL, et al.2016. A human brain network derived from coma-causing brainstem lesions. Neurology, 87:2427–2434.
35.
FrazierTW, YoungstromEA, NaugleRI. 2007. The latent structure of attention-deficit/hyperactivity disorder in a clinic-referred sample. Neuropsychology, 21:45–64.
36.
FriedmanNP, MiyakeA. 2017. Unity and diversity of executive functions: individual differences as a window on cognitive structure. Cortex, 86:186–204.
37.
GaoW, AlcauterS, SmithJK, GilmoreJH, LinW. 2015. Development of human brain cortical network architecture during infancy. Brain Struct Funct, 220:1173–1186.
38.
GarveyM, AvenevoliS, AndersonK. 2016. The National Institute of Mental Health Research Domain Criteria and Clinical Research in child and adolescent psychiatry. J Am Acad Child Adolesc Psychiatry, 55:93–98.
39.
GatesKM, FisherZ, ArizmendiC, HenryTR, DuffyK, MuchaPJ. 2019. Assessing the robustness of cluster solutions obtained from sparse count matrices. Psychol Methods [Epub ahead of print]; DOI: 10.1037/met0000204.
40.
GatesKM, LaneST, VarangisE, GiovanelloK, GuiskewiczK. 2017. Unsupervised classification during time-series model building. Multivariate Behav Res, 52:129–148.
41.
GatesKM, MolenaarPC. 2012. Group search algorithm recovers effective connectivity maps for individuals in homogeneous and heterogeneous samples. Neuroimage, 63:310–319.
42.
GatesKM, MolenaarPC, HillaryFG, SlobounovS. 2011. Extended unified SEM approach for modeling event-related fMRI data. Neuroimage, 54:1151–1158.
43.
GatesKM, MolenaarPC, IyerSP, NiggJT, FairDA. 2014. Organizing heterogeneous samples using community detection of GIMME-derived resting state functional networks. PLoS One, 9:e91322.
44.
GeorgiadesS, SzatmariP, BoyleM, HannaS, DukuE, ZwaigenbaumL, et al.2013. Investigating phenotypic heterogeneity in children with autism spectrum disorder: a factor mixture modeling approach. J Child Psychol Psychiatry, 54:206–215.
45.
GhirardiL, BrikellI, Kuja-HalkolaR, FreitagCM, FrankeB, AshersonP, et al.2018. The familial co-aggregation of ASD and ADHD: a register-based cohort study. Mol Psychiatry, 23:257–262.
GioiaGA, IsquithPK, RetzlaffPD, EspyKA. 2002. Confirmatory factor analysis of the Behavior Rating Inventory of Executive Function (BRIEF) in a clinical sample. Child Neuropsychol, 8:249–257.
48.
GrangerCWJ.1969. Investigating causal relations by econometric models and cross-spectral methods. Econometrica, 37:424–438.
49.
GuimeraR, AmaralLAN. 2005. Cartography of complex networks: modules and universal roles. J Stat Mech, 2005:P02001.
50.
HaslamN, WilliamsB, PriorM, HaslamR, GraetzB, SawyerM. 2006. The latent structure of attention-deficit/hyperactivity disorder: a taxometric analysis. Aust N Z J Psychiatry, 40:639–647.
51.
HenryTR, FeczkoE, CordovaM, EarlE, WilliamsS, NiggJT, et al. M. 2019. Comparing directed functional connectivity between groups with confirmatory subgrouping GIMME. Neuroimage, 188:642–653.
52.
HolmesAJ, PatrickLM. 2018. The myth of optimality in clinical neuroscience. Trends Cogn Sci, 22:241–257.
53.
Huang-PollockC, ShapiroZ, Galloway-LongH, WeigardA. 2017. Is poor working memory a transdiagnostic risk factor for psychopathology?. J Abnorm Child Psychol, 45:1477–1490.
54.
InselT, CuthbertB, GarveyM, HeinssenR, PineDS, QuinnK, et al.2010. Research domain criteria (RDoC): toward a new classification framework for research on mental disorders. Am J Psychiatry, 167:748–751.
55.
KenworthyL, AnthonyLG, NaimanDQ, CannonL, WillsMC, Luong-TranC, et al.2014. Randomized controlled effectiveness trial of executive function intervention for children on the autism spectrum. J Child Psychol Psychiatry, 55:374–383.
56.
KimC, CillesSE, JohnsonNF, GoldBT. 2012. Domain general and domain preferential brain regions associated with different types of task switching: a meta-analysis. Hum Brain Mapp, 33:130–142.
57.
KimJ, ZhuW, ChangL, BentlerPM, ErnstT. 2007. Unified structural equation modeling approach for the analysis of multisubject, multivariate functional MRI data. Hum Brain Mapp, 28:85–93.
58.
LaheyBB, KruegerRF, RathouzPJ, WaldmanID, ZaldDH. 2017. A hierarchical causal taxonomy of psychopathology across the life span. Psychol Bull, 143:42–186.
59.
LaneST, GatesKM. 2017. Automated selection of robust individual-level structural equation models for time series data. Struct Equ Modeling, 24:768–782.
60.
LeitnerY.2014. The co-occurrence of autism and attention deficit hyperactivity disorder in children—what do we know?. Front Hum Neurosci, 8:268.
61.
LenrootRK, YeungPK. 2013. Heterogeneity within Autism Spectrum Disorders: what have we learned from neuroimaging studies?. Front Hum Neurosci, 7; 733.
62.
LicinioJ, WongML. 2013. A novel conceptual framework for psychiatry: vertically and horizontally integrated approaches to redundancy and pleiotropism that co-exist with a classification of symptom clusters based on DSM-5. Mol Psychiatry, 18:846–848.
63.
LiebRW, BohnertAM. 2017. Relations between executive functions, social impairment, and friendship quality on adjustment among high functioning youth with autism spectrum disorder. J Autism Dev Disord, 47:2861–2872.
64.
LopezBR, LincolnAJ, OzonoffS, LaiZ. 2005. Examining the relationship between executive functions and restricted, repetitive symptoms of Autistic Disorder. J Autism Dev Disord, 35:445–460.
65.
LordC, RisiS, LambrechtL, Cook JrEH, LeventhalBL, DiLavorePC, et al.2000. The Autism Diagnostic Observation Schedule—Generic: a standard measure of social and communication deficits associated with the spectrum of autism. J Autism Dev Disord, 30:205–223.
66.
LordC, RutterM, DiLavorePC, RisiS, GothamK, BishopSL. 2012. Autism diagnostic observation schedule: ADOS-2. Los Angeles, CA: Western Psychological Services.
67.
LordC, RutterM, Le CouteurA. 1994. Autism Diagnostic Interview-Revised: a revised version of a diagnostic interview for caregivers of individuals with possible pervasive developmental disorders. J Autism Dev Disord, 24:659–685.
68.
LothE, SpoorenW, HamLM, IsaacMB, Auriche-BenichouC, BanaschewskiT, et al.2016. Identification and validation of biomarkers for autism spectrum disorders. Nat Rev Drug Discov, 15:70.
69.
LubkeGH, HudziakJJ, DerksEM, van BijsterveldtTC, BoomsmaDI. 2009. Maternal ratings of attention problems in ADHD: evidence for the existence of a continuum. J Am Acad Child Adolesc Psychiatry, 48:1085–1093.
70.
MarcusDK, BarryTD. 2011. Does attention-deficit/hyperactivity disorder have a dimensional latent structure? A taxometric analysis. J Abnorm Psychol, 120:427–442.
71.
MarekS, HwangK, ForanW, HallquistMN, LunaB. 2015. The contribution of network organization and integration to the development of cognitive control. PLoS Biol, 13:e1002328.
72.
MartelMM, PanPM, HoffmannMS, GadelhaA, do RosarioMC, MariJJ, et al.2017. A general psychopathology factor (P factor) in children: structural model analysis and external validation through familial risk and child global executive function. J Abnorm Psychol, 126:137–148.
73.
MartelMM., RobertsB, GremillionM, Von EyeA, NiggJT.2011. External validation of bifactor model of ADHD: explaining heterogeneity in psychiatric comorbidity, cognitive control, and personality trait profiles within DSM-IV ADHD. J Abnorm Child Psychol, 39:1111.
74.
MazefskyCA, AndersonR, ConnerCM, MinshewN. 2011. Child Behavior Checklist Scores for school-aged children with autism: preliminary evidence of patterns suggesting the need for referral. J Psychopathol Behav Assess, 33:31–37.
75.
McTeagueLM, GoodkindMS, EtkinA. 2016. Transdiagnostic impairment of cognitive control in mental illness. J Psychiatr Res, 83:37–46.
76.
MillerHL, RagozzinoME, CookEH, SweeneyJA, MosconiMW. 2015. Cognitive set shifting deficits and their relationship to repetitive behaviors in autism spectrum disorder. J Autism Dev Disord, 45:805–815.
77.
MorganSE, WhiteSR, BullmoreET, VertesPE. 2018. A network neuroscience approach to typical and atypical brain development. Biol Psychiatry Cogn Neurosci Neuroimaging, 3:754–766.
78.
NiendamTA, LairdAR, RayKL, DeanYM, GlahnDC, CarterCS. 2012. Meta-analytic evidence for a superordinate cognitive control network subserving diverse executive functions. Cogn Affect Behav Neurosci, 12:241–268.
79.
NomiJS, UddinLQ. 2015. Developmental changes in large-scale network connectivity in autism. Neuroimage Clin, 7:732–741.
80.
PessoaL.2014. Understanding brain networks and brain organization. Phys Life Rev, 11:400–435.
81.
PowerJD, BarnesKA, SnyderAZ, SchlaggarBL, PetersenSE. 2012. Spurious but systematic correlations in functional connectivity MRI networks arise from subject motion. Neuroimage, 59:2142–2154.
82.
PowerJD, CohenAL, NelsonSM, WigGS, BarnesKA, ChurchJA, et al.2011. Functional network organization of the human brain. Neuron, 72:665–678.
83.
PruimRH, MennesM, van RooijD, LleraA, BuitelaarJK, BeckmannCF. 2015. ICA-AROMA: a robust ICA-based strategy for removing motion artifacts from fMRI data. Neuroimage, 112:267–277.
84.
RaeCL, NombelaC, RodriguezPV, YeZ, HughesLE, JonesPS, et al.2016. Atomoxetine restores the response inhibition network in Parkinson's disease. Brain, 139(Pt 8):2235–2248.
85.
ReichW, WelnerZ, HerjanicB. 1997. Diagnostic interview for children and adolescents-IV (DICA-IV). Toronto, Canada: Multi-Health Systems.
86.
RobertsBA, MartelMM, NiggJT. 2017. Are there executive dysfunction subtypes within ADHD?. J Atten Disord, 21:284–293.
87.
RommelseNN, FrankeB, GeurtsHM, HartmanCA, BuitelaarJK. 2010. Shared heritability of attention-deficit/hyperactivity disorder and autism spectrum disorder. Eur Child Adolesc Psychiatry, 19:281–295.
88.
RommelseNN, GeurtsHM, FrankeB, BuitelaarJK, HartmanCA. 2011. A review on cognitive and brain endophenotypes that may be common in autism spectrum disorder and attention-deficit/hyperactivity disorder and facilitate the search for pleiotropic genes. Neurosci Biobehav Rev, 35:1363–1396.
89.
RommelseNNJ, van der MeerJMJ, HartmanCA, BuitelaarJK. 2016. Cognitive profiling useful for unraveling cross-disorder mechanisms. Clin Psychol Sci, 4:957–970.
90.
RosenthalM, WallaceGL, LawsonR, WillsMC, DixonE, YerysBE, KenworthyL. 2013. Impairments in real-world executive function increase from childhood to adolescence in autism spectrum disorders. Neuropsychology, 27:13.
91.
SatterthwaiteTD, WolfDH, ErusG, RuparelK, ElliottMA, GennatasED, et al.2013. Functional maturation of the executive system during adolescence. J Neurosci, 33:16249–16261.
92.
SatterthwaiteTD, WolfDH, RuparelK, ErusG, ElliottMA, EickhoffSB, et al.2013. Heterogeneous impact of motion on fundamental patterns of developmental changes in functional connectivity during youth. Neuroimage, 83:45–57.
93.
ScottWA.1962. Cognitive complexity and cognitive flexibility. Sociometry, 25:405–414.
94.
SeeleyWW, MenonV, SchatzbergAF, KellerJ, GloverGH, KennaH, et al.2007. Dissociable intrinsic connectivity networks for salience processing and executive control. J Neurosci, 27:2349–2356.
95.
ShanmuganS, WolfDH, CalkinsME, MooreTM, RuparelK, HopsonRD, et al.2016. Common and dissociable mechanisms of executive system dysfunction across psychiatric disorders in youth. Am J Psychiatry, 173:517–526.
96.
SikoraDM, VoraP, CouryDL, RosenbergD. 2012. Attention-deficit/hyperactivity disorder symptoms, adaptive functioning, and quality of life in children with autism spectrum disorder. Pediatrics, 130Suppl 2:S91–S97.
97.
SmithSM, MillerKL, Salimi-KhorshidiG, WebsterM, BeckmannCF, NicholsTE, et al.2011. Network modelling methods for FMRI. Neuroimage, 54:875–891.
SupekarK, MenonV. 2012. Developmental maturation of dynamic causal control signals in higher-order cognition: a neurocognitive network model. PLoS Comput Biol, 8:e1002374.
100.
TammL, NakoneznyPA, HughesCW. 2014. An open trial of a metacognitive executive function training for young children with ADHD. J Atten Disord, 18:551–559.
101.
UddinLQ.2015. Salience processing and insular cortical function and dysfunction. Nat Rev Neurosci, 16:55–61.
102.
UddinLQ, SupekarK, LynchCJ, KhouzamA, PhillipsJ, FeinsteinC, et al.2013. Salience network-based classification and prediction of symptom severity in children with autism. JAMA Psychiatry, 70:869–879.
103.
UddinLQ, SupekarKS, RyaliS, MenonV. 2011. Dynamic reconfiguration of structural and functional connectivity across core neurocognitive brain networks with development. J Neurosci, 31:18578–18589.
104.
van RooijD, AnagnostouE, ArangoC, AuziasG, BehrmannM, BusattoGF, et al.2018. Cortical and subcortical brain morphometry differences between patients with autism spectrum disorder and healthy individuals across the lifespan: results from the ENIGMA ASD Working Group. Am J Psychiatry, 175:359–369.
105.
VatanseverD, ManktelowAE, SahakianBJ, MenonDK, StamatakisEA. 2016. Cognitive flexibility: a default network and basal ganglia connectivity perspective. Brain Connect, 6:201–207.
106.
VijSG, NomiJS, DajaniDR, UddinLQ. 2018. Evolution of spatial and temporal features of functional brain networks across the lifespan. Neuroimage, 173:498–508.
107.
WaterhouseL, GillbergC. 2014. Why autism must be taken apart. J Autism Dev Disord, 44:1788–1792.
108.
WatsonCG. 2018. brainGraph: graph theory analysis of brain MRI data (version R package version 2.2.0). https://cran.r-project.org/package=brainGraph Last accessed Nov.7, 2019.
109.
WuT, ZangY, WangL, LongX, LiK, ChanP. 2007. Normal aging decreases regional homogeneity of the motor areas in the resting state. Neurosci Lett, 423:189–193.
110.
XiaCH, MaZ, CiricR, GuS, BetzelRF, KaczkurkinAN, et al.2018. Linked dimensions of psychopathology and connectivity in functional brain networks. Nat Commun, 9:3003.
111.
XiaM, WangJ, HeY. 2013. BrainNet viewer: a network visualization tool for human brain connectomics. PLoS ONE, 8:e68910.
112.
YerysBE, TuncB, SatterthwaiteTD, AntezanaL, MosnerMG, BertolloJR, et al.2019. Functional connectivity of frontoparietal and salience/ventral attention networks have independent associations with co-occurring attention-deficit/hyperactivity disorder symptoms in children with autism. Biol Psychiatry Cogn Neurosci Neuroimaging, 4:343–351.
113.
YerysBE, WallaceGL, SokoloffJL, ShookDA, JamesJD, KenworthyL. 2009. Attention deficit/hyperactivity disorder symptoms moderate cognition and behavior in children with autism spectrum disorders. Autism Res, 2:322–333.
114.
YerysBE, WolffBC, MoodyE, PenningtonBF, HepburnSL. 2012. Brief Report: impaired Flexible Item Selection Task (FIST) in school-age children with autism spectrum disorders. J Autism Dev Disord, 42:2013–2020.
115.
ZaldDH, LaheyBB. 2017. Implications of the hierarchical structure of psychopathology for psychiatric neuroimaging. Biol Psychiatry Cogn Neurosci Neuroimaging, 2:310–317.
116.
ZhongJ, Rifkin-GraboiA, TaAT, YapKL, ChuangKH, MeaneyMJ, QiuA. 2014. Functional networks in parallel with cortical development associate with executive functions in children. Cereb Cortex, 24:1937–1947.
117.
ZuoXN, KellyC, Di MartinoA, MennesM, MarguliesDS, BangaruS, et al.2010. Growing together and growing apart: regional and sex differences in the lifespan developmental trajectories of functional homotopy. J Neurosci, 30:15034–15043.
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
