Problem behavior in young children referred with language difficulties: Relations to language and intentional communication

Abstract

Background and aims

This exploratory study aims to examine the relative contribution of language and intentional communication to internalizing and externalizing problem behavior.

Methods

Twenty-nine Dutch-speaking children (age range 24–46 months) referred with language difficulties participated in this study. For the majority of children, these early language difficulties appeared to be part of a broader neurodevelopmental disorder, mainly autism spectrum disorder. Parent ratings on the Achenbach Child Behavior Checklist 1½–5 were predicted from children’s language level and intentional communicative abilities, the latter being assessed by both parent report and direct observation. In all series of hierarchical regression analyses, chronological age and nonverbal mental age were included as covariates.

Results

Parents commonly reported withdrawal, emotionally reactive behavior, attention problems, and aggressive behavior. Parent-rated intentional communication was the most important predictor of internalizing problem behavior and played an important role in the prediction of aggressive behavior as well. However, chronological age and/or nonverbal mental age also predicted parent-rated levels of externalizing problem behavior, especially attention problems.

Conclusions

The relation between language difficulties and problem behavior may be influenced by maturation and children’s ability to communicate intentionally.

Implications: Language proficiency should, therefore, be independently assessed from children’s intentional communicative abilities which, in turn, may differ across contexts.

Keywords

Language difficulties intentional communication problem behavior early childhood

Introduction

A delayed onset of first words or reduced vocabulary size often readily attracts parental attention and is a major reason for referral to speech-language pathologists or child mental health services (Camarata, 2014; Kozlowski, Matson, Horovitz, Worley, & Neal, 2011). Young children with language difficulties do not constitute a homogeneous group but vary in the type, severity, and developmental trajectory of the language problems experienced (Desmarais, Sylvestre, Meyer, Bairati, & Rouleau, 2008; Jansen et al., 2013). Moreover, many of these children experience additional difficulties across domains of functioning. Besides limited cognitive abilities (Buschmann et al., 2008; Carson, Klee, Perry, Muskina, & Donaghy, 1998b), motor problems (Hill, 2001), and social deficits (Carson et al., 1998b; McCabe, 2005; Stanton-Chapman, Justice, Skibbe, & Grant, 2007), problem behavior is frequently reported by parents and teachers of young children with language difficulties (Carson et al., 1998b; Henrichs et al., 2013; McCabe, 2005; Stanton-Chapman et al., 2007). Language difficulties can also be an early marker of different neurodevelopmental disorders, with language disorder (LD), autism spectrum disorder (ASD), intellectual disability (ID), and/or attention-deficit/hyperactivity disorder (ADHD) being particularly common (Ek et al., 2012; Miniscalco, Nygren, Hagberg, Kadesjö, & Gillberg, 2006).

Uncertainty about young children’s language development and the expectation that problem behavior will diminish as language proficiency increases (Gallagher, 1999) may result in a wait-and-see approach to addressing problem behavior. Research on typically developing children, however, revealed that problem behavior may have a negative influence on child and family functioning (Fuchs, Klein, Otto, & von Klitzing, 2013). Parents of children who display problem behavior are at risk for experiencing elevated levels of strain and stress which, in turn, may disrupt their parenting skills (Beernink, Swinkels, Van der Gaag, & Buitelaar, 2012; Long, Gurka, & Blackman, 2008; Vaughan, Feinn, Bernard, Brereton, & Kaufman, 2013). As problem behavior may, either directly or indirectly, compromise the development of young children with language difficulties, it is important to understand those factors associated with its onset and persistence if intervention is to be successful. This study focused on the differential relationship with language and intentional communication, thereby separating children’s language proficiency (i.e. structural aspects of language such as the number of words known or the number of syntactic structures mastered) from their ability to use verbal and nonverbal means to communicate about their thoughts, needs, and feelings in everyday situations.

Relationship between language and problem behavior

Over the past few decades, researchers from various disciplines have studied the relationship between language difficulties and internalizing (e.g. anxious/depressive behavior and withdrawal) as well as externalizing (e.g. attention problems, conduct problems, and hyperactivity) problem behavior. These studies varied widely in the sample characteristics (e.g. age and setting from which the children were recruited) as well as in the instruments, informants, and criteria used to identify problems within the language or behavioral domain (Curtis, Frey, Watson, Hampton, & Roberts, 2018; Gallagher, 1999).

Differences in terminology for labeling language difficulties

Over the past few decades, researchers and clinicians have used different terms to label children with language difficulties. These terms differ in their connotation and reflect how LDs have been conceptualized over time by different professional groups. Since 1980, the term ‘specific language impairment’ (SLI) has been commonly used for children whose persistent language difficulties are not explained by ID, sensory or motor impairments, or neurological or physical conditions (Bishop, 2014; Reilly et al., 2014). Recently, researchers started a debate about whether the label developmental language disorder (DLD) should replace SLI (Bishop, 2017; Bishop et al., 2017). In this manuscript, the term ‘language difficulties’ will be used as an overarching term to refer to children who experience temporary or persistent receptive and/or expressive language problems which can be related to several developmental disorders (and not only LD). Whenever other terms are used, these correspond to the terms used by authors whose work is reported on.

Differences in age span

Inconsistent associations between language difficulties and problem behavior have been reported for children under the age of three. Rescorla and Alley (2001) found comparable levels of mother-reported problem behavior in 33 children with expressive language delays (mean age: 25.17 months) and typically developing controls, individually matched for age and socioeconomic status (SES). Henrichs et al. (2013), by contrast, found modest concurrent and predictive associations between vocabulary delay (assessed at 18 and 30 months) and parent-reported problem behavior (measured at 18 and 36 months) in a population-based sample of 5497 children from the Netherlands. Nevertheless, most of the variance in problem behavior was explained by covariates such as demographic variables, parenting stress, and the child’s birth weight. A significant relationship between language difficulties and problem behavior was also found in one of the studies conducted by Rescorla, Ross, and McClure (2007). However, these associations may have been specific to children with neurodevelopmental delays and/or possible ASD. When these children were excluded from analyses, only the relationship between language difficulties and withdrawn behavior remained significant.

More robust findings on the relation between language difficulties and problem behavior have been reported for preschool and school-age children, which may indicate that these associations strengthen over time (Benner, Nelson, & Epstein, 2002; Curtis et al., 2018; Rescorla et al., 2007). These studies can be divided into three groups: (1) those that reported high rates of (undiagnosed) language difficulties in children with problem behavior (reviews by Benner et al., 2002; Hollo, Wehby, & Oliver, 2014), (2) those that identified limited language abilities as a risk marker for problem behavior in children with various neurodevelopmental disorders (e.g. Dominick, Ornstein Davis, Lainhart, Tager-Flusberg, & Folstein, 2007; Sigafoos, 2000), and (3) those that found an increased prevalence of internalizing and externalizing problem behavior in children with limited language abilities (e.g. Bornstein, Hahn, & Suwalsky, 2013; St Clair, Pickles, Durkin, & Conti-Ramsden, 2011; Yew & O’Kearney, 2013).

Differences in the evaluation of influencing factors

Direct—though bidirectional—relationships between language difficulties and problem behavior have been proposed, acknowledging: (1) the importance of language in the regulation of emotions and behavior by facilitating executive control (Gallagher, 1999; Roben, Cole, & Armstrong, 2013) and (2) the interference of problem behavior with typical language acquisition. Nevertheless, the exact nature of the language–behavior relationship remains unknown. Carpenter and Drabick (2011) integrated potential risk, mediating, and moderating factors into a developmental framework while acknowledging that for different groups of children different pathways will result in the co-occurrence of language difficulties and problem behavior. Although their framework focused on externalizing problem behavior only, it is likely that the same notion applies to the relationship between language difficulties and internalizing problem behavior. Hence, it is important to move beyond frequency counts and correlational designs and examine factors that may influence the language–behavior relationship.

Children with receptive or mixed receptive–expressive language difficulties, for example, appear to be particularly prone to the development of problem behavior (Beitchman, Hood, Rochon, & Peterson, 1989; Beitchman et al., 1996; Chow & Wehby, 2018). However, these children are also more likely to experience cognitive delays (Buschmann et al., 2008), which may be a shared risk factor for language difficulties and problem behavior (Benasich, Curtiss, & Tallal, 1993). Another factor that may affect the language–behavior relationship is children’s ability to communicate intentionally (Carpenter & Drabick, 2011), which does not necessarily correspond to their language level (Ketelaars, Cuperus, Van Daal, Jansonius, & Verhoeven, 2009; Landa, 2005).

Pragmatic difficulties as additional predictor

Children with limited intentional communicative abilities may display problem behavior to meet their functional communication needs. The notion that problem behavior sometimes serves specific communicative functions (e.g. protesting or attention seeking) is captured in the “communication hypothesis” based on the work of Carr and Durand (1985). Congruent with this hypothesis is the reduction of problem behavior after acquisition of functionally equivalent replacement behaviors, which may either be linguistic in nature or consist of some form of augmentative or alternative communication (e.g. gestures or graphic symbols) (Gerow, Davis, Radhakrishnan, Gregori, & Rivera, 2018; Walker & Snell, 2013).

The ability to communicate intentionally is part of the pragmatic system which also comprises the domains of presupposition and discourse management (Landa, 2005). Several studies have examined the relationship between pragmatic abilities and problem behavior in school-age children. St Clair et al. (2011), for example, studied the developmental trajectories of individuals with a history of SLI across a nine-year time frame and found a continuous negative relationship between pragmatic abilities and problem behavior. Mackie and Law (2014), in addition, reported pragmatic difficulties for boys with externalizing problem behavior which seemed independent of their language proficiency.

Other studies took this research one step further and explored whether pragmatic difficulties contributed to problem behavior above and beyond language level. In a sample of children with SLI, Lindsay, Dockrell, and Strand (2007) found that language ability was the only significant predictor of teacher-rated behavior problems. Law, Rush, and McBean (2014), by contrast, reported that pragmatic abilities partially mediated the relationship between language difficulties and problem behavior in their sample of primary school children recruited from socially disadvantaged backgrounds. Also in a group of 4- to 7-year-old children with ASD, pragmatic language deficits were clearly associated with child anxiety and externalizing problems, whereas the relation with structural language skills was relatively small for anxiety and absent for externalizing problems (Rodas, Eisenhower, & Blacher, 2017). In a recently published longitudinal study on the abilities of children initially referred for language concerns, Roy and Chiat (2014) used the Social Responsiveness Scale (SRS: Constantino & Gruber, 2005) to measure social-communicative ability. High levels of problem behavior scores were reported for 9- to 11-year-old children with limited social-communicative abilities. In comparison to children with receptive language difficulties only, these children were at a higher risk for peer problems, limited prosocial behavior, and hyperactivity. The SRS, however, measures ASD-related preoccupations as well as social-communicative abilities. Moreover, the authors did not report whether the stronger association between limited social-communicative abilities and problem behavior existed when these children were younger, a developmental period where intervention is most likely to be effective.

Purpose of the current study

The aim of the current exploratory study was to evaluate the relative contribution of language and intentional communication to parent-rated levels of problem behavior in young children with language difficulties. We studied whether intentional communication makes a larger contribution to internalizing and externalizing problem behavior than language level. In accordance with the communication hypothesis (Carr & Durand, 1985) we hypothesized that the ability to express oneself would be more important than the form that is used to request or share information. Put differently, we expected that children with limited language abilities who were somehow able to communicate about their needs and wishes would be less likely to display problem behavior than children who were unable to meet their interpersonal communication needs.

Previous studies focused on the relationship between language, a broader range of pragmatic abilities, and problem behavior and have found inconsistent results. These studies relied on questionnaires to assess pragmatic ability and mainly included school-age children. Furthermore, they focused on the interrelatedness of abilities in specific groups of children, such as those diagnosed with SLI or ASD. However, children with language difficulties are often referred at an early age when diagnostic decision making is difficult and often requires repeated assessment across domains of functioning (Gillberg, 2010).

To reflect this clinical complexity, this study included consecutively referred 24- to 46-month-old children with language difficulties. In addition, it extended previous studies by (1) focusing on intentional communication as this is among the first observable pragmatic abilities in young children and those with limited language proficiency, (2) using a semi-structured observation to complement a parent report measure of intentional communicative ability, and (3) including toddlers as well as preschool aged children.

Methods

Participants

Participants were prospectively and consecutively recruited from two outpatient centers for children with developmental (language) difficulties in Leuven (Flanders, Belgium). The sample comprised 36 monolingual Dutch-speaking children (30 boys, 6 girls) between 24 and 46 months of age (M = 33.36, SD = 6.82 months). For inclusion in the study, participants had to meet the following criteria: (1) perform below the 16th percentile on the receptive and/or expressive language scale of a standardized language test or below percentile 3 on one or more of its subscales; (2) obtain a nonverbal mental age equivalent score of 15 months or more; (3) normal hearing, no severe motor impairments, or uncorrectable sensory impairments; and (4) no frank neurological signs or known genetic syndrome. Based on criteria 2–4 only children with very severe impairments that prevented them to complete the assessment instruments used in this study were excluded. Children with missing data on any of the dimensions of interest (n = 7) were excluded from the sample. There are no significant differences on any of the background characteristics or used measures between the included and excluded participants. The final sample constituted of 29 children (23 boys, 6 girls) with a mean age of 33 months (SD = 6.92 months).

The current study reports data from the first phase of a longitudinal research project on the abilities and developmental trajectories of young children with language difficulties. Approximately three years later, the clinical files of all children were reviewed and best-estimate (BE) diagnoses of LD, ID, and ASD were made by the first two authors. BE diagnoses were established on the basis of follow-up data and the diagnostic categories of interest were divided into four levels of diagnostic confidence for which criteria were specified in advance. Using DSM-5 criteria, a BE diagnosis of ID precluded a BE diagnosis of LD. By contrast, DSM-5 allows a diagnosis of LD in children with ASD and marked language difficulties (American Psychiatric Association, 2013). There was complete agreement between the raters, except for one child for whom a consensus was reached after joint review of the information available. For 8 out of the 29 children whose data are reported in the current study, the information available was insufficient for a BE diagnosis to be made, as no follow-up data were present for these children. Three out of the 21 children did not receive a BE diagnosis of any of the disorders of interest. For the remaining children (n = 18), the following BE diagnoses were established: LD (n = 2), ID (n = 1), ASD (n = 6), ASD with co-occurring ID (n = 4), or ASD with co-occurring LD (n = 5). Note that a BE diagnosis of LD was only given to children with persistent language difficulties despite intervention and a diagnosis of LD was not made if children met criteria for a BE diagnosis of ID.

Instruments

Language ability

None of the norm-referenced Dutch language measures available to date covered the heterogeneity in language abilities across the participating children. Therefore, different instruments had to be used with the choice dependent on children’s individual level of receptive and expressive language ability. The Dutch version of the Reynell Developmental Language Scales (RTOS: Schaerlaekens, Zink, & Van Ommeslaeghe, 2003) is suited for children with language age equivalent scores from 24 to 46 months and assesses more formal aspects of language such as lexicon, semantics, syntax, and grammatical morphology. The Dutch version of the NonSpeech Test for Receptive and Expressive Language (NNST: Zink & Lembrechts, 2000) evaluates early emerging (non)verbal abilities in children with language age equivalent scores from 12 to 21 months. Both instruments yield a composite score of receptive as well as expressive language and have been considered reliable (regarding RTOS, internal consistency: λ−2 = .79–.95; regarding NNST, internal consistency: λ−2 = .83–.95) and valid measures of language abilities in young children (Egberink, Vermeulen, & Frima, 2014; Zink, Van Ommeslaeghe, Stroobants, Janssen, & Schaerlaekens, 1993). Expressive language scores for children with limited on-task behavior were derived from the short-forms of the Dutch version of the MacArthur-Bates Communicative Development Inventories (N-CDIs: Zink & Lejaegere, 2003, 2007). Adequate psychometric properties (internal consistency: α = .97–.98) have been reported for this parental questionnaire which is suited for children with language age equivalent scores between 8 and 37 months (Zink & Lejaegere, 2003, 2007). With regard to receptive language ability, all scores were based on individual assessment (NNST: n = 7; RTOS: n = 22). Expressive language ability was either directly assessed (NNST: n = 15; RTOS: n = 8) or based on parent report (N-CDIs: n = 6). For those participants for whom both measures were available, parent-reported expressive abilities were highly correlated to those measured by formal language testing (r_s = .83).

Intentional communication

Both the Caregiver Questionnaire and Behavior Sample of the Communication and Symbolic Behavior Scales Developmental Profile (CSBS DP: Wetherby & Prizant, 2002) evaluate early emerging (pre)linguistic abilities. The Communication Scale of either measure addresses the occurrence and function of children’s intentional communicative behavior and was therefore included in this study. With regard to the Caregiver Questionnaire, this scale comprises 10 items that have to be rated as not yet or rarely present, sometimes present, or often or usually present (internal consistency: α = .93; test–retest reliability: r = .86). Questions cover the communicative function domains of behavior regulation (request or protest: three items), social interaction (direct someone’s attention to self: four items), and joint attention (direct someone’s attention to an object or event: two items). In addition, there is a more general question regarding children’s abilities to convey their message in different ways. Raw scores were summed to create a composite score, with higher scores indicating more intentional communicative behavior.

During the semi-structured Behavior Sample, intentional communication is elicited by means of attractive play materials (e.g. balloon, bubbles, and books). Video recordings were transcribed and coded to assess the frequency and function of children’s intentional communicative behavior. Intentional communication was defined as a gesture, vocalization, and/or verbalization which was spontaneously directed toward an adult (whether the parent or the evaluator) and served a communicative function. Identified communicative acts were ascribed to the function categories of behavior regulation, social interaction, or joint attention. Acts of which the meaning could not be inferred unambiguously (for example in the case of unintelligible speech) were categorized as ‘function unclear’ (Wetherby & Prizant, 1993). For each part of the CSBS DP Behavior Sample, the original coding scheme incorporates three coding opportunities per communicative function. An adapted version of this coding scheme was used (Maljaars, Noens, Jansen, Scholte, & Van Berckelaer-Onnes, 2011) as frequency counts of all communicative acts allow a more detailed analysis of children’s intentional communicative abilities (Landa, 2005). The second author double-coded 20% of the transcriptions with a percentage agreement of 97% for identification of communicative acts. Percentage agreement across communicative functions ranged from 81 to 90%.

Problem behavior

The presence of problem behavior was assessed using the preschool version of the Child Behavior Checklist (CBCL 1½–5: Achenbach & Rescorla, 2000). This parental questionnaire consists of 99 problem items that have to be rated as not true, somewhat or sometimes true, or very or often true, depending on the child’s behavior in the preceding two months. The items cluster around seven empirically derived syndrome scales that can be reduced to two broadband factors. The first factor, internalizing problem behavior, comprises four syndrome scales (Emotionally Reactive, Anxious/Depressed, Somatic Complaints, and Withdrawn) whereas the second factor, externalizing problem behavior, comprises two syndrome scales (Attention Problems and Aggressive Behavior). The Sleep Problem syndrome scale is not related to either broadband factor but is included in the Total Problems scale which sums scores across each of the 99 problem items. Raw scores are converted to percentile ranks and T scores which can be classified into the normal range, the borderline clinical range (broadband factors: T scores 60–63; syndrome scales: T scores 65–69), or the clinical range (broadband factors: T scores ≥ 64; syndrome scales: T scores ≥70). Adequate psychometric properties of reliability (internal consistency: α = .66–.92; test–retest reliability: r = .68–.92) and validity have been reported for this questionnaire and the CBCL 1½–5 was found to distinguish between clinically referred and non-referred children with 84.2% of them classified correctly (Achenbach & Rescorla, 2000). The seven-syndrome factor structure was replicated for Flanders, Belgium (Ivanova et al., 2010) though further research on the validity of the Dutch version of the CBCL 1½–5 has not been published to date. Nevertheless, data that support the construct and criterion validity of the CBCL in young, Dutch-speaking children are available for the CBCL/2–3 (Achenbach, 1992), which differed in only two items from the CBCL 1½–5 (Koot, Van den Oord, Verhulst, & Boomsma, 1997). Questionnaires were scored by means of the Assessment Data Manager—version 9.1 which incorporates the multicultural norms provided by Achenbach and Rescorla (2010).

Nonverbal intelligence

The range of nonverbal cognitive abilities among participants could not be captured by any of the norm-referenced Dutch intelligence measures available to date. Therefore, nonverbal cognitive ability scores were based on administration of the Dutch version of the Bayley Scales of Infant Development (BSID-II-NL: n = 12) or the Snijders-Oomen Non-verbal Intelligence Test (SON-R 2½–7: n = 17). The nonverbal scale of the BSID-II-NL (Van der Meulen, Ruiter, Lutje Spelberg, & Smrkovsky, 2002) is suited for children with mental age equivalent scores from 12 to 30 months and has adequate psychometric properties (internal consistency: λ−2 = .79, test–retest reliability: r = .75). The revised version of the SON-R 2½–7 (Tellegen, Winkel, Wijnberg-Williams, & Laros, 1998) was administered to children with more mature cognitive abilities. The SON-R 2½–7 is a well-established measure for which new standardization data that adjust for the Flynn-effect (i.e. systematic rise in IQ scores over time) became available in 2013 (internal consistency: α = .90, test–retest reliability: r = .79) (Tellegen & Laros, 2013).

ASD-related characteristics

Children with ASD often present with internalizing as well as externalizing problem behavior (Bauminger, Solomon, & Rogers, 2010). Hence, a measure of ASD symptom severity was included in the current study. The presence of ASD-related characteristics was assessed by means of the Autism Diagnostic Observation Schedule (ADOS: Lord et al., 2012) which is a semi-structured observation that contains standardized activities and social presses to elicit communication as well as social behavior and promotes play as well as the imaginative use of objects. Module 1 or 2 was administered depending on the expressive language abilities of the child. The severity score was used in the current study which is a severity measure of ASD-related characteristics that can be compared across modules (Gotham, Pickles, & Lord, 2009). The mean severity score is this sample was 3.3 (SD = 1.25; range: 1–7).

Procedure

Parents of all children who met inclusion criteria received oral and written information on the content of the research project. After parental informed consent was obtained, information on children’s language and cognitive abilities was retrieved from their clinical files. Parents completed both questionnaires and were, together with their child, invited for administration of the CSBS DP Behavior Sample and the ADOS. For 79% of the children, administration of all measures (including the previously administered language and cognitive tests) was completed within a three-month period (range 0–5 months, M = 2.34, SD = 1.34). The design of the study was approved by the medical ethical board of the University Hospitals of Leuven.

Data analyses

The language and/or nonverbal cognitive abilities of many participants were below age-expected levels. As the choice of instruments was adapted accordingly, the chronological age of these children often exceeded that of the standardization sample. To obtain a standard unit of measurement, raw scores of all cognitive and language measures were converted to age equivalent scores prior to analyses. Tables to convert raw scores into age equivalents (based on 50th percentile) were available in the manual of each instrument. In addition, coded communicative acts (both the total number and the number of communicative acts ascribed to each of the four function categories) were transformed to rates per minute to correct for individual differences in duration of the CSBS DP Behavior Sample (M = 19.50, SD = 3.65 minutes). Examination of the data revealed that the distribution of some variables was rather skewed to the right, indicating that too many children obtained relatively low scores on the domains of language and some of the CBCL 1½–5 syndrome scales. Therefore, non-parametric statistical techniques (Spearman’s correlation coefficient (r_s) and Wilcoxon signed-rank test) were used to analyze the relationship among predictor variables as well as the relationship between predictor variables and the presence of internalizing and externalizing problem behavior. Because of the small sample size, exact tests were used whenever possible. Both broadband factors and specific syndromes scales were used in the regression analyses, as mixed results on total scores or broadband scales in previous studies suggest that a more nuanced approach to behavior problems may be required. Moreover, other studies suggest a different relationship with language for specific scales within one broadband factor (e.g. Carson, Klee, Lee, Williams, & Perry, 1998a; Hooper, Roberts, Zeisel, & Poe, 2003).

The relative contribution of language and intentional communication to parent-rated levels of problem behavior was assessed by means of hierarchical multiple regression analyses. This technique enables the prediction of an outcome from several predictor variables which do not need to be normally distributed (Williams, Grajales, & Kurkiewicz, 2013). As our sample size is rather small, the models are conservative and there is a larger chance on Type II errors. In each series, chronological age and nonverbal mental age were included as covariates. Although they both reflect a child’s maturity, it may be important to distinguish between the two as older children may have experienced more frustration and may therefore be more likely to act out behaviorally or withdraw from social interaction.

There was no multicollinearity between predictor variables as most correlations were below .80, except for the correlation between receptive language and nonverbal mental age (r_s = .84). Moreover, variance inflation factors for variables in all models were well below 10 and all tolerance statistics exceeded the value of 0.1. Other basic assumptions for hierarchical regression analysis (independent residuals and normally distributed errors, linearity, and homoscedasticity) were met as well. With regard to the regression models for emotionally reactive behavior and aggressive behavior, one case had a standardized residual above two. For each model, however, the value of Cook’s distance was below one and leverage values were within the boundary of two times the average. Therefore, this case does not seem to have a large effect on the regression models (Stevens, 2002) and was included in the analyses.

Results

Problem behavior

Parents reported similar levels of internalizing and externalizing problem behavior (z = .001, p = .502) which in turn were highly correlated to each other (r_s = .72). About half of the children (45%) scored within the borderline or clinical range of at least one of the CBCL 1½–5 syndrome scales. Withdrawn behavior was most commonly reported, followed by emotionally reactive behavior, aggressive behavior, and attention problems (Table 1). Compared to the CBCL 1½–5 standardization sample, significantly more children attained elevated scores on the withdrawn (X²_(1, _n _= 29) = 20.91, p < .001) or emotionally reactive (X²_(1, _n _{= 29)} = 6.35, p = .025) syndrome scales. After application of the Bonferroni correction for multiple comparisons, only differences in withdrawn behavior remained significant.

Table 1.

Parent-rated problem behavior (n = 29) in comparison to the CBCL 1½–5 standardization sample.

	Mean T score (SD)	Range
CBCL 1½–5 scales	Mean T score (SD)	Normal (%)	Elevated^a (%)	Χ²	p
Broadband factors
Total problems	52.45 (11.42)	69.0	31.0	3.34	.086
Internalizing problems	52.55 (10.26)	79.3	20.7	0.14	.808
Externalizing problems	52.41 (11.51)	75.9	24.1	0.74	.465
Syndrome scales
Emotionally reactive	56.86 (7.65)	79.3	20.7	6.35	.025
Anxious/Depressed	52.03 (3.81)	96.6	3.4	0.82	.512
Somatic complaints	53.00 (5.33)	96.6	3.4	0.82	.512
Withdrawn	60.59 (8.83)	69.0	31.0	20.91	<.001^†
Sleep problems	52.59 (4.79)	96.6	3.4	0.82	.512
Attention problems	55.03 (7.30)	86.2	13.8	1.32	.288
Aggressive behavior	56.34 (8.07)	82.8	17.2	3.37	.078

CBCL: Child Behavior Checklist.

Combined category for children with T scores from 60 (broadband factors) or 65 (syndrome scales) onwards which are considered to be in the borderline or clinical range (Achenbach & Rescorla, 2000).

†

p < .005 (adjusted p-value after Bonferroni correction for multiple comparisons based on the number of factors and scales).

Children under the age of three (n = 17) displayed more internalizing and externalizing problem behavior when compared to children of 36 months or older (n = 12), with the exception of anxious/depressive behavior, somatic complaints, and withdrawn behavior. No significant differences between boys and girls on any of the scales were found. After Bonferroni correction, differences in T scores for externalizing problem behavior (<36 months: M = 57.59, SD = 10.66; ≥36 months: M = 45.08, SD = 8.48, p = .001), emotionally reactive behavior (<36 months: M = 59.41, SD = 8.35; ≥36 months: M = 53.25, SD = 4.81, p = .004), and aggressive behavior (<36 months: M = 59.71, SD = 8.94; ≥36 months: M = 51.38, SD = 2.71, p = .002) remained significant. Correlations between the ADOS severity score and internalizing (r_s = .20) as well as externalizing (r_s = −.05) problem behavior were low.

Further analyses on the syndrome scales are restricted to the four CBCL 1½–5 syndrome scales that showed the most variation among participants: withdrawn, emotionally reactive (both belonging to the broadband factor for internalizing problem behavior), attention problems, and aggressive behavior (both comprising the broadband factor for externalizing problem behavior).

Language and intentional communication

After controlling for chronological age and nonverbal mental age, levels of parent-rated problem behavior were predicted from children’s language level and their ability to communicate intentionally. Information on children’s abilities across each of these predictor variables is summarized in Table 2. Most children experienced mixed receptive–expressive language difficulties (n = 16), whereas others experienced receptive (n = 1) or expressive (n = 12) language difficulties only. Children produced between 1.20 and 5.04 communicative acts per minute (M = 3.36, SD = 0.92). Acts of which the function was unclear accounted, on average, for 8.32% (SD = 7.08%) of the total number of communicative acts. Children used most of their (non)verbal communication for behavior regulation and, to a lesser extent, for the purpose of joint attention (z = −1.72, p = .044). The number of communicative acts for social interaction purposes was significantly lower (p-values <.001), but similar to the number of acts categorized as function unclear (z = −1.00, p = .327). The correlation between parent-rated and number of observed intentional communication acts was low (r_s = .24) (see Table 3).

Table 2.

Means (and standard deviations) for predictor variables of parent-rated problem behavior (n = 29).

	Range	M	SD
Chronological age (months)	24–46	33.28	6.92
Nonverbal mental age (months)	15.0–47.0	30.57	8.50
Receptive language (months)	11.0–40.5	25.70	7.96
Expressive language (months)	11.0–34.5	20.77	5.80
IC parent ratings	8.0–20.0	15.91	2.90
IC observation
Total (rate per minute)	1.20–5.34	3.39	0.98
Behavior regulation (rate per minute)	0.56–3.43	1.66	0.71
Social interaction (rate per minute)	0.00–1.05	0.29	0.25
Joint attention (rate per minute)	0.10–2.67	1.17	0.64
Function unclear (rate per minute)	0.00–0.94	0.27	0.23

IC: intentional communication.

Table 3.

Spearman correlation matrix between predictor variables and parent-rated problem behavior (n = 29).

	CA	NVMA	RLa	ELa	IC PR	IC OB Total	IC OB BR	IC OB SI	IC OB JA
Internalizing behavior problems	−.34	−.31	−.28	−.38*	−.49**	−.01	.23	−.22	−.15
Emotionally reactive	−.32	−.33	−.20	−.31	−.51**	−.02	.03	−.04	−.05
Withdrawn	−.17	−.15	−.19	−.19	−.35	.05	.35	−.33	−.08
Externalizing behavior problems	−.49**	−.54**	−.46*	−.42*	−.51**	−.27	−.17	−.09	−.18
Attention problems	−.31	−.59**	−.47**	−.48**	−.49**	−.08	.05	−.02	−.15
Aggressive behavior	−.47**	−.45*	−.38*	−.35	−.48**	−.31	−.18	−.12	−.19
Nonverbal mental age (NVMA)	.78***	–
Receptive language (RLa)	.71***	.84***	–
Expressive language (ELa)	.74***	.76***	.77***	–
Intentional communication—parent ratings (IC PR)	.25	.55**	.46*	.47*	–
Intentional communication—observations (IC OB total)	.06	.19	.25	.04	.24	–

BR: behavior regulation; CA: chronological age; JA: joint attention; SI: social interaction.

*p < .05; **p < .01; ***p < .001.

Prediction of problem behavior

Chronological age, nonverbal mental age, receptive language, expressive language, and parent-rated intentional communication were negatively correlated with levels externalizing problem behavior (Table 3). Negative correlations of expressive language and parent-rated intentional communication with internalizing problem behavior were found as well. Correlations between problem behavior and number of observed communication acts were generally low. Therefore, these ratings were not included in the series of hierarchical regression analyses performed to assess the relative contribution of language and intentional communication to parent-rated levels of problem behavior.

Table 3 also summarizes all information on the relationships between predictor variables. Strong and positive correlations between chronological age, nonverbal mental age, and language ability were found (r_s ≥ .71). In each series of hierarchical regression analyses, chronological age and nonverbal age were entered first as covariates (model 1), followed by receptive as well as expressive language ability (model 2), and parent-rated intentional communication (model 3). For the broadband factors and each of the retained CBCL 1½–5 syndrome scales, Table 4 summarizes information on the fit of each model as well as its parameters.

Table 4.

Hierarchical regression analyses for parent-reported problem behavior (n = 29).

CBCL 1½–5	Model	Predictor	β	t	p	R ²	p Δ R²
Internalizing problem behavior	1	Age	−.18	−0.58	.569	.08	.364
		NVMA	−.11	−0.34	.734
	2	Age	−.03	−0.09	.932	.12	.566
		NVMA	.04	0.10	.925
		RLa	−.09	−0.25	.806
		ELa	−.28	−0.84	.410
	3	Age	−.41	−1.20	.244	.34	.011
		NVMA	.48	1.23	.230
		RLa	−.05	−0.14	.886
		ELa	−.04	−0.12	.904
		IC PR	−.62	−2.75	.011
Emotionally reactive	1	Age	−.18	−0.59	.561	.09	.288
		NVMA	−.14	−0.44	.664
	2	Age	−.04	−0.11	.913	.13	.575
		NVMA	−.25	−0.62	.539
		RLa	.28	0.76	.455
		ELa	−.33	−1.00	.327
	3	Age	−.42	−1.24	.229	.35	.010
		NVMA	.20	0.51	.617
		RLa	.32	1.00	.330
		ELa	−.09	−0.29	.774
		IC PR	−.62	−2.79	.010
Withdrawn	1	Age	−.05	−0.15	.880	.05	.497
		NVMA	−.19	−0.60	.555
	2	Age	−.06	−0.16	.876	.12	.393
		NVMA	.15	0.38	.709
		RLa	−.50	−1.36	.188
		ELa	.10	0.30	.765
	3	Age	−.46	−1.36	.188	.37	.007
		NVMA	.62	1.62	.119
		RLa	−.45	−1.41	.171
		ELa	.35	1.18	.250
		IC PR	−.66	−2.97	.007
Externalizing problem behavior	1	Age	−.12	−0.44	.661	.29	.012
		NVMA	−.44	−1.59	.123
Attention problems	1	Age	.33	1.31	.202	.41	.001
		NVMA	−.87	−3.46	.002
Aggressive behavior	1	Age	−.24	−0.86	.401	.23	.034
		NVMA	−.26	−0.92	.365
	2	Age	−.35	−1.08	.289	.26	.607
		NVMA	−.12	−0.34	.739
		RLa	−.29	−0.85	.404
		ELa	.26	0.86	.398
	3	Age	−.64	−1.96	.062	.39	.037
		NVMA	.22	0.58	.565
		RLa	−.25	−0.81	.428
		ELa	.44	1.52	.142
		IC PR	−.48	−2.22	.037

Age: chronological age; CBCL: Child Behavior Checklist; ELa: expressive language; IC PR: intentional communication—parent ratings; NVMA: nonverbal mental age; RLa: receptive language.

With regard to internalizing problem behavior (broadband factor and subscales), inclusion of chronological age, nonverbal mental age, and language did not explain a significant proportion of the variance in withdrawn or emotionally reactive behavior. In each case, parent-rated intentional communication was the only significant predictor, indicating that fewer intentional communicative abilities predicted higher levels of problem behavior. Inclusion of parent-rated intentional communication accounted for an additional 22% of the variance in emotionally reactive behavior, and for an additional 25% of the variance in withdrawn behavior. Together, the predictors in model 3 explained approximately one-third of the variation in problem behavior (internalizing problem behavior: R² = .34, F_(1,23) = 7.57, p = .011; emotionally reactive: R² = .35, F_(1,23) = 7.80, p = .010; withdrawn: R² = .37, F_(1,23) = 8.79, p = .007).

The results for each of the CBCL 1½–5 syndrome scales belonging to the broadband factor for externalizing problem behavior diverged, and therefore no significant predictors were found in the model with the broadband factor. Nonverbal mental age was the only significant predictor of attention problems (β = −.87, p = .002), indicating that lower nonverbal cognitive abilities predicted higher levels of parent-rated problem behavior. Together with chronological age, this predictor accounted for 41% of the variance in attention problems (F_(2,26) = 8.87, p = .001). Aggressive behavior, by contrast, was best predicted by chronological age (β = −.64, p = .062) and parent-rated intentional communication (β = −.48, p = .037), indicating that lower age and fewer intentional communicative abilities predicted higher levels of aggressive behavior. Together the predictors included in model 3 explained 39% of the variation in aggressive behavior (F_(1,23) = 4.92, p = .037) with chronological age being the most important predictor.

Discussion

In the last few decades, the co-occurrence of language difficulties and problem behavior has attracted considerable attention. While little is known about potential mechanisms underlying this relationship, Carpenter and Drabick (2011) stressed the importance of differentiating language from intentional communication. In a heterogeneous group of 24- to 46-month-old children referred with language difficulties, this study examined whether intentional communication makes a larger contribution to problem behavior than language level. Because of the small sample size, the results of this exploratory study should be interpreted with caution.

Early language difficulties as sign of a broader neurodevelopmental disorder

Although participants were selected because of their receptive and/or expressive language difficulties, many turned out to have a neurodevelopmental disorder with ASD being most common. Longitudinal, prospective studies on the developmental outcomes of children with language difficulties also found elevated rates of neurodevelopmental disorders other than LD. Miniscalco et al. (2006), for example, followed a community sample of 30-month-old children up to seven years of age and concluded that these children were at high risk for ASD as well as ADHD. These findings underline the importance of repeated diagnostic assessment in young children with language difficulties.

Problem behavior and its predictors

The generalizability of our findings may be limited to those children whose early language difficulties are part of a broader neurodevelopmental disorder as they were overrepresented in the current sample. Nevertheless, as the correlations between the amount of problem behavior and the presence of ASD-related characteristics were low, the high rates of problem behavior found within the current study could not be fully explained by the overrepresentation of children with ASD within our sample. The results of this study, therefore, converged with previous findings suggesting elevated levels of parent-rated problem behavior in children with language difficulties (e.g. Lindsay et al., 2007; McCabe, 2005; Stanton-Chapman et al., 2007). Almost half of the children in this study scored within the borderline or clinical range of at least one CBCL 1½–5 syndrome scale. Withdrawal and emotionally reactive behavior were the most common types of internalizing problem behavior, whereas attention problems and aggressive behavior predominated as externalizing problem behaviors.

Internalizing and externalizing problem behaviors were predicted from children’s intentional communicative abilities, after adjustment for chronological age and nonverbal mental age. Parent-rated intentional communication was in our model the only significant predictor of internalizing problem behaviors. However, the absence of significant results for the other predictor variables has to be interpreted with caution, due to the small sample size. Children with limited intentional communicative abilities are less able to express their thoughts, needs, and feelings or may experience difficulties with specific communicative functions. This may result in feelings of insufficiency and withdrawal from social interaction, which in turn diminishes children’s opportunities to practice and expand their emerging verbal and nonverbal communication skills (Carpenter & Drabick, 2011). Moreover, limited communication for joint attention purposes may impede further language development as, at least for children with ASD, these abilities are closely intertwined (Bottema-Beutel, 2016).

Parent-reported intentional communication was an important predictor for aggressive behavior as well, indicating that children who have limited intentional communicative abilities may be more likely to act out behaviorally (Gallagher, 1999; Roben et al., 2013). Maturity, however, also seemed to play an important role as nonverbal mental age and chronological age were significant predictors of attention problems and aggressive behavior, respectively. Many of the previous studies did not take chronological and/or nonverbal mental age into account and this raises the question whether some types of externalizing problem behavior are, at least for some children, more related to immaturity than to language or communicative difficulties per se. Although elevated levels of problem behavior may persist over time (Benasich et al., 1993; Bornstein et al., 2013; St Clair et al., 2011), many of the isolated behaviors decrease during development. Tantrums, aggressiveness, and oppositional behaviors are displayed by the majority of children during the ‘terrible twos’, suggesting that these behaviors are often developmentally appropriate and may be manifestations of emerging self-awareness and independence (Carter, Briggs-Gowan, & Ornstein Davis, 2004; Tremblay et al., 1999).

Assessment of intentional communication

Low correlations were found between parent-rated and number of observed intentional communication acts. This could be due to differences between the measures as almost half of the questions of the CSBS DP Caregiver Questionnaire comprised communicative behavior for social interaction purposes (i.e. children’s ability to direct someone’s attention toward oneself), which was the least observed function during the CSBS DP Behavior Sample. Eliciting intentional communication by means of attractive play materials may promote object-centered interactions (i.e. requesting, protesting, or commenting), while underestimating children’s ability to communicate for social interaction (Maljaars et al., 2011). Moreover, as one of the parents was seated next to the child this may have reduced children’s necessity to call for their attention.

On the other hand, the absence of a relationship between these measures may reflect real differences in children’s abilities to communicate their intentions in open-ended (i.e. every day activities) versus more structured situations. Pragmatic abilities are by definition context-dependent which denotes that the way in which pragmatic abilities are assessed may have an influence on the outcome (Adams, 2002; Adams & Lloyd, 2005; Landa, 2005). Several studies have compared (semi-)structured assessment procedures (including elicitation tasks) with both conversational measures and parent ratings of pragmatic abilities and concluded that the introduction of structure alters children’s behavior and enhances their performance (Adams & Lloyd, 2005; Bishop & Adams, 1991; Volden & Phillips, 2010). Parent ratings of intentional communication may be a more accurate reflection of children’s abilities across contexts (Ketelaars et al., 2009) and, therefore, more closely related to functional outcome measures such as the presence of problem behavior. More structured sampling, on the other hand, reveals information on children’s communicative abilities in less demanding and more scaffolded contexts and may, together with parent report or naturalistic observation, inform intervention planning (Adams, 2002; Landa, 2005).

Implications for clinical practice

Because of rapid developmental changes, many practitioners are hesitant to define and assess problem behavior in infants, toddlers, and preschool-aged children. Moreover, behavior considered problematic at older ages (e.g. tantrums) is developmentally appropriate in early childhood. The intensity, duration, and/or frequency of certain problem behaviors as well as the impact on child and family functioning may, nevertheless, differentiate typical from atypical behavior (Carter et al., 2004). Early problem behavior may be associated with long-term adjustment problems (Benasich et al., 1993; Bornstein et al., 2013; Campbell, 1995) and elevated levels of parental strain and stress (Beernink et al., 2012; Long et al., 2008; Vaughan et al., 2013). Hence, attention for problem behavior is important, especially in at-risk children with language difficulties.

Interventions addressing problem behavior vary along a continuum of intensity, ranging from psycho-education and parent training offered in group formats, to individualized programs for parents and children (Powell, Dunlap, & Fox, 2006). Based on a review of available intervention studies, Dunlap et al. (2006) concluded that assessment of the function of problem behavior is a necessary first step for any individualized intervention program. Children whose limited intentional communicative abilities lead them to refrain from social interaction or to display problem behavior should learn alternative means to communicate (Mirenda, 1997; Walker & Snell, 2013). These should be suited to their level of sense-making and special attention must be given to the generalization of these behaviors across contexts (Mirenda, 1997; Noens & Van Berckelaer-Onnes, 2004).

Once replicated, the results of this study indicate that intentional communicative abilities should be assessed in children with substantial language difficulties presenting for diagnostic evaluation. Nevertheless, the number of norm-referenced measures that assess intentional communication and more encompassing pragmatic abilities is currently limited for children under the age of four and those with limited language proficiency. The development of such instruments—which should consist of informant report and direct observation—is nevertheless important. Informant report may be used for screening as well as more comprehensive assessment of children’s intentional communicative abilities. Observational measures, on the other hand, will be time intensive and are therefore more likely to be limited to those children who screen positive for intentional communication problems. Clinicians can explicitly ask parents, caregivers, and teachers about children’s abilities to request, protest, and share their experiences and emotions. In addition, clinicians should be alert for these behaviors during assessment and observation periods, especially in unstructured and more ambiguous situations. Attention should be paid to both the quantity and quality of children’s intentional communicative behavior, which will reveal information on the number of intentional communicative acts and the range of functions used as well as the appropriateness of children’s communicative behaviors within specific contexts (Landa, 2005).

Limitations and directions for further research

The study was limited by its sample size, the briefness of the questionnaire used to assess children’s intentional communicative abilities, the use of different instruments to measure the same concept, and consequently the use of age equivalents in our statistical analyses. The retained regression models should, therefore, be validated on independent, larger samples in future studies. The regression models explained a significant but relatively small amount of variance in internalizing and externalizing problem behavior, indicating that other important factors—besides maturity and intentional communication—were not considered in this study. Future research should include additional child-specific factors (e.g. temperament (Gartstein, Putnam, & Rothbart, 2012)) as well as contextual factors (e.g. family SES, maternal well-being, and parenting practices (Bretherton et al., 2014; Rijlaarsdam et al., 2013)) to assess their relative importance in the development of problem behavior. In addition, information from different informants will help to avoid confirmation biases associated with parent or teacher ratings across domains of functioning (Law & Stringer, 2014). As many of the participants were not enrolled in day-care centers or preschool programs, this was not possible in the current study. Longitudinal studies are needed to examine whether the relative importance of maturity, language, and intentional communication varies with age and whether these associations are specific to children with neurodevelopmental disorders. Moreover, longitudinal studies may identify developmental trajectories of risk and resiliency which may differ for specific groups of children (Carpenter & Drabick, 2011).

Conclusions

To the best of our knowledge, this study is among the first to examine the relative contribution of language and intentional communication to problem behavior in a heterogeneous group of 24- to 46-month-old children referred with language difficulties. The results converged with previous studies which reported on the co-occurrence of language difficulties and problem behavior. Parent-rated intentional communication was the most important predictor of internalizing problem behavior and played an important role in the prediction of aggressive behavior as well. However, maturity also predicted externalizing problem behavior, especially with regard to attention problems.

Footnotes

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was funded by a doctoral dissertation grant from the KU Leuven awarded to Rianne Jansen and a grant from the Research Council of the KU Leuven (IDO/080/013) awarded to Jean Steyaert and Ilse Noens.

ORCID iD

Jarymke Maljaars

References

Achenbach

T. M.

(1992). Manual for the Child Behavior Checklist/2-3 and 1992 profile. Burlington: University of Vermont, Department of Psychiatry.

Achenbach

T. M.

Rescorla

L. A.

(2000). Manual for the ASEBA preschool forms & profiles. Burlington: University of Vermont, Research Center for Children, Youth, & Families.

Achenbach, T. M., & Rescorla, L. A. (2010). Multicultural supplement to the Manual for the ASEBA Preschool Forms & Profiles. Burlington, VT: University of Vermont, Research Center for Children, Youth, & Families.

Adams

(2002). Practitioner review: The assessment of language pragmatics. Journal of Child Psychology and Psychiatry, 43, 973–987. doi:10.1111/1469-7610.00226

Adams

Lloyd

(2005). Elicited and spontaneous communicative functions and stability of conversational measures with children who have pragmatic language impairments. International Journal of Language & Communication Disorders, 40, 333–347. doi:10.1080/13682820400027768

American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders (5th ed.). Arlington, VA: American Psychiatric Publishing.

Bauminger

Solomon

Rogers

S. J.

(2010). Externalizing and internalizing behaviors in ASD. Autism Research, 3, 101–112. doi:10.1002/aur.131

Beernink

A.-C. E.

Swinkels

S. H. N.

Van der Gaag

R. J.

Buitelaar

J. K.

(2012). Effects of attentional/hyperactive and oppositional/aggressive problem behaviour at 14 months and 21 months on parenting stress. Child and Adolescent Mental Health, 17, 113–120. doi:10.1111/j.1475-3588.2011.00616.x

Beitchman

J. H.

Hood

Rochon

Peterson

(1989). Empirical classification of speech/language impairment in children II. Behavioral characteristics. Journal of the American Academy of Child & Adolescent Psychiatry, 28, 118–123. doi:10.1097/00004583-198901000-00022

10.

Beitchman

J. H.

Wilson

Brownlie

E. B.

Walters

Inglis

Lancee

(1996). Long-term consistency in speech/language profiles: II. Behavioral, emotional, and social outcomes. Journal of the American Academy of Child & Adolescent Psychiatry, 35, 815–825. doi:10.1097/00004583-199606000-00022

11.

Benasich

A. A.

Curtiss

Tallal

(1993). Language, learning, and behavioral disturbances in childhood: A longitudinal perspective. Journal of the American Academy of Child & Adolescent Psychiatry, 32, 585–594. doi:10.1097/00004583-199305000-00015

12.

Benner

G. J.

Nelson

J. R.

Epstein

M. H.

(2002). Language skills of children with EBD: A literature review. Journal of Emotional and Behavioral Disorders, 10, 43–56. doi:10.1177/106342660201000105

13.

Bishop

D. V. M.

(2014). Ten questions about terminology for children with unexplained language problems. International Journal of Language & Communication Disorders, 49, 381–415. doi:10.1111/1460-6984.12101

14.

Bishop

D. V. M.

(2017). Why is it so hard to reach agreement on terminology? The case of developmental language disorder (DLD). International Journal of Language and Communication Disorders, 52, 671–680. doi:10.1111/1460-6984.12335

15.

Bishop

D. V. M.

Adams

(1991). What do referential communication tasks measure? A study of children with specific language impairment. Applied Psycholinguistics, 12, 199–215. doi:10.1017/S0142716400009140

16.

Bishop

D. V. M.

Snowling

M. J.

Thompson

P. A.

Greenhalgh

, & the CATALISE-2 Consortium. (2017). Phase 2 of CATALISE: A multinational and multidisciplinary Delphi consensus study of problems with language development: Terminology. Journal of Child Psychology and Psychiatry, 58, 1068–1080. doi:10.1111/jcpp.12721

17.

Bornstein

M. H.

Hahn

C.-S.

Suwalsky

J. T. D.

(2013). Language and internalizing and externalizing behavioral adjustment: Developmental pathways from childhood to adolescence. Development and Psychopathology, 25, 857–878. doi:10.1017/S0954579413000217

18.

Bottema-Beutel

(2016). Associations between joint attention and language in autism spectrum disorder and typical development: A systematic review and meta-regression analysis. Autism Research, 9, 1021–1035. doi:10.1002/aur.1624

19.

Bretherton

Prior

Bavin

Cini

Eadie

Reilly

(2014). Developing relationships between language and behaviour in preschool children from the early language in Victoria Study: Implications for intervention. Emotional and Behavioural Difficulties, 19, 7–27. doi:10.1080/13632752.2013.854956

20.

Buschmann

Jooss

Rupp

Dockter

Blaschtikowitz

Heggen

Pietz

(2008). Children with developmental language delay at 24 months of age: Results of a diagnostic work-up. Developmental Medicine & Child Neurology, 50, 223–229. doi:10.1111/j.1469-8749.2008.02034.x

21.

Camarata

(2014). Early identification and early intervention in autism spectrum disorders: Accurate and effective? International Journal of Speech-Language Pathology, 16, 1–10. doi:10.3109/17549507.2013.858773

22.

Campbell

S. B.

(1995). Behavior problems in preschool children: A review of recent research. Journal of Child Psychology and Psychiatry, 36, 113–149. doi:10.1111/j.1469-7610.1995.tb01657.x

23.

Carpenter

J. L.

Drabick

D. A. G.

(2011). Co-occurrence of linguistic and behavioural difficulties in early childhood: A developmental psychopathology perspective. Early Child Development and Care, 181, 1021–1045. doi:10.1080/03004430.2010.509795

24.

Carr

E. G.

Durand

V. M.

(1985). Reducing behavior problems through functional communication training. Journal of Applied Behavior Analysis, 18, 111–126. doi:10.1901/jaba.1985.18-111

25.

Carson

D. K.

Klee

Lee

Williams

K. C.

Perry

C. K.

(1998a). Children’s language proficiency at ages 2 and 3 as predictors of behavior problems, social and cognitive development at age 3. Journal of Children’s Communication Development, 19, 21–30. doi:10.1177/152574019801900203

26.

Carson

D. K.

Klee

Perry

C. K.

Muskina

Donaghy

(1998b). Comparisons of children with delayed and normal language at 24 months of age on measures of behavioral difficulties, social and cognitive development. Infant Mental Health Journal, 19, 59–75. doi:10.1002/(SICI)1097-0355(199821)19:1<59::AID-IMHJ4>3.0.CO;2-V

27.

Carter

A. S.

Briggs-Gowan

M. J.

Ornstein Davis

(2004). Assessment of young children’s social-emotional development and psychopathology: Recent advances and recommendations for practice. Journal of Child Psychology and Psychiatry, 45, 109–134. doi:10.1046/j.0021-9630.2003.00316.x

28.

Chow

J. C.

Wehby

J. H.

(2018). Associations between language and problem behavior: A systematic review and correlational meta-analysis. Educational Psychology Review, 30, 61–82. doi:10.1007/s10648-016-9385-z

29.

Constantino

J. N.

Gruber

C. P.

(2005). Social Responsiveness Scale (SRS). Torrance, CA: Western Psychological Services.

30.

Curtis

P. R.

Frey

J. R.

Watson

C. D.

Hampton

L. H.

Roberts

M. Y.

(2018). Language disorders and problem behaviors: A meta-analysis. Pediatrics, 142, e20173551. doi:10.1542/peds.2017-3551

31.

Desmarais

Sylvestre

Meyer

Bairati

Rouleau

(2008). Systematic review of the literature on characteristics of late-talking toddlers. International Journal of Language & Communication Disorders, 43, 361–389. doi:10.1080/13682820701546854

32.

Dominick

Ornstein Davis

Lainhart

Tager-Flusberg

Folstein

(2007). Atypical behaviors in children with autism and children with a history of language impairments. Research in Developmental Disabilities, 28, 145–162. doi:10.1016/j.ridd.2006.02.003

33.

Dunlap

Strain

P. S.

Fox

Carta

J. J.

Conroy

Smith

B. J.

, … Sowell

(2006). Prevention and intervention with young children’s challenging behavior: Perspectives regarding current knowledge. Behavioral Disorders, 32, 29–45.

34.

Egberink

I. J. L.

Vermeulen

C. S. M.

Frima

R. M.

(2014). COTAN beoordeling 2002, Nederlandstalige NonSpeech Test [COTAN review 2002, Dutch version of the NonSpeech Test for Receptive and Expressive Language]. Retrieved from http://www.cotandocumentatie.nl

35.

Norrelgen

Westerlund

Dahlman

Hultby

Fernell

(2012). Teenage outcomes after speech and language impairment at preschool age. Neuropsychiatric Disease and Treatment, 8, 221–227. doi:10.2147/NDT.S30106

36.

Fuchs

Klein

A. M.

Otto

von Klitzing

(2013). Prevalence of emotional and behavioral symptoms and their impact on daily life activities in a community sample of 3 to 5-year-old children. Child Psychiatry & Human Development, 44, 493–503. doi:10.1007/s10578-012-0343-9

37.

Gallagher

T. M.

(1999). Interrelationships among children’s language, behavior, and emotional problems. Topics in Language Disorders, 19(2), 1–15.

38.

Gartstein

M. A.

Putnam

S. P.

Rothbart

M. K.

(2012). Etiology of preschool behavior problems: Contributions of temperament attributes in early childhood. Infant Mental Health Journal, 33, 197–211. doi:10.1002/imhj.21312

39.

Gerow

Davis

Radhakrishnan

Gregori

Rivera

(2018). Functional communication training: The strength of evidence across disabilities. Exceptional Children, 85, 86–103. doi:10.1177/0014402918793399

40.

Gillberg

(2010). The ESSENCE in child psychiatry: Early symptomatic syndromes eliciting neurodevelopmental clinical examinations. Research in Developmental Disabilities, 31, 1543–1551. doi:10.1016/j.ridd.2010.06.002

41.

Gotham

Pickles

Lord

(2009). Standardizing ADOS scores for a measure of severity in autism spectrum disorders. Journal of Autism and Developmental Disorders, 39, 693–705. doi:10.1007/s10803-008-0674-3

42.

Henrichs

Rescorla

Donkersloot

Schenk

J. J.

Raat

Jaddoe

V. W. V.

, … Tiemeier

(2013). Early vocabulary delay and behavioral/emotional problems in early childhood: The Generation R Study. Journal of Speech, Language, and Hearing Research, 56, 553–566. doi:10.1044/1092-4388(2012/11-0169)

43.

Hill

E. L.

(2001). Non-specific nature of specific language impairment: A review of the literature with regard to concomitant motor impairments. International Journal of Language & Communication Disorders, 36, 149–171. doi:10.1080/13682820010019874

44.

Hollo

Wehby

J. H.

Oliver

R. M.

(2014). Unidentified language deficits in children with emotional and behavioral disorders: A meta-analysis. Exceptional Children, 80, 169–186. doi:10.1177/001440291408000203

45.

Hooper

S. R.

Roberts

J. E.

Zeisel

S. A.

Poe

(2003). Core language predictors of behavioral functioning in early elementary school children: Concurrent and longitudinal findings. Behavioral Disorders, 29, 10–24. doi:10.1177/019874290302900103

46.

Ivanova

M. Y.

Achenbach

T. M.

Rescorla

L. A.

Harder

V. S.

Ang

R. P.

Bilenberg

, … Verhulst

F. C.

(2010). Preschool psychopathology reported by parents in 23 societies: Testing the seven-syndrome model of the Child Behavior Checklist for ages 1.5–5. Journal of the American Academy of Child & Adolescent Psychiatry, 49, 1215–1224. doi:10.1016/j.jaac.2010.08.019

47.

Jansen

Ceulemans

Grauwels

Maljaars

Zink

Steyaert

Noens

(2013). Young children with language difficulties: A dimensional approach to subgrouping. Research in Developmental Disabilities, 34, 4415–4124. doi:10.1016/j.ridd.2013.08.028

48.

Ketelaars

M. P.

Cuperus

J. M.

Van Daal

Jansonius

Verhoeven

(2009). Screening for pragmatic language impairment: The potential of the Children’s Communication Checklist. Research in Developmental Disabilities, 30, 952–960. doi:10.1016/j.ridd.2009.01.006

49.

Koot

H. M.

Van den Oord

E. J. C. G.

Verhulst

F. C.

Boomsma

D. I.

(1997). Behavioral and emotional problems in young preschoolers: Cross-cultural testing of the validity of the Child Behavior Checklist/2–3. Journal of Abnormal Child Psychology, 25, 183–196. doi:10.1023/A:1025791814893

50.

Kozlowski

A. M.

Matson

J. L.

Horovitz

Worley

J. A.

Neal

(2011). Parents’ first concerns of their child’s development in toddlers with autism spectrum disorders. Developmental Neurorehabilitation, 14, 72–78. doi:10.3109/17518423.2010.539193

51.

Landa

R. J.

(2005). Assessment of social communication skills in preschoolers. Mental Retardation and Developmental Disabilities Research Reviews, 11, 247–252. doi:10.1002/mrdd.20079

52.

Law

Rush

McBean

(2014). The relative roles played by structural and pragmatic language skills in relation to behaviour in a population of primary school children from socially disadvantaged backgrounds. Emotional and Behavioural Difficulties, 19, 28–40. doi:10.1080/13632752.2013.854960

53.

Law

Stringer

(2014). The overlap between behaviour and communication and its implications for mental health in childhood: The elephant in the room. Emotional and Behavioural Difficulties, 19, 2–6. doi:10.1080/13632752.2013.854959

54.

Lindsay

Dockrell

J. E.

Strand

(2007). Longitudinal patterns of behaviour problems in children with specific speech and language difficulties: Child and contextual factors. British Journal of Educational Psychology, 77, 811–828. doi:10.1348/000709906X171127

55.

Long

C. E.

Gurka

M. J.

Blackman

J. A.

(2008). Family stress and children’s language and behavior problems: Results from the national survey of children’s health. Topics in Early Childhood Special Education, 28, 148–157. doi:10.1177/0271121408318678

56.

Lord

Rutter

DiLavore

P. C.

Risi

Gotham

Bishop

(2012). Autism diagnostic observation schedule, second edition (ADOS-2): Manual. Part I: Modules 1–4. Torrance, CA: Western Psychological Services.

57.

Mackie

Law

(2014). The functional communication skills of boys with externalizing behaviour with and without co-occurring language difficulties. Emotional and Behavioural Difficulties, 19, 89–105. doi:10.1080/13632752.2013.854961

58.

Maljaars

Noens

Jansen

Scholte

Van Berckelaer-Onnes

(2011). Intentional communication in nonverbal and verbal low-functioning children with autism. Journal of Communication Disorders, 44, 601–614. doi:10.1016/j.jcomdis.2011.07.004

59.

McCabe

P. C.

(2005). Social and behavioral correlates of preschoolers with specific language impairment. Psychology in the Schools, 42, 373–387. doi:10.1002/pits.20064

60.

Miniscalco

Nygren

Hagberg

Kadesjö

Gillberg

(2006). Neuropsychiatric and neurodevelopmental outcome of children at age 6 and 7 years who screened positive for language problems at 30 months. Developmental Medicine & Child Neurology, 48, 361–366. doi:10.1017/S0012162206000788

61.

Mirenda

(1997). Supporting individuals with challenging behavior through functional communication training and AAC: Research review. Augmentative and Alternative Communication, 13, 207–225. doi:10.1080/07434619712331278048

62.

Noens

Van Berckelaer-Onnes

(2004). Making sense in a fragmentary world: Communication in people with autism and learning disability. Autism, 8, 197–218. doi:10.1177/1362361304042723

63.

Powell

Dunlap

Fox

(2006). Prevention and intervention for the challenging behaviors of toddlers and preschoolers. Infants & Young Children, 19, 25–35.

64.

Reilly

Tomblin

Law

McKean

Mensah

F. K.

Morgan

, … Wake

(2014). Specific language impairment: A convenient label for whom? International Journal of Language and Communication Disorders, 49, 416–451. doi:10.1111/1460-6984.12102

65.

Rescorla

Alley

(2001). Validation of the Language Development Survey (LDS): A parent report tool for identifying language delay in toddlers. Journal of Speech, Language, and Hearing Research, 44, 434–445. doi:10.1044/1092-4388(2001/035)

66.

Rescorla

Ross

G. S.

McClure

(2007). Language delay and behavioral/emotional problems in toddlers: Findings from two developmental clinics. Journal of Speech, Language, and Hearing Research, 50, 1063–1078. doi:10.1044/1092-4388(2007/074)

67.

Rijlaarsdam

Stevens

G. W. J. M.

Van der Ende

Hofman

Jaddoe

V. W. V.

Mackenbach

J. P.

, … Tiemeier

(2013). Economic disadvantage and young children’s emotional and behavioral problems: Mechanisms of risk. Journal of Abnormal Child Psychology, 41, 125–137. doi:10.1007/s10802-012-9655-2

68.

Roben

C. K. P.

Cole

P. M.

Armstrong

L. M.

(2013). Longitudinal relations among language skills, anger expression, and regulatory strategies in early childhood. Child Development, 84, 891–905. doi:10.1111/cdev.12027

69.

Rodas

N. V.

Eisenhower

Blacher

(2017). Structural and pragmatic language in children with ASD: Longitudinal impact on anxiety and externalizing behaviors. Journal of Autism and Developmental Disorders, 47, 3479–3488. doi:10.1007/s10803-017-3265-3

70.

Roy

Chiat

(2014). Developmental pathways of language and social communication problems in 9–11 year olds: Unpicking the heterogeneity. Research in Developmental Disabilities, 35, 2534–2546. doi:10.1016/j.ridd.2014.06.014

71.

Schaerlaekens

Zink

Van Ommeslaeghe

(2003). Reynell Taalontwikkelingsschalen. Handleiding, tweede versie. Lisse, The Netherlands: Swets & Zeitlinger.

72.

Sigafoos

(2000). Communication development and aberrant behavior in children with developmental disabilities. Education and Training in Mental Retardation and Developmental Disabilities, 35, 168–176.

73.

St Clair

M. C.

Pickles

Durkin

Conti-Ramsden

(2011). A longitudinal study of behavioral, emotional and social difficulties in individuals with a history of specific language impairment (SLI). Journal of Communication Disorders, 44, 186–199. doi:10.1016/j.jcomdis.2010.09.004

74.

Stanton-Chapman

T. L.

Justice

L. M.

Skibbe

L. E.

Grant

S. L.

(2007). Social and behavioral characteristics of preschoolers with specific language impairment. Topics in Early Childhood Special Education, 27, 98–109. doi:10.1177/02711214070270020501

75.

Stevens

(2002). Applied multivariate statistics for the social sciences (4th ed.). Hillsdale, NJ: Erlbaum.

76.

Tellegen

P. J.

Laros

J. A.

(2013). Snijders-Oomen Niet-verbale intelligentietest III. Nederlands-Duitse normen 2010. [Snijders-Oomen Non-verbal Intelligence Test III. Dutch-German standardization data 2010]. Göttingen, Germany: Hogrefe.

77.

Tellegen

P. J.

Winkel

Wijnberg-Williams

B. J.

Laros

J. A.

(1998). Snijders-Oomen Niet-verbale intelligentietest, SON-R 2½–7: Handleiding en verantwoording [Snijders-Oomen Non-verbal Intelligence Test, SON-R 2½–7: Manual]. Lisse, The Netherlands: Swets Test Publishers. Tremblay

R. E.

Japel

Perusse

Mcduff

Boivin

Zoccolillo

Montplaisir

(1999). The search for the age of ‘onset’ of physical aggression: Rousseau and Bandura revisited. Criminal Behavior and Mental Health, 9, 8–23. doi:10.1002/cbm.288

78.

Van der Meulen

B. F.

Ruiter

S. A. J.

Lutje Spelberg

H. C.

Smrkovsky

(2002). Bayley Scales of Infant Development – Second edition – Nederlandse bewerking [Bayley Scales of Infant Development – Second edition – Dutch version]. Lisse, The Netherlands: Swets Test Publishers.

79.

Vaughan

E. L.

Feinn

Bernard

Brereton

Kaufman

J. S.

(2013). Relationships between child emotional and behavioral symptoms and caregiver strain and parenting stress. Journal of Family Issues, 34, 534–556. doi:10.1177/0192513X12440949

80.

Volden

Phillips

(2010). Measuring pragmatic language in speakers with autism spectrum disorders: Comparing the Children’s Communication Checklist-2 and the test of pragmatic language. American Journal of Speech-Language Pathology, 19, 204–212. doi:10.1044/1058-0360(2010/09-0011)

81.

Walker

V. L.

Snell

M. E.

(2013). Effects of augmentative and alternative communication on challenging behavior: A meta-analysis. Augmentative and Alternative Communication, 29, 117–131. doi:10.3109/07434618.2013.785020

82.

Wetherby

Prizant

(1993). Communication and symbolic behavior scales – Normed edition. Baltimore, MD: Paul H. Brookes.

83.

Wetherby

Prizant

(2002). Communication and symbolic behavior scales: Developmental profile – First normed edition. Baltimore, MD: Paul H Brookes.

84.

Williams

M. N.

Grajales

C. A. G.

Kurkiewicz

(2013). Assumptions of multiple regression: Correcting two misconceptions. Practical Assessment, Research, & Evaluation, 18, 11.

85.

Yew

S. G. K.

O’Kearney

(2013). Emotional and behavioural outcomes later in childhood and adolescence for children with specific language impairments: Meta-analyses of controlled prospective studies. Journal of Child Psychology and Psychiatry, 54, 516–624. doi:10/1111/jcpp/12009

86.

Zink

Lejaegere

(2003). N-CDIs: Korte vormen [N-CDIs: Short forms]. Leuven, Belgium: Acco.

87.

Zink

Lejaegere

(2007). N-CDI 3. Leuven, Belgium: Acco.

88.

Zink

Lembrechts

(2000). NNST: Nederlandstalige NonSpeech Test. Handleiding [NNST: Dutch version of the NonSpeech test for receptive and expressive language. Manual]. Leuven, Belgium: Acco.

89.

Zink

Van Ommeslaeghe

Stroobants

Janssen

Schaerlaekens

(1993). Reynell Taalontwikkelingsschalen. Psychometrische verantwoording en analyse [Reynell scales for language development. Psychometric data]. Nijmegen, The Netherlands: Berkhout Nijmegen B.V.