Abstract
Valid information on early social-emotional competence is essential to diagnose, treat, and prevent behavioral problems in children and adolescents. Particularly in young children, social-emotional competence is frequently measured using parent and teacher ratings that frequently exhibit low agreement. Therefore, the present study on n = 532 three-year-olds (47% girls) examined whether sibling status might explain discrepancies between the two informant groups. First, multi-trait multi-informant analyses explored the construct validity of a short measure of three facets of social-emotional competence. Then, group comparisons evaluated the size of the observed method effects for only children and children with siblings. Results showed low convergent validity between parent and teacher ratings for aggressive behavior, cooperative behavior, and emotional self-regulation. Sibling status in the family contributed little to the observed discrepancies between parents and teachers. Thus, a comprehensive assessment of social-emotional competence in children requires a multi-informant approach to capture the construct breadth.
Keywords
Social-emotional competence consists of various skills, knowledge, and abilities facilitating socially competent behavior (Kanning, 2002). The foundations of social-emotional competence are already laid in early childhood. Developmental problems or delays as early as age three can lead to problems up to adolescence and even adulthood. For example, high social-emotional competence throughout the life course has been repeatedly shown to predict social and academic success (e.g., Barry & Wigfield, 2002; Denham et al., 2014; 2009; Greco & Morris, 2005; Stepp et al., 2011). Therefore, it is crucial to implement intervention strategies at an early stage and, thus, to validly assess social-emotional competence as early as kindergarten. Researchers typically rely on observer ratings from different informants (multi-informant perspective) such as kindergarten teachers and parents to obtain an overall impression of a child’s social-emotional competence. However, parent and teacher ratings of children’s social-emotional competence often correlate rather poorly (Achenbach et al., 1987; Rescorla et al., 2012). Low correlations could be problematic if decisions about early interventions, such as delaying school entry, behavioral therapy, or psychotherapy, are made based on a single informant assessment. Reasons for these low correlations might be, for example, different social settings that different informants observe or informant bias. As of yet, no study has considered sibling status as a source of informant bias, although stereotypes for children with and without siblings, such as only children being less socially competent, exist (Thompson, 1974). When considering the ratings of social-emotional competence and sibling status in correlation analyses, it is not clear whether the different ratings are due to a child’s social-emotional development being affected by growing up with or without a sibling or whether they are based on underlying stereotypes that lead to different ratings. By applying a multi-trait–multimethod model in a latent-variable framework (Eid, 2000; Eid et al., 2003) to parent and teacher ratings of kindergarteners, the present study not only examines the construct validity of three sub-dimensions of social-emotional competence (aggressive behavior, cooperative behavior, and emotional self-regulation) but also whether method effects differ because of sibling status.
Social-Emotional Competence
Social-emotional competence is defined as the effectiveness in interaction and develops from early childhood onwards (Denham et al., 2014). It consists of cognitive, emotional, and behavioral aspects (Denham, 2006). Socially and emotionally competent individuals are able to reach their own goals in a socially accepted manner over time and across situations (Kanning, 2002). Social-emotional competence fosters friendships (Barry & Wigfield, 2002), decreases social anxieties, and even reduces the propensity to commit crimes (Greco & Morris, 2005; Stepp et al., 2011). In the educational context, it is positively associated with a person’s educational success (Denham et al., 2009, 2014).
Children as young as 3 years face many new tasks that refer to several sub-dimensions of social-emotional competence. They have to integrate themselves into a group of peers, learn new norms and rules of conduct, and make compromises (Denham et al., 2009). Therefore, this study focused on three sub-dimensions of social and emotional competence that develop early and are strongly related to each other: (1) cooperative behavior and (2) (non-)aggressive behavior are indicators of external social-emotional competence, where one’s social behavior is oriented on other’s goals and needs. (3) Emotional self-regulation refers to internal processes of social-emotional competence such as the awareness, regulation, and expression of own emotions but also the understanding of others' emotions.
Agreement between Informants and Potential Influences of Siblings
Many studies (Dinnebeil et al., 2013; Fält et al., 2018) and metastudies (Achenbach et al., 1987; Renk & Phares, 2004; Rescorla et al., 2012) on social and disruptive behavior of kindergarteners, revealed low correlations between ratings from teachers and parents. These studies showed that correlations were higher for externalizing than internalizing problem behavior (Rescorla et al., 2012; Winsler & Wallace, 2002). Also, higher informant-correlations has been found for 6–to 11-year-olds than for adolescents or kindergarteners (Achenbach et al., 1987; Renk & Phares, 2004), for girls than for boys (Gagnon et al., 1992; Grills & Ollendick, 2003), and for higher educated mothers as raters (Gagnon et al., 1992). So far, the effect of sibling status on independent ratings has been hardly investigated. To our knowledge, only one study used independent ratings to assess the social-emotional competence of siblings: Downey and Condron (2004) investigated the impact of siblings on social-emotional competence using parent and teacher ratings. They report that kindergarteners with one or two siblings were perceived as more socially competent by the teachers than children with none or more than two siblings. No such differences were found for parent ratings (Downey & Condron, 2004).
Considering the multidimensionality of assessments due to different informants, it is unclear if the sibling status affects children’s true social-emotional competence or the informant bias: On the one hand, only children might show higher social-emotional competence than children with siblings or vice versa. On the other hand, informant biases (differences in the method variance) could be moderated because the child for assessment is an only child or a child with siblings. Reasons for this are manifold. First, it is unclear whether parents and teachers were affected by stereotypes regarding their ratings and whether they were influenced equally. Only children were often associated with negative stereotypes such as being selfish, lonely, socially estranged, self-centered, unlikable, or maladjusted (e.g., Polit & Falbo, 1987; Sulloway, 1995; Thompson, 1974). Second, families with more than one child could have more comparison possibilities within their family than families with only one child. The assessments of parents could therefore vary across groups.
Research Questions
Until now, only a few studies have investigated the influence of informant bias (method factor) for assessments of social-emotional competence of kindergarteners (Ferreira et al., 2021; Low et al., 2015; Yu et al., 2015). Particularly, sibling status has been neglected in previous studies as a reason for informant bias. Consequently, we addressed the following questions: 1. Do kindergarten teachers and parents rate children’s social-emotional competence comparably, or do informant-specific effects bias the assessment? 2. Are multi-informant assessments of social-emotional competence comparable for only children and children with siblings, or does sibling status affect informant-specific effects?
Method
Participants
The data were part of a larger research project (see Weinert et al., 2013) and originally included 547 three-year-old children from two German federal states (Bavaria, Hesse). Because for 14 children no valid responses were observed, the analyzed sample reduced to n = 532 children (M = 39.2 months; SD = 0.2; 47% girls) that each were evaluated by one kindergarten teacher (M = 39.4 years; SD = 0.5; 94% women) and one parent (M = 34.6 years; SD = 0.2; 95% women). About 23% (n = 120) of the children had no siblings, while for the rest (n = 412) the median was 1 (min = 1, max = 5) biological, adopted, foster, or stepsibling. 29% (n = 119) of children with siblings were firstborns and around 4% (n = 16) were multiples (twins). Sociodemographic differences between the two child groups were negligible (see Table S1 in the supplemental material), with a slightly lower percentage of girls among only children (45% vs. 48%) but a comparable mean age (Cohen’s d = 0.09).
Measure
Items of the Social-Emotional Competence Scale.
Note. * Negatively worded items were reverse scored.
McDonald’s Omega Reliabilities for Social-Emotional Competence Scales by Informant.
Note. n = 532 children in total, n = 120 only children and n = 412 children with siblings.
Analytical Approach
The construct validity of the social-emotional competence scales was examined using multi-trait multi-method (MTMM) analyses in a confirmatory factor analytic (CFA) framework. Following Eid and colleagues (2003), we estimated a correlated trait—correlated method minus-one (CTC(M-1)) model that specified three correlated trait factors for cooperative behavior, aggressive behavior, and emotional self-regulation and three correlated method factors for parent ratings (see Figure 1). In this approach, a reference method must be selected based on theoretical assumptions. Teacher ratings were selected as a reference method due to their professional training and daily routine with children. In contrast, the method factors presented unique variances in parent ratings. We modeled different method factors for each subscale to examine the degree of method effects generalized across scales. Trait and method factors were allowed to correlate among themselves but not with each other. From this model, two indices were derived (Eid et al., 2003): (a) the consistency coefficient reflected the part of the variance of the non-reference method indicator explained by the comparison standard, that is, how well differences detected in parent ratings can be predicted by differences in the teacher ratings; (b) the method-specificity coefficient represented the unexplained part of the variance of a non-reference method indicator, that is, the influence of a specific method, in our case, the parent. Consistency and method-specificity coefficients in a CTC(M-1) model can be calculated for observed and true scores (Eid et al., 2003). CTC(M-1) Model for Social-Emotional Competence. Note. Agg = aggressive behavior, Coop = cooperative behavior, Emo = emotional self-regulation, T = Trait factors, M = Method factors, Teacher and parent rating of aggressive behavior (AT1-3 & AP1-3), cooperative behavior (CT1-3 & CP1-3), and emotional self-regulation (ET1-3 & EP1-3).
Convergent validity can be inferred if the latent correlations between teacher and parent ratings, that is, the square root of the consistency coefficients (Eid et al., 2003), are large, thus, indicating that parents and teachers rated children similarly. Moreover, consistency coefficients should be larger than the method-specificity coefficient. In contrast, discriminant validity can be inferred if the correlations between the latent trait factors in the CTC(M-1) model are lower than 1.00, that is, r < .85 (Brown, 2006). Moreover, correlations between the method factors for different traits show whether the method effects generalize across scales and parents consistently rate differently compared to teachers (Eid et al., 2003).
Differences in the construct validity of the administered scales for only children and children with siblings were studied using multi-group CFAs. We investigated measurement invariance (Steenkamp & Baumgartner, 1998; Vandenberg & Lance, 2000) by comparing increasingly restrictive models. Configural measurement invariance was supported if the MTMM model (without any cross-group constraints) fitted comparably in both child groups (Meredith, 1993), while metric measurement invariance was inferred if cross-group constraints on the factor loadings did not impair the model fit. Scalar measurement invariance with constrained factor loadings and thresholds was also supported.
We used a full information maximum likelihood (Enders, 2010) estimator to handle missing values in all indicator variables (item non-response = 38%). The CFAs were estimated in Mplus version 7 (Muthén & Muthén, 1998-2012) with a weighted least square estimator with adjusted mean and variance χ2 test of model fit (WLSMV estimator; Nussbeck et al., 2006). Model fit was evaluated in line with prevalent standards interpreting comparative fit indices (CFI) ≥ .95, root mean square errors of approximation (RMSEA) ≤ .08, and weighted root mean square residuals (WRMR) ≤ 1.0 as “acceptable” and models with CFI ≥.97, RMSEA ≤.05, and WRMR ≤.90 as “good” fitting (DiStefano et al., 2018; Hu & Bentler, 1999). Model comparisons were based on Satorra-Bentler-χ2-difference tests (Satorra & Bentler, 2001) and differences in CFIs for which values ≤ −.01 indicated comparable models (Cheung & Rensvold, 2002).
Results
Multi-Trait Multi-Informant Analyses of Social-emotional Competence
Standardized Loading Parameters of the CTC(M-1)Model for the Total Sample and the Child Group Subsamples.
Note. Teacher and parent rating of aggressive behavior (AT1-3 & AP1-3), cooperative behavior (CT1-3 & CP1-3), and emotional self-regulation (ET1-3 & EP1-3). Blank cells indicate factor loadings fixed to zero by definition of the model. CT-C(M−1) = correlated trait–correlated method minus one model.
Convergent and Discriminant Validity.
Note. Teacher and parent rating of aggressive behavior (AT1-3 & AP1-3), cooperative behavior (CT1-3 & CP1-3), and emotional self-regulation (ET1-3 & EP1-3). n = 532. Con. = Consistency, Spec. = Method specificity, Corr. = Latent correlation with the standard method (√consistency).
The Role of Sibling Status
Model Fit of the Multi-Group CTC(M-1) Model.
Note. n = 120 only children and n = 412 children with siblings. * significant at p < .01.
Correlations of the Trait and Method Factors in the CTC(M − 1) Model for Child Group Subsamples.
Note. Correlations from the sibling’s sample are shown below the diagonal; correlations from the only child sample are shown above the diagonal; n = 120 only children and n = 412 children with siblings. Variance fixed to 1. * significant at p < .01.
Given threshold invariance was supported (see Table 5), we also examined mean level differences. Only children were evaluated significantly lower on aggressive behavior, d = .35, p < .05, emotional self-regulation, d = .51, p < .05, and co-operative behavior d = .32, p < .05. Moreover, all three method factors showed no significant (p > .05) mean-level differences.
Discussion
Previous research has shown that the correlation between children’s social-emotional competence ratings collected from two different informants is often relatively low (Dinnebeil et al., 2013; Rescorla et al., 2012). Reasons for these results could be different interpretations between informants, differences in social settings, or different thresholds for identifying behavior (De los Reyes et al., 2013). Still, only a few studies investigated these low correlations further (Ferreira et al., 2021; Low et al., 2015). In our MTMM analyses, it was possible to examine multi-informant assessments in more detail, which has not been done often to investigate assessments of children’s social-emotional competence (Ferreira et al., 2021; Gomez, 2014; Low et al., 2015; Yu et al., 2015). The present study led to three central findings. First, the administered scales exhibited rather good reliabilities, and metric invariance between child groups was supported. Second, the MTMM model demonstrated discriminant validity between traits in line with other studies (Ferreira et al., 2021; Gomez, 2014) but only limited convergent validity across informants. Third, despite a whole range of stereotypes about only children, sibling status has, so far, not been considered a reason for low agreement between informants. However, independent analyses for only children and children with siblings lead to similar results as for the total sample with adequate discriminant validity between traits but low convergent validity.
Our findings suggest that parents and teachers do indeed rate children’s competence differently. This could be due to different social contexts. Further analyses with different rating scales and more ratings from both social contexts would be needed to test these assumptions thoroughly. Until then, an important decision on the child’s future should not rely on one perspective only. Furthermore, results show no significant difference between ratings of only children and children with siblings. Only descriptive results indicate that parents with one or more than one child might apply different thresholds for the child’s competence. They, therefore, agree more or less with the teacher rating on each sub-dimension. For example, parent ratings of cooperative behavior of only children are more similar to teacher ratings. One reason for that might be that parents and teachers refer to the same or a similar social context where the child encounters peers. Another reason might be that scale properties such as positively versus negatively worded items lead to higher consistency between teachers and parents of only children. Finally, about one third of children with siblings were firstborns. These children are likely to resemble only children in that they were temporary only children, especially at the age of three when a sibling has just arrived.
Limitations
This research was not without limitations. The relatively small sample prohibited more detailed analyses based on this data, such as the effects of the number of siblings, birth order, or birth spacing on the ratings of social-emotional competence of parents and teachers. One shortcoming of the method is that results are not symmetrical. By changing the reference method from teacher rating to parent rating, fit indices might be different (Eid et al., 2003). However, secondary analyses with a changed reference method confirmed the robustness of our main results (see Tables S5–S8 in the supplemental material).
Conclusion
Our results indicate that kindergarten teachers and parents do not rate the competence of children comparably. This could be attributed to the fact that the informants observe children in different social situations. Additionally, our results show no general differences in regard to sibling status. However, descriptive results indicate minor differences in the comparability of parent and teacher ratings by sibling status as a function of the sub-dimensions of social-emotional competence. Therefore, sibling status explains only a small, not significant, part of the low agreement between informants.
Supplemental Material
sj-pdf-1-jpa-10.1177_07342829221077503 – Supplemental Material for Parent and Teacher Assessments of Social-emotional Competence in Three-Year-Old Children: Does Sibling Status Matter?
Supplemental Material, sj-pdf-1-jpa-10.1177_07342829221077503 for Parent and Teacher Assessments of Social-emotional Competence in Three-Year-Old Children: Does Sibling Status Matter? in Carina Schönmoser, Claudia Karwath, and Timo Gnambs in Journal of Psychoeducational Assessment
Footnotes
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
