Moral emotions in early childhood: Validation of the Moral Emotions Questionnaire (MEQ)

Abstract

Moral emotions are experienced in daily life and are crucial for mediating appropriate social behaviors, as they prevent individuals from committing transgressions. In this study, caregivers of 377 children aged between 2.5 and 6.5 years old completed the Moral Emotions Questionnaire (MEQ), a parent report aimed to separately identify the presence of shame, guilt, and pride behaviors in early childhood. To validate this newly developed questionnaire, a confirmatory factor analysis and measurement invariance were conducted, and internal consistency, and concurrent validity were tested. Outcomes confirmed that the three moral emotions can be individually identified through the MEQ, even at such an early age. The MEQ scales showed acceptable internal consistencies and the associations between the three moral emotions and externalizing behaviors, internalizing behaviors, and social competence were in accordance with previous research, therefore confirming concurrent validity.

Keywords

Childhood development emotion psychosocial development confirmatory factor analysis validation self-conscious emotions moral emotions

Introduction

Moral emotions arise when people feel judged, or expect to be judged by others, or judge their own behavior in terms of right and wrong. Within the spectrum of moral emotions, shame, guilt, and pride, are considered self-conscious emotions. Self-conscious emotions are those which require a self-evaluative process, in which the individual continuously evolves according to the social norms (Sznycer, 2019; Tracy & Robins, 2004a). The experience—or even just the anticipation—of these emotions serves as a behavior regulation mechanism, urging people to abide by the rules and to avoid committing moral transgressions (Blasi, 1999; Tangney et al., 2007). Moral emotions also serve a communicative function: when people express moral emotions, they openly acknowledge that their behavior was incongruent (as in guilt or in shame) or congruent (as in pride) with the group’s rules, norms, and values, and that they take responsibility for their behavior. This helps them to be accepted—or reaccepted—as a valuable group member (Shariff & Tracy, 2011; Stearns & Parrott, 2012).

Shame, guilt, and pride have been appointed as the more focal self-conscious emotions, their development and relation with other aspects of psychosocial functioning have long been studied (Sznycer, 2019; Tracy & Robins, 2004a). Lower levels of moral emotions have been associated with externalizing behaviors (e.g., bullying, aggression, delinquency, psychopathy) in early adolescence and adulthood (Blair et al., 2001; Holmqvist, 2008; Menesini & Camodeca, 2008; Olthof, 2012). These findings point to the importance of examining the development of moral emotions at the youngest possible age.

Moral understanding and morally guided behaviors can be observed early in life. For example, 2-year-old children are able to stop themselves from doing something that is prohibited, and then also hesitate after disobeying a rule (Kochanska & Aksan, 2006). Studies have confirmed that children as young as 2 or 3 years old already show behaviors that suggest the experience of moral emotions, such as signs of distress, avoiding eye contact, confessions, and/or reparative behaviors after a transgression (Barrett et al., 1993; Kochanska et al., 1995), as well as behaviors that suggest an attempt to attract attention when they achieved something on their own (Stipek et al., 1992). This is in line with the development of self-concept, which has been suggested to emerge around the 30 months of age (Bullock & Lutkenhaus, 1990). The development of moral emotions accompanies self-concept development, because to be able to evaluate whether their actions are congruent or incongruent with societal norms and values, children first need to be able to distinguish themselves from others and then focus the attention on their individual actions (Hart & Matsuba, 2007). This learning process, which seems to be a major importance on the first 4 years of life, occurs through the daily interactions that children have (Dahl & Killen, 2018a, 2018b). Since they are born, children experience and observe behaviors of aid and harm from their families toward them, and also between other people (Dahl, 2015; Hammond et al., 2017). These experiences help them shape their knowledge about right and wrong, and together with a constant development of their self-concept, allow them to guide their behaviors toward others (Dahl & Killen, 2018a, 2018b). To the best of our knowledge, no instrument is yet available that can facilitate the systematic study of the different moral emotions in the preschool years, when children first show signs of these emotions. Therefore, our aim was to develop and validate a questionnaire that can uniquely identify different moral emotions (shame, guilt, and pride) in early childhood.

Distinguishing Moral Emotions

Different moral emotions can be distinguished according to their associated social goals and behaviors. Shame and guilt could both arise from the same antecedent, namely, when a transgression is committed. In fact, the same transgression can evoke shame in one person and guilt in another (Lewis, 2014; Tangney et al., 2007). Shame arises when a transgressor thinks that the harm is irreparable, as it reflects a failure of the whole self, and/or that others will attribute to him or her a negative, unwanted identity (Barrett et al., 1993; Olthof, 2012). Shame can be observed through submissive behaviors (e.g., making oneself appear smaller, avoiding eye contact) and the avoidance of others’ attention by the transgressor. Although the action tendencies of trying to avoid the attention after children commit transgressions or cause harm to others are reported to already be exhibited by 2-year-old children (Barrett et al., 1993), research suggests that shame emerges after age 3, when children become more aware of themselves, and their behaviors in relation to social norms and rules (Kagan, 2005; Lewis, 1998). Shame prevents the individual from acting against social norms, thereby preventing negative judgment from others (Barrett, 1998a; Bedford & Hwang, 2003; Cole et al., 2006; Dempsey, 2017; Fung, 1999; Midlarsky et al., 2006; Yoshioka & Choi, 2005). Previous studies have shown that in older children, adolescents, and adults, excessive shame often leads to increased levels of anger and to persistent feelings of inferiority or failure, which can contribute to anxiety and depression (Bennett et al., 2005; Broekhof et al., 2018; Harper & Arias, 2004; Tangney et al., 2007).

In contrast, guilt arises if the transgressor has a sense of responsibility over the transgression. In this case, the damage only reflects his or her behavior in this specific situation and not in relation to the whole self (Barrett et al., 1993; Ferguson et al., 2000; Tracy & Robins, 2004a). Guilt elicits a need to repair harm done and restore the relationship. Thus, guilt serves as a prosocial behavior motivator (Dempsey, 2017). Guilt behaviors are usually aimed at drawing attention to the wrongdoing and trying to make up for it, as in confessing, making apologies, or trying to repair the damage (Tangney et al., 2007). A study conducted by Zahn-Waxler et al. (1992) has shown that action tendencies related to guilt start to manifest in children as young as 15 months of age, with children showing intent to repair the harm they caused to others. However, most studies report that these reparative behaviors became more prominent after 2 years of age (Barrett, 1998b; Cole et al., 1992; Kochanska et al., 1994, 2002), suggesting that guilt is developed throughout the second year of life. Previous studies have shown that children who more frequently experience guilt are more attentive and competent when dealing with others, show more prosocial behaviors, and have better quality friendships and relationships (Baumeister et al., 1994; Estrada-hollenbeck & Heatherton, 1995; Kochanska & Aksan, 2006). Conversely, children who show fewer guilt behaviors tend to exhibit more disruptive behaviors and show more aggression and conduct problems, which may negatively affect their social relationships (Frick & Morris, 2004; Kochanska & Aksan, 2006).

Pride arises when one feels responsible for accomplishing something that exceeds the expectations of others in a positive way, while also feeling internally positive about oneself (Orth et al., 2010; Tracy & Robins, 2004b, 2007). Pride may play an important role when establishing and maintaining social interactions with peers, as it not only promotes the value of one person in the eyes of others (Mauro et al., 1992), but is also internally rewarding to the individual. As such, it motivates one to act according to social norms to feel valued by others (Cheng et al., 2010; Sznycer et al., 2017, 2018; Williams & DeSteno, 2008). The behavioral components of pride (e.g., head held high, making eye contact) are aimed at attracting attention from significant others. Still, in this case, the attention is focused on oneself or one’s accomplishment (Tracy & Robins, 2004b). Children, who are 2 years of age, already manifest action tendencies related to pride, like calling for attention after achieving something by themselves (Hart & Matsuba, 2007; Stipek et al., 1992), which continuously developed throughout childhood. Previous studies have shown that pride increases children’s confidence in initiating interactions and resolving conflicts (Cheng et al., 2010; Mauro et al., 1992; Sznycer et al., 2017, 2018; Williams & DeSteno, 2008).

Assessment of Moral Emotions in Early Childhood

To date, not many instruments for measuring moral emotions in early childhood are available, despite the crucial role of moral emotions in the development of psychosocial functioning. Observational studies that measure responses to emotion-evoking events are the most commonly used method for assessing moral emotions in young children. In these studies, children are instructed to complete tasks that are either designed to allow them to succeed or to set them up to fail (Belsky et al., 1997; Kelley et al., 2000; Ketelaar et al., 2015). Observing children in staged situations or in field observations is very informative, but can be time consuming. Questionnaires allow researchers to obtain information from a large number of participants in a short amount of time. Moreover, given that young children spend a lot of time in the vicinity of their parents, parents can be a valuable source of information about their child’s moral behaviors across a variety of settings and situations with different people.

To the best of our knowledge, the only questionnaire to date that focuses on young children’s moral emotions is “The Conscience Measure Questionnaire” (also referred to as the “My Child” measure) by Kochanska et al. (1994). The “My Child” questionnaire is a parental report on children’s awareness of wrongdoing and their willingness to stop or repair an incorrect behavior (Kochanska et al., 1994). However, the “My Child” questionnaire does not capture pride, and seems mostly oriented toward guilt, not shame. Therefore, we aimed to develop a new parent report questionnaire addressing the three major moral emotions (i.e., guilt, shame, pride) in young children, while focusing on behavioral indices related to each emotion separately.

Current Study

The development of moral emotions is crucial for psychosocial development, with shame, guilt, and pride being related to distinct outcomes regarding externalizing and internalizing symptoms and social competence. However, no instruments that distinguish between these three moral emotions for children at their emergent age have been developed. Therefore, this study aimed to establish the factor structure, internal consistency, and validity of a newly developed parent report questionnaire for assessing shame, guilt, and pride in young children: the Moral Emotions Questionnaire (MEQ).

First, we examined whether the hypothesized three-factor structure of the MEQ was confirmed (shame, guilt, and pride) by conducting a confirmatory factor analysis (CFA). Second, we examined the internal consistencies of the individual scales from the MEQ. The third aim of this study was to establish the concurrent validity of the MEQ by examining relations with other aspects of social–emotional functioning (i.e., internalizing behaviors, externalizing behaviors, and social competence). Finally, measurement invariance was assessed across gender, and two age groups (under 4 years of age and 4 or older) to verify whether the responses of caregivers to the items were equivalent considering those factors.

In this questionnaire, shame is operationalized as the tendency to show submissive behavior or to withdraw or escape from a situation. Based on the literature, we hypothesized that higher levels of shame would be associated with higher levels of internalizing and externalizing problems (Ferguson et al., 1999; Tangney et al., 1992, 2014; Thomaes et al., 2011). Guilt was operationalized as an other-oriented, adaptive emotion. This emotion is associated with the urge to confess, apologize for, or repair a wrongdoing as a sign of remorse. Therefore, guilt was expected to be positively related to social competence and negatively related to externalizing problems (Ferguson et al., 1999; Roos et al., 2014; Stuewig et al., 2010; Tangney et al., 1996). Pride is operationalized as a tendency to draw the attention of others toward an individual accomplishment, and exhibit enjoyment in regards to others’ appraisals of oneself. Based on literature, behavioral indices for pride were expected to be associated with higher levels of social competence (Hooge et al., 2011; Kluwin et al., 2002; Mascolo & Fischer, 1995).

Method

Participants and Procedure

Caregivers of a total of 377 children aged between 2.5 and 6.5 years old (mean age = 54 months, SD = 13 months; 55% of boys) participated in the study. Mothers completed the questionnaire for 301 children; fathers completed the questionnaire for 37 children; for 11 children, both parents completed the questionnaires together; and for another 11 children, the questionnaire was completed by other caregivers. Mothers aged between 25 and 47 years old (mean age = 37 years, SD = 4 years), while fathers aged between 24 and 61 years old (mean age = 39, SD = 6 years). Concerning the socioeconomic status of the participating families, 22% did not report this information. From the families who replied the majority belonged to the middle level of socioeconomic status (44%), followed by families in the high level of economic status (25%), and finally by families with low economic status (9%). The participants were recruited via daycare centers, preschools, and elementary schools in the Netherlands. Children were excluded from the study if they had any apparent developmental delays or mental health disorders, such as attention-deficit hyperactivity disorder (ADHD) or autism spectrum disorders. Parents were informed about the goals and execution of the study, how data were to be handled and stored to guarantee their privacy, and about the voluntary nature of their participation. All parents provided written consent to participate in the study. Parents filled in the questionnaires either on paper or via a website. Approval for the study was obtained from the ethics committee of the Medical Ethics Committee of the Leiden University Medical Center (ethics approval number: P08.140/SH/sh).

Outcome Variables and Materials

Moral Emotions Questionnaire

The MEQ (Table 1) aims to assess behavioral responses associated with three distinct moral emotions: shame (eight items), guilt (eight items), and pride (nine items). This initial 25-item version of the MEQ was developed by a team of developmental psychologists and psychology students, some of whom had children within the intended age range, in a multi-step procedure.

Table 1.

Items of the MEQ.

Shame
1	My child hides when he/she has done something wrong
4	When my child has done something wrong, he/she does not look at me
7	When my child thinks he/she has done something stupid, he/she hits him-/herself
9	My child quickly walks away when he/she has done something he/she is not allowed to do
12	When my child does something wrong he/she makes a negative comment about him-/herself (e.g., “I am stupid”)
16	My child gets upset when he/she has done something wrong
17	When my child has broken something, he/she tries to hide it from me
20	My child is afraid of making mistakes
Guilt
3	When my child does something he/she is not allowed to do, he/she tries to make up for it (e.g., saying sorry)
6	My child shows that he/she regrets something
10	My child comes to me when he/she has broken something
13	When my child has broken something of someone else, he/she tries to repair it
18	My child cries when he/she has accidentally hurt someone
21	When my child does something wrong (e.g., spill something), he/she tries to fix it (e.g., fetches a cloth)
23	My child does not respond when I scold him/her for doing something he/she is now allowed to do (R)
24	When my child breaks something of someone else, he/she wants to make up for it
Pride
2	When my child has done something remarkable, he/she comes over to show me
5	When my child has done something remarkable, I can tell that he/she is happy about it
8	When he/she has accomplished something difficult, my child looks at me
11	When my child receives a compliment, he/she smiles
14	My child tries to do well
15	My child wants me to come over and take a look when he/she has accomplished something difficult
19	My child does not respond when I praise him/her for accomplishing something difficult (R)
22	My child likes receiving compliments
25	When my child has done something well, he/she says something positive about himself/herself

Note. MEQ: Moral Emotions Questionnaire. (R) denotes a reverse scored item.

In Step 1, each member of the team formulated items to measure guilt, shame, and/or pride, based on their experience with young children, their knowledge of the literature on moral emotions, and their experience with conducting observational studies on moral emotions.

In Step 2, the combined list of items was discussed in the team, and inappropriate items were deleted (e.g., only suitable for older children, not involving observable behavior, or overlapping content with other items) or revised (e.g., when an item was not specific enough). At this stage, 25 items passed this selection.

In Step 3, parents of 106 children were asked to fill out the first version of the MEQ consisting of 25 items, and additional questionnaires. Parents rated the degree to which each item represented their child’s behavior in the last 2 months on a 3-point scale (0 = never, 1 = sometimes, and 2 = often). For these parents, an extra response category for “not applicable” (NA) was available. Parents were instructed to choose this option only when their child had not been in that particular situation in the past 2 months. Frequencies of the response categories based on these first 106 participants showed that, for any of the items, no more than 25% of the parents had selected the option NA, and for 21 out of 25 items, this option was selected by less than 10% of parents.

In Step 4, based on the low use of the category NA, this response category was now removed from the questionnaire. No items were revised or removed at this stage. The questionnaire was now administered to another 271 parents, who were instructed to answer all items by selecting one response from the following options: “never,” “sometimes,” or “often.”

In Step 5, data from Steps 3 (n = 106) and 4 (n = 271) were analyzed collectively to examine the psychometric properties of the MEQ (variance, factor structure, internal consistency, and inter-item correlation). NA answers were analyzed in two ways: recoded as 0 and recoded as missing values. Both methods of data analyses showed similar outcomes. Therefore, in the final data analyses, NA answers from the first sample of 106 parents were recoded into 0 because, most likely, parents who did not have the NA option would have opted for “never” when their child had not been in that particular situation.

Internalizing and Externalizing Behaviors

To obtain information about the prevalence of internalizing and externalizing behaviors, the Early Childhood Inventory-4 (ECI-4; Sprafkin et al., 2002) parent checklist was used. This checklist measures the behavioral symptoms of the most prevalent disorders in the Diagnostic and Statistical Manual of Mental Disorders (4th ed.; DSM-IV) among preschool children. The ECI-4 parent checklist contains 108 items that screen for 15 emotional and behavioral disorders. Parents rated the extent to which their child showed each behavior on a 4-point scale (0 = never, 1 = sometimes, 2 = often, and 3 = very often).

For the scale for internalizing behaviors (25 items), we combined items that belonged to the following scales: major depressive disorder (10 items), separation anxiety (8 items), social phobia (3 items), and generalized anxiety (4 items).

For the scale for externalizing behaviors (18 items), we combined items belonging to the oppositional defiant disorder (8 items) and conduct disorder (10 items) scales.

The psychometric properties of these scales are shown in Table 2. The internalizing and externalizing behavior scales showed good reliability with Cronbach’s alpha values of .77 and .80, respectively. The inter-item correlation of the internalizing behaviors scale (.13) was lower than desired, but close to acceptable values, and the value for the externalizing behaviors scale was acceptable (.18).

Table 2.

Internal Consistencies of the Indices for Internalizing Behaviors, Externalizing Behaviors, and Social Competence.

	No. items	n	Mean (SD)	Cronbach’s alpha [95% CI]	Inter-item correlation (range)
Internalizing behaviors^a	25	278	0.27 (0.16)	.77 [.72, .80]	.13 (.62)
Externalizing behaviors^a	18	284	0.29 (0.20)	.80 [.77, .83]	.18 (.69)
Social competence^b	7	291	1.56 (0.31)	.64 [.57, .70]	.20 (.32)

Note. SD: standard deviation; CI: confidence interval. ^aFour-point scale (0 = never, 1 = sometimes, 2 = often, and 3 = very often); ^b3-point scale (0 = not true, 1 = somewhat true, and 2 = certainly true).

Social Competence

To obtain information about social competence, we followed the procedure of Veiga et al. (2017) and used the prosocial behaviors scale, and positive items from the peer problems scale in the Strengths and Difficulties Questionnaire (SDQ; Goodman, 1997; Muri et al., 2003). The SDQ was administered to parents, who were asked about their child’s peer relationships (two items; i.e., “Has at least one good friend,” “Generally liked by other children”); and prosocial behavior (five items). Parents rated on a 3-point scale (0 = not true, 1 = somewhat true, and 2 = certainly true), the degree to which each item represented their child’s behavior in the last 3 months. As reported in Table 2, this scale showed acceptable reliability with Cronbach’s alpha value of .64, the inter-item correlation value was also acceptable (.20).

Statistical Analyses

First, to assess construct validity, we conducted a CFA, where the proposed 25-item measurement model (Table 1) was tested. Due to non-normality of the data, the CFA was performed with maximum likelihood estimation with Satorra & Bentler’s (1994) correction, to safeguard against deviations. Goodness of fit of the model was evaluated using the root mean square error of approximation (RMSEA < .06; Hu & Bentler, 1999), the comparative fit index (CFI > .90; Bollen, 1989), and the standardized root mean square residual (SRMR < .08; Little, 2013). Furthermore, the value for the Akaike information criterion (AIC) was also used for comparison between the models. The AIC does not have an absolute norm. When comparing between models, the lower AIC value indicated the better fit (Pho et al., 2019).

To evaluate whether measurement properties of the MEQ were invariant across genders and age groups (under 4 years of age and 4 or older), a multigroup CFA was conducted. Following the procedure suggested by Milfont and Fischer (2010), testing three levels of measurement invariance sequentially: configural, metric, and scalar. Configural invariance is meant to confirm whether the model in equivalent for the groups that are being compared. In this step, the model structure in both genders and age groups was analyzed without any constraints. Metric invariance is meant to confirm whether the meaning of the items of the scale is similar for the groups being compared. In this step, the model structure in both genders and age groups was analyzed constraining all the factor loadings. In the case that metric invariance was not met, partial invariance was analyzed, after freeing the invariant items (Byrne et al., 1989). If metric (partial) invariance is met, scalar invariance can be tested, which allows to confirm that members of each group being compared are rated similarly when using the scale. In this step, the model structure in both genders and age groups was analyzed constraining the item intercepts. To test the metric and scalar invariance, three model fit indices variations were considered: the decrease of the CFI value should not be more than .01 in comparison to the previous model; the variation of the RMSEA between models should be less than .015; and the variation of the SRMR between models should be less than .030 (Chen, 2007; Cheung & Rensvold, 2002). Items to be freed for partial invariance analyses were chosen based on their univariate modification indices, and also on the Lagrange multiplier test. This test shows the effect of releasing an equality constraint simultaneously between groups (Martín-Puga et al., 2020; Rosseel, 2012). Gender and age group differences were assessed if at least 50% of the items comprising a given factor were invariant (Steenkamp & Baumgartner, 1998; Vandenberg & Lance, 2000). As significant correlations were found between age and the scale guilt (r = .18; p < .05) and also with the scale pride (r = .15; p < .05), further analyses were conducted with partial correlations corrected for age.

Second, considering the ordered categorical nature of the items, we assessed the internal consistencies of the obtained MEQ scales using McDonald’s omega and inter-item correlations (Crutzen & Peters, 2017). Third, Pearson’s correlations (with Bonferroni correction for multiple comparisons) with the internalizing and externalizing behaviors of the ECI-4, and the social competence scale of the SDQ, were conducted to examine concurrent validity. Prior inspection to the scatterplots of the associations between the three moral emotions and the concurrent measures, indicated only linear trends, therefore, only linear associations were studied. The CFA in this study was conducted using R’s (version 4.0.2) lavaan package (version 0.6-6; Rosseel, 2012); and the lavTestScore function of the same package was used to conduct the Lagrange multiplier test. All the other statistical analyses were performed with the IBM SPSS (version 21).

Results

Confirmatory Factor Analysis

Items on the MEQ that were negatively formulated (see Table 1) were reverse coded, so that, higher scores represented more expression of the moral emotion. The original 25-item MEQ (Table 1) was fitted with the hypothesized three-factor structure and yielded a poor fit (Model 1 in Table 3). To improve model fit, factor loadings and modification indices were analyzed. That is, items with low factor loadings (< .30), and high modification indices (> 10) were further analyzed in terms of their content. The content of each item was considered before exclusion, so that, any deletion was not solely based on statistical outcomes, but also on the theoretical appropriateness of the item on the intended factor. Items who showed conceptual overlap with another scale or concept were removed. This procedure resulted in the deletion of six items (7, 20, 12, 19r, 5, and 16) and resulted in Model 2. Although this model showed an improved fit, two more items were removed resulting in Model 3 (Figure 1): Item 10 was excluded due to high loading on the non-intended factor; Item 24 was removed due to theoretical overlap with Item 13. The values for goodness of fit for the final model (Model 3) showed appropriate levels. The SRMR and the RMSEA were below the desired cut-off points. The AIC of Model 3 was the lowest, therefore indicating a better fit. Although the CFI did not reach the cut-off point of > .90, our CFI value (.871) was within the acceptable range, given that the RMSEA of the null model value for Model 3 was .119 (Veiga et al., 2019). The CFI is a comparative index that ranges from 0 to 1, in which the proposed model is compared to a null model wherein all measured variables are uncorrelated. The null model should have a poor fit, with a very large chi square (Ching et al., 2014). Kenny et al. (2014) have suggested that if the value of the RMSEA of the null model is < .158, the CFI is less informative. Thus, a threshold of > .90 could be too strict for our model. Correlations between error terms of items were analyzed. No significant correlations were found, indicating that there was no overlap between items from different latent variables.

Table 3.

Results of the Confirmatory Factor Analysis.

	χ²	df	RMSEA [90% CI]	CFI	Null RMSEA	SRMR	AIC
Model 1	656.28*	272	.065 [.059, .072]	.687	.124	.079	12,868.49
Model 2	323.74*	149	.059 [.050, .068]	.805	.126	.066	10,378.46
Model 3	198.30*	116	.046 [.035, .056]	.871	.119	.056	9,074.00

Note. RMSEA: root mean square error of approximation; CFI: comparative fit index; SRMR: standardized root mean square residual; AIC: Akaike information criterion. N = 377. *p < .001.

Figure 1.

CFA of the Three-Factor Structure of the MEQ.

Measurement Invariance Across Gender

To test the invariance of the model across gender, a multigroup CFA was performed to the final model (Table 4). First, the fit statistics obtained from the configural (baseline) model showed an acceptable fit, χ² (232, N = 306) = 296.162, p < .01; CFI = .891; RMSEA = .040; SRMR = .065. In the next step, the testing of the metric invariance showed a significant change in the model fit (ΔCFI = −.020; ΔRMSEA = .003; ΔSRMR = .007), indicating that the model was variant across genders, and metric invariance could not be assumed. Partial metric invariance was then tested by freeing the constraint on Item 11, and a nonsignificant change in the model fit was obtained, as compared to the configural model (ΔCFI = .005; ΔRMSEA = −.002; ΔSRMR = .004). This indicates that except for Item 11, the factor loadings were invariant across genders, and partial metric invariance could be assumed. Next, the testing of the scalar invariance showed a significant change in the model fit (ΔCFI = −.052; ΔRMSEA = .008; ΔSRMR = .004). After freeing the equality constraints on the intercepts of Items 15 and 18, partial scalar invariance could be achieved (ΔCFI = .002; ΔRMSEA = −.001; ΔSRMR = .001).

Table 4.

Fit Indices for Measurement Invariance Models Across Gender and Age Group (< 4 yo; ⩾ 4 yo).

Parameter	Model fit indices						Model fit differences
Parameter	χ²	df	CFI	RMSEA [90% CI]	Null RMSEA	SRMR	ΔCFI	ΔRMSEA	ΔSRMR
Gender
Configural	296.142*	232	.891	.040 [.025, .054]	.117	.065
Metric	321.816*	246	.871	.043 [.028, .055]	.117	.072	−.020	.003	.007
Partial metric^a	305.620*	245	.896	.038 [.022, .052]	.117	.069	.005	−.002	.004
Scalar^a	351.840*	259	.844	.046 [.033, .057]	.117	.073	−.052	.008	.004
Partial scalar^a,b	316.327*	256	.898	.037 [.021, .050]	.117	.070	.002	−.001	.001
Age group
Configural	316.517*	232	.869	.046 [.033, .059]	.121	.065
Metric	326.208*	246	.874	.044 [.030, .056]	.121	.068	.005	−.002	.003
Scalar	423.264*	260	.753	.060 [.050, .070]	.121	.076	−.121	.016	.008
Partial scalar^c	341.172*	255	.866	.045 [.031, .057]	.121	.069	−.008	.001	.001

Note. CFI: comparative fit index; RMSEA: root mean square error of approximation; CI: confidence interval; SRMR: standardized root mean square residual. N = 377. *p < .01.

Equality constraint on the factor loading of Item 11 was freed from the model.

Equality constraints on the intercepts of Items 11, 15, and 18 were freed from the model.

Equality constraints on the intercepts of Items 8, 9, 14, 17, and 23 were freed from the model.

Because partial scalar invariance could be assumed, and more than 50% of the items for each factor were invariant, the means of the two gender groups can be compared for the three moral emotions. As shown in Table 6, caregivers of boys acknowledged their children to show less guilt, t(375) = −2.75, p = .006, and pride, t(373) = −4.17, p < .001, compared to girls. An additional inspection on the latent means showed that boys and girls differed in pride, E_(girls–_boys) = .074 (Var_(girls–_boys) = 0.022); p = .001, but not in guilt and shame.

Measurement Invariance Across Age Groups

To test the invariance of the model across age groups, a multigroup CFA was performed to the final model (Table 4), with one group comprising participant younger than 4 years of age, and the other group comprising participants who were 4 years or older. First, the fit statistics obtained from the configural (baseline) model showed an acceptable fit, χ² (264, N = 306) = 316.517, p < .001; CFI = .869; RMSEA = .046; SRMR = .065. In the next step, the testing of the metric invariance showed a nonsignificant change in the model fit (ΔCFI = .005; ΔRMSEA = −.002; ΔSRMR = .003), indicating that the model was invariant across the two age groups, and metric invariance could be assumed. Next, the testing of the scalar invariance showed a significant change in the model fit (ΔCFI =−.121; ΔRMSEA = .016; ΔSRMR = .008). Further analyses indicated that the equality constraints on the intercepts of Items 8, 9, 14, 17, and 23 should be freed, and partial scalar invariance could be achieved afterwards (ΔCFI = −.008; ΔRMSEA = .001; ΔSRMR = .001).

The assumption of partial scalar invariance (with at least 50% of the items for each factor were invariant) allows the means of the two age groups to be compared for the three moral emotions. As shown in Table 6, caregivers of the children younger than 4 years of age acknowledged their children to show less guilt, t(375) = −2.77, p = .006, and pride, t(220) = −2.83, p = .005, compared to children who are 4 years or older. An additional inspection on the latent means also showed that children younger than 4 years of age and children who were 4 years or older, differed in guilt, E_(⩾ _{4 yo} – _< _{4 yo)} = .153 (Var_(⩾ _{4 yo} – _< _4yo) = .046); p = .001, and pride, E_(⩾ _{4 yo} – _< _{4 yo)} = .040 (Var_(⩾ _{4 yo} – _< _{4 yo)} = .020); p = .043, but not in shame.

Reliability

Table 5 shows partial correlations corrected for age between the MEQ scales. Guilt was positively associated with pride, yet not to a degree that suggests collinearity. No other significant correlations between the MEQ scales were found.

Table 5.

Correlations Between Indices for Moral Emotions (Corrected for Age).

	1	2	3
1. MEQ–shame	–	−.13*	.03
2. MEQ–guilt		–	.27**
3. MEQ–pride			–

Note. MEQ: Moral Emotions Questionnaire. N = 377. *p < .05; **p < .006 (after Bonferroni correction).

McDonald’s omega and inter-item correlation coefficients for the three MEQ scales are reported in Table 6. The outcomes show that the internal consistency per scale is acceptable for shame (.62), guilt (.62), and pride (.63).

Table 6.

Mean Total Mean by Age Category, and Internal Consistencies of the MEQ Scales.

	No. items	Ω	IIC (range)	Mean (SD)	Age group comparison				Gender group comparison
	No. items	Ω	IIC (range)	Mean (SD)	< 4 yo, mean (SD)	⩾ 4 yo, mean (SD)	95% CI of differences	d	Boys, mean (SD)	Girls, mean (SD)	95% CI of differences	d
Shame	4	.62	.28 (.37)	.61 (.40)	.55 (.37)	.63 (.42)	[−.16, .01]	.00	.61 (.41)	.61 (.40)	[−.08, .08]	.02
Guilt*	6	.62	.21 (.37)	1.22 (.35)	1.15 (.35)	1.25 (.35)	[−.18, −.03]*	.03	1.17 (.34)	1.27 (.36)	[−.17, −.03]*	.00
Pride*	7	.63	.20 (.33)	1.78 (.25)	1.73 (.27)	1.81 (.23)	[−.13, −.02]*	.04	1.73 (.27)	1.83 (.20)	[−.15, −.05]*	.03

Note. Ω: McDonald’s omega; IIC: inter-item correlation; CI: confidence interval; d = Cohen’s d. The MEQ was scored on a 3-point scale (0 = never, 1 = sometimes, and 2 = often). Total N = 377 (n = 127 for < 4 years; n = 250 for ⩾ 4 years; n = 208 for boys; n = 169 for girls). *p < .05. Age category and gender comparisons showed that for both groups differences were found for the guilt and pride scales.

Concurrent Validity

As shown in Table 7, shame was positively associated with externalizing and internalizing behaviors. Guilt was negatively associated with externalizing behaviors and positively associated with social competence. Pride was positively associated with social competence. No other significant correlations were observed.

Table 7.

Correlations of MEQ Scales With Indices for Internalizing and Externalizing Problems, and With Social Competence (Corrected for Age).

	Internalizing behaviors	Externalizing behaviors	Social competence
MEQ–shame	.249*	.299*	−.070
MEQ–guilt	−.034	−.304*	.388*
MEQ–pride	.082	.030	.200*

Note. N = 205. *p < .006 (after Bonferroni correction).

Discussion

The outcomes of this study suggest that the three moral emotions, that is, guilt, shame, and pride, can be identified separately in early childhood through the MEQ. The originally proposed 25-item model was not confirmed. However, after extracting eight items due to theoretical–statistical reasons (e.g., low factor loadings; high loading on the non-intended factor; theoretical appropriateness of the item; overlap with other 17-item model based on the hypothesized three-factor structure, with a satisfactory goodness of fit). Although eight items were deleted, the final 17-item model still represents the intended constructs, considering that besides the statistical results, the appropriateness of each item in its intended scale was considered in each step. This resulted in the deletion of items from each scale that referred to more general behaviors, and therefore lacked an action tendency related to its intended construct (e.g., in shame—“My child is afraid of making mistakes,” “My child gets upset when he/she has done something wrong”). Furthermore, only items that clearly reflected the action tendencies of its intended scale were kept. For example, in our questionnaire, shame was operationalized as the tendency to show submissive behavior or to withdraw or escape from a situation. Therefore, looking at the remaining four items from the final model, we see that they reflect these action tendencies (e.g., “My child hides when he/she has done something wrong,” “My child quickly walks away when he/she has done something he/she is not allowed to do”).

Measurement invariance analysis across gender showed that the factor loading of one item from pride differed across gender. Furthermore, the intercepts of three items (one item from guilt and two items from pride) were variant across gender. While previous studies have shown that female participants tend to report significantly more action tendencies of shame, guilt, and pride than male participants (Beißert & Hasselhorn, 2016; Else-quest et al., 2012; Etxebarria et al., 2019; Kushnir et al., 2016), our results seem to further show that the action tendencies of pride could be different in girls and boys. As for measurement invariance across age groups (< 4 or ⩾ 4 years), the analysis showed that all the items have similar meanings for caregivers. Yet, intercept invariance across age groups was only achieved after releasing the equality constraints on five items (one item from guilt, two items from shame, and two items from pride). This indicates that the younger group in this study was at the early stages of developing moral emotions, and therefore was expected to show less action tendencies related to shame, guilt, and pride, compared to the older age group. Although only partial invariance was achieved, the proportion of invariant items on each factor was above the required level (i.e., ⩾ 50%). Therefore the constructs can be considered as equally calibrated across groups, and group means could be compared (Steenkamp & Baumgartner, 1998; Vandenberg & Lance, 2000). Although acceptable, all three scales showed low internal consistencies. However, the concurrent validity of these three scales was further confirmed by the relationships of the three moral emotions with externalizing behaviors, internalizing behaviors, and social competence. These correlations conformed exactly with our hypotheses, and were in line with the literature on this topic (Ferguson et al., 1999; Hooge et al., 2011; Kluwin et al., 2002; Mascolo & Fischer, 1995; Roos et al., 2014; Stuewig et al., 2010; Tangney et al., 1996; Thomaes et al., 2011).

As mentioned earlier, previous studies focusing on young children have not yet distinguished between the three moral emotions included in our newly developed questionnaire. Not only did this study show that shame, guilt, and pride can be distinctly observed in preschool children; it also showed that these emotions turned out to have distinct relationships with other domains of social–emotional functioning.

First, relations found in our results confirm previous studies that characterize guilt as an adaptive emotion (Baumeister et al., 1994; Broekhof et al., 2018; Estrada-hollenbeck & Heatherton, 1995; Frick & Morris, 2004; Kochanska & Aksan, 2006). Parents reported that children who expressed more guilt behaviors after a transgression showed fewer externalizing behaviors (e.g., aggression or rule-breaking) and higher levels of social competence. These findings confirm outcomes from previous studies that were focused on older children and adolescents (Baumeister et al., 1994; Broekhof et al., 2018; Estrada-hollenbeck & Heatherton, 1995; Frick & Morris, 2004; Kochanska & Aksan, 2006).

Second, we found that pride, much like guilt, also serves an adaptive purpose. In line with other studies (Hooge et al., 2011; Kluwin et al., 2002; Mascolo & Fischer, 1995), our results showed that pride was related to better social competence. This suggests that pride allows children to feel confident enough to interact with peers, and that showing others that you are a valuable asset to the group indeed helps you to be evaluated positively by others.

Third, shame seems to serve a different function, in line with previous studies, our study portrays shame as a maladaptive emotion; higher levels of shame behaviors were related to more internalizing and externalizing behaviors (Tangney et al., 1992; Thomaes et al., 2011). The social context and content of transgressions in this study may help explain these outcomes, for example, the shame items in the questionnaire all involved wrongdoing, for example, breaking something or disobeying a rule. In those cases, children were commonly expected to make eye contact and admit their transgression, thus showing their guilt. Avoidant behaviors were usually perceived as trying to avoid the blame or punishment (Barrett et al., 1993; Estrada-hollenbeck & Heatherton, 1995; Stuewig et al., 2010), which might explain the maladaptive function of shame behaviors in those contexts. Yet an unwanted identity (a core feature of shame) does not necessarily involve harm or wrongdoing, as formulated in the items in the questionnaire. Instead, shame can also arise from walking around with a bad haircut or accidentally tripping over a carpet, or other clumsy behaviors. These shame-only occasions with no guilt involved, where no explicit harm was done to another, were not included in this questionnaire. However, including such items could provide a more adaptive function of shame. This other context for shame might be a valuable addition in future studies.

This study did have some limitations that need to be highlighted. First, parents were our only informants in regard to their children’s manifestations of morally guided behavior and psychosocial functioning. Second, future studies should compare these parent reports with reports from other informants (e.g., preschool teachers) and field observations, to further confirm the validity of the MEQ. Third, cross-cultural studies with the MEQ could be informative, because the literature on these emotions often discusses the different functions of moral emotions in Western, individualistic-oriented cultures versus Eastern, collectivistic-oriented cultures (Bedford & Hwang, 2003; Cole et al., 2006; Fung, 1999; Midlarsky et al., 2006; Yoshioka & Choi, 2005). For individuals from Eastern cultures, the collective harmony (group) is more important than individual independence. In these societies, failing to achieve the group demands causes the individual to feel as a failure. This sense of failure is extended to their families, which even puts a higher pressure on the individual to behave well within the norms and values of that group (Bedford & Hwang, 2003). Shame, in collectivistic-oriented cultures, seems to have a self-regulative function, preventing individuals from acting against social norms (Bedford & Hwang, 2003; Cole et al., 2006; Fung, 1999; Midlarsky et al., 2006; Yoshioka & Choi, 2005). Guilt, seems to have the same functions cross-culturally (Bedford & Hwang, 2003; Merolla et al., 2013). As for pride, previous studies have shown that in collectivistic cultures it is emphasized that a person should feel pride in situations that positively contribute to others (e.g., helping someone else; Stipek, 1998; Stipek et al., 1989). Also, it is reported that Eastern individuals do not emphasize the value of self (Heine et al., 1999), and therefore individual pride, is less prominent and even discouraged in these societies. Therefore, cross-cultural studies with the MEQ are needed as the results obtained may only hold true in Western populations. Fourth, the cross-sectional nature of this study prevents us from drawing conclusions about the directionality of the relationships we found between the three moral emotions and externalizing behaviors, internalizing behaviors, and social competence. We assume that it is the anticipation of the moral emotions that has an effect on the way children behave toward others, and not the other way around. Yet, this must be confirmed in longitudinal studies. Fifth, future longitudinal studies should endeavor to identify which factors underlie the development of moral emotions. Insight into which factors support the development of moral emotions will improve our understanding of how to develop effective preemptive interventions to stimulate the development of these crucial emotions. Sixth, future studies should also look into the relation between the MEQ scales and observational data, to further contribute for its validity. Finally, for this study, the participants were evaluated in a single time point. Future studies should assess test re-test reliability, not only to improve internal validity, but also to understand the stability of this measure over time.

In sum, the MEQ appears to be a promising and reliable instrument for evaluating the extent to which young children experience and display distinct moral emotions, through the parents’ perspective. The MEQ does offer some important advantages: first, the MEQ is an easy-to-administer instrument. It is far less time consuming and more cost-effective than observational methods. Second, it provides an ecologically valid way to assess moral emotions in early childhood, as parents are able to report on their children’s moral behavior over time and across situations. Furthermore, the MEQ gives us the opportunity to examine new research questions, including those that concern early impairments in moral development and their underlying causes, or how moral development further affects other social–emotional functioning domains. Answers to these questions could help professionals understand the basis and consequences of possible impairments in moral development, and create strategies that promote children’s moral, emotional, and social development.

Footnotes

Acknowledgements

The authors thank the parents and children who participated in this study and the students who helped collect the data. They also thank Shannon Yuen for her English language editing and review.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was financially supported by the Fundação para a Ciência e Tecnologia (FCT), Portugal (grant number: SFRH/BD/144768/2019), and the Care for the Young: Innovation and Development program by ZonMw (grant number: 80-82430-98-8025).

ORCID iDs

Brenda M. S. da Silva

Yung-Ting Tsou

References

Barrett

(1998a). A functionalist perspective to the development of emotions. In Mascolo

Griffin

(Eds.), What Develops in Emotional Development? (pp. 109–133). Plenum Press. https://doi.org/10.1007/978-1-4899-1939-7_5

Barrett

(1998b). The origins of guilt in early childhood. In Bybee

(Ed.), Guilt and Children (pp. 75–90). Academic Press. https://doi.org/10.1016/b978-012148610-5/50004-7

Barrett

Cole

Zahn-Waxler

(1993). Avoiders versus Amenders—Implications for the investigation of guilt and shame during toddlerhood? Cognition and Emotion, 7(6), 481–505. https://doi.org/10.1080/02699939308409201

Baumeister

R. F.

Stillwell

A. M.

Heatherton

T. F.

(1994). Guilt : An interpersonal approach. Psychological Bulletin, 115(2), 243–267. https://doi.org/10.1037/0033-2909.115.2.243

Bedford

O. A.

Hwang

K. K.

(2003). Guilt and shame in Chinese culture: A cross-cultural framework from the perspective of morality and identity. Journal for the Theory of Social Behaviour, 33(2), 127–144. https://doi.org/10.1111/1468-5914.00210

Beißert

H. M.

Hasselhorn

(2016). Individual differences in moral development: Does intelligence really affect children’s moral reasoning and moral emotions? Frontiers in Psychology, 7, 1961. https://doi.org/10.3389/fpsyg.2016.01961

Belsky

Domitrovich

C. E.

Crnic

(1997). Temperament and parenting antecedents of individual differences in 3-year olds’ pride and shame reactions. Child Development, 68(3), 456–466. https://doi.org/10.1016/s0163-6383(96)90484-1

Bennett

Sullivan

Lewis

(2005). Young children’s adjustment as a function of maltreatment, shame, and anger. Child Maltreatment, 10(4), 311–323. https://doi.org/10.1177/1077559505278619

Blair

R. J. R.

Monson

Frederickson

(2001). Moral reasoning and conduct problems in children with emotional and behavioural difficulties. Personality and Individual Differences, 31(5), 799–811. https://doi.org/10.1016/S0191-8869(00)00181-1

10.

Blasi

(1999). Emotions and moral motivation. Journal for the Theory of Social Behaviour, 29(1), 1–19. https://doi.org/10.1111/1468-5914.00088

11.

Bollen

K. A.

(1989). A new incremental fit index for general structural equation models. Sociological Methods & Research, 17(3), 303–316. https://doi.org/10.1177/0049124189017003004

12.

Broekhof

Bos

M. G. N.

Camodeca

Rieffe

(2018). Longitudinal associations between bullying and emotions in deaf and hard of hearing adolescents. Journal of Deaf Studies and Deaf Education, 23(1), 17–27. https://doi.org/10.1093/deafed/enx036

13.

Bullock

Lutkenhaus

(1990). Who am I ? Self-understanding in toddlers. Merrill-Palmer Quarterly, 36(2), 217–238.

14.

Byrne

B. M.

Muthèn

Shavelson

R. J.

(1989). Testing the equivalence of factor covariance and mean structure. The Issue of Partial Measurement Invariance, 105(3), 456–466. https://doi.org/10.1037/0033-2909.105.3.456

15.

Chen

F. F.

(2007). Sensitivity of goodness of fit indexes to lack of measurement invariance. Structural Equation Modeling: A Multidisciplinary Journal, 14(3), 464–504. https://doi.org/10.1080/10705510701301834

16.

Cheng

J. T.

Tracy

J. L.

Henrich

(2010). Pride, personality, and the evolutionary foundations of human social status. Evolution and Human Behavior, 31(5), 334–347. https://doi.org/10.1016/j.evolhumbehav.2010.02.004

17.

Cheung

G. W.

Rensvold

R. B.

(2002). A evaluating goodness-of-fit indexes for testing measurement invariance. Structural Equation Modeling: A Multidisciplinary Journal, 9(2), 233–255. https://doi.org/10.1207/S15328007SEM0902_5

18.

Ching

G. S.

Lien

W.-C.

Chao

P.-C.

(2014). Developing a scale to measure the situational changes in short-term study abroad programs. International Journal of Research Studies in Education, 3(5), 53–71. https://doi.org/10.5861/ijrse.2014.771

19.

Cole

P. M.

Barrett

Zahn-Waxler

(1992). Emotion displays in two-year-olds during mishaps. Child Development, 63(2), 314–324. https://doi.org/10.1111/j.1467-8624.1992.tb01629.x

20.

Cole

P. M.

Tamang

B. L.

Shrestha

(2006). Cultural variations in the socialization of young children’s anger and shame. Child Development, 77(5), 1237–1251. https://doi.org/10.1111/j.1467-8624.2006.00931.x

21.

Crutzen

Peters

G. J. Y.

(2017). Scale quality: Alpha is an inadequate estimate and factor-analytic evidence is needed first of all. Health Psychology Review, 11(3), 242–247. https://doi.org/10.1080/17437199.2015.1124240

22.

Dahl

(2015). The developing social context of infant helping in two U.S. samples. Child Development, 86(4), 1080–1093. https://doi.org/10.1111/cdev.12361

23.

Dahl

Killen

(2018a). A developmental perspective on the origins of morality in infancy and early childhood. Frontiers in Psychology, 9, 1736. https://doi.org/10.3389/fpsyg.2018.01736

24.

Dahl

Killen

(2018b). Moral reasoning: Theory and research in developmental science. In Stevens’ handbook of experimental psychology and cognitive neuroscience (pp. 1–31). https://doi.org/10.1002/9781119170174.epcn410

25.

Dempsey

H. L.

(2017). A comparison of the social-adaptive perspective and functionalist perspective on guilt and shame. Behavioral Sciences, 7, 83. https://doi.org/10.3390/bs7040083

26.

Else-quest

N. M.

Higgins

Allison

Morton

L. C.

Else-quest

N. M.

Higgins

Allison

Morton

L. C.

(2012). Gender differences in self-conscious emotional experience: A meta-analysis. Psychological Bulletin, 138, 947–981. https://doi.org/10.1037/a0027930

27.

Estrada-hollenbeck

Heatherton

T. F.

(1995). Avoiding and alleviating guilt through prosocial behavior. In: Guilt and children (pp. 215–231). Academic Press. https://doi.org/10.1016/B978-012148610-5/50011-4

28.

Etxebarria

Conejero

Pascual

José

Barón

Apodaca

(2019). Moral pride, more intense in girls than in boys? Journal of Moral Education, 48, 230–246. https://doi.org/10.1080/03057240.2018.1469478

29.

Ferguson

Stegge

Eyre

H. L.

Vollmer

Ashbaker

(2000). Context effects and the (mal)adaptive nature of guilt and shame in children. Genetic, Social, and General Psychology Monographs, 126(3), 319–345.

30.

Ferguson

Stegge

Miller

E. R.

Olsen

M. E.

(1999). Guilt, shame, and symptoms in children. Developmental Medicine & Child Neurology, 35(2), 347–357. https://doi.org/10.1037/0012-1649.35.2.347

31.

Frick

Morris

(2004). Temperament and developmental pathways to conduct problems. Journal of Clinical Child and Adolescent Psychology, 33(1), 54–68. https://doi.org/10.1207/S15374424JCCP3301

32.

Fung

(1999). Becoming a moral child: The socialization of shame among young Chinese children. Ethos, 27(2), 180–209. https://doi.org/10.1525/eth.1999.27.2.180

33.

Goodman

(1997). The Strengths and Difficulties Questionnaire: A research note. Journal of Child Psychology and Psychiatry, 38(5), 581–586. https://doi.org/10.1111/j.1469-7610.1997.tb01545.x

34.

Hammond

S. I.

Al-Jbouri

Edwards

Feltham

L. E.

(2017). Infant helping in the first year of life: Parents’ recollection of infants’ earliest prosocial behaviors. Infant Behavior and Development, 47, 54–57. https://doi.org/10.1016/j.infbeh.2017.02.004

35.

Harper

F. W. K.

Arias

(2004). The role of shame in predicting adult anger and depressive symptoms among victims of child psychological maltreatment. Journal of Family Violence, 19(6), 367–375. https://doi.org/10.1007/s10896-004-0681-x

36.

Hart

Matsuba

M. K.

(2007). The development of pride and moral life. In Tracy

J. L.

Robins

R. W.

Tangney

J. P.

(Eds.), The self-conscious emotions: Theory and research (pp. 114–133). Guilford Press.

37.

Heine

S. J.

Lehman

D. R.

Markus

H. R.

Kitayama

(1999). Is there a universal need for positive self-regard? Psychological Review, 106(4), 766–794. https://doi.org/10.1037/0033-295X.106.4.766

38.

Holmqvist

(2008). Psychopathy and affect consciousness in young criminal offenders. Journal of Interpersonal Violence, 23(2), 209–224. https://doi.org/10.1177/0886260507309341

39.

Hooge

Zeelenberg

Breugelmans

(2011). A functionalist account of shame-induced behaviour. Cognition and Emotion, 25(5), 939–946. https://doi.org/10.1080/02699931.2010.516909

40.

Bentler

P. M.

(1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6(1), 1–55. https://doi.org/10.1080/10705519909540118

41.

Kagan

(2005). Human morality and temperament. In Carlo

Edwards

C. P.

(Eds.), Nebraska symposium on motivation: Vol. 51. Moral motivation through the lifespan: Theory, research and application (pp. 1–32). University of Nebraska Press. https://doi.org/10.1017/CBO9781107415324.004

42.

Kelley

S. A.

Brownell

C. A.

Campbell

(2000). Mastery motivation and self-conscious affect in toddlers: Relations with maternal control and evaluative feedback. Child Development, 71(4), 1061–1071. https://doi.org/10.1016/S0163-6383(96)90591-3

43.

Kenny

D. A.

Kaniskan

McCoach

D. B.

(2014). The performance of RMSEA in models with small degrees of freedom. Sociological Methods and Research, 44, 486–507. https://doi.org/10.1177/0049124114543236

44.

Ketelaar

Wiefferink

C. H.

Frijns

J. H. M.

Broekhof

Rieffe

(2015). Preliminary findings on associations between moral emotions and social behavior in young children with normal hearing and with cochlear implants. European Child & Adolescent Psychiatry, 24, 1369–1380. https://doi.org/10.1007/s00787-015-0688-2

45.

Kluwin

T. N.

Stinson

M. S.

Colarossi

G. M.

(2002). Social processes and outcomes of in-school contact between deaf and hearing peers. Journal of Deaf Studies and Deaf Education, 7(3), 200–213. https://doi.org/10.1093/deafed/7.3.200

46.

Kochanska

Aksan

(2006). Children’s conscience and self-regulation. Journal of Personality, 74(6), 1588–1161. https://doi.org/10.1111/j.1467-6494.2006.00421.x

47.

Kochanska

Casey

R. J.

Fukumoto

(1995). Toddlers’ sensitivity to standard violations. Child Development, 66(3), 643–656. https://doi.org/10.1111/j.1467-8624.1995.tb00895.x

48.

Kochanska

DeVet

Goldman

Murray

Putnam

S. P.

(1994). Matemal reports of conscience development and temperament in young children. Child Development, 65, 852–868. https://doi.org/10.1111/j.1467-8624.1994.tb00788.x

49.

Kochanska

Gross

J. N.

Lin

M. H.

Nichols

K. E.

(2002). Guilt in young children: Development, determinants, and relations with a broader system of standards. Child Development, 73(2), 461–482. https://doi.org/10.1111/1467-8624.00418

50.

Kushnir

Godinho

Hodgins

D. C.

Hendershot

C. S.

Cunningham

J. A.

(2016). Gender differences in self-conscious emotions and motivation to quit gambling. Journal of Gambling Studies, 32, 969–983. https://doi.org/10.1007/s10899-015-9574-6

51.

Lewis

(1998). Emotional competence and development. In Pushkar

Bukowski

Schwartzman

Stack

White

(Eds.), Improving competence across the lifespan (pp. 27–36). Plenum Press. https://doi.org/10.1007/0-306-47149-3_3

52.

Lewis

(2014). The rise of consciousness and the development of emotional life. Guilford Press.

53.

Little

T. D.

(2013). Longitudinal structural equation modeling. Guildford Press.

54.

Martín-Puga

M. E.

Justicia-Galiano

M. J.

Gómez-Pérez

M. M.

Pelegrina

(2020). Psychometric properties, factor structure, and gender and educational level invariance of the Abbreviated Math Anxiety Scale (AMAS) in Spanish children and adolescents. Assessment. Advance online publication. https://doi.org/10.1177/1073191120980064

55.

Mascolo

Fischer

(1995). Developmental transformations in appraisals for pride, shame and guilt. In Tangney

J. P.

Fischer

K. W.

(Eds.), Self-conscious emotions: The psychology of shame, guilt, embarrassment, and pride (pp. 64–113). Guilford Press.

56.

Mauro

Sato

Tucker

(1992). The role of appraisal in human emotions: A cross-cultural study. Journal of Personality and Social Psychology, 62(2), 301–317. https://doi.org/10.1037/0022-3514.62.2.301

57.

Menesini

Camodeca

(2008). Shame and guilt as behaviour regulators: Relationships with bullying, victimization and prosocial behaviour. British Journal of Developmental Psychology, 26(2), 183–196. https://doi.org/10.1348/026151007X205281

58.

Merolla

A. J.

Zhang

Sun

(2013). Forgiveness in the United States and China: Antecedents, consequences, and communication style comparisons. Communication Research, 40(5), 595–622. https://doi.org/10.1177/0093650212446960

59.

Midlarsky

Venkataramani-Kothari

Plante

(2006). Domestic violence in the Chinese and South Asian immigrant communities. Annals of the New York Academy of Sciences, 1087, 279–300. https://doi.org/10.1196/annals.1385.003

60.

Milfont

Fischer

(2010). Testing measurement invariance across groups: Applications in cross-cultural research. International Journal of Psychological Research, 3(1), 111–121. https://doi.org/10.21500/20112084.857

61.

Muris

Meesters

Van den Berg

(2003). The Strengths and Difficulties Questionnaire (SDQ): Further evidence for its reliability and validity in a community sample of Dutch children and adolescents. European Child & Adolescent Psychiatry, 12(1), 1–8. https://doi.org/10.1007/s00787-003-0298-2

62.

Olthof

(2012). Anticipated feelings of guilt and shame as predictors of early adolescents’ antisocial and prosocial interpersonal behaviour. European Journal of Developmental Psychology, 9(3), 371–388. https://doi.org/10.1080/17405629.2012.680300

63.

Orth

Robins

R. W.

Soto

C. J.

(2010). Tracking the trajectory of shame, guilt, and pride across the life span. Journal of Personality and Social Psychology, 99(6), 1061–1071. https://doi.org/10.1037/a0021342

64.

Pho

K.-H.

Lukusa

T. M.

(2019). Comparison among Akaike information criterion, Bayesian information criterion and Vuong’s test in model selection: A case study of violated speed regulation in Taiwan. Journal of Advanced Engineering and Computation, 3(1), 293. https://doi.org/10.25073/jaec.201931.220

65.

Roos

Hodges

E. V. E.

Salmivalli

(2014). Do guilt- and shame-proneness differentially predict prosocial, aggressive, and withdrawn behaviors during early adolescence? Developmental Psychology, 50(3), 941–946. https://doi.org/10.1037/a0033904

66.

Rosseel

(2012). lavaan: An R package for structural equation modeling. Journal of Statistical Software, 48(2), 1–36. http://www.jstatsoft.org/v48/i02/

67.

Satorra

Bentler

P. M.

(1994). Corrections to test statistics and standard errors in covariance structure analysis. In Eye

A. V.

Clogg

C. C.

(Ed.), Latent variables analysis: Applications for development research (pp. 399–419). Sage.

68.

Shariff

A. F.

Tracy

J. L.

(2011). What are emotion expressions for? Current Directions in Psychological Science, 20(6), 395–399. https://doi.org/10.1177/0963721411424739

69.

Sprafkin

Volpe

R. J.

Gadow

K. D.

(2002). A DSM—IV—Referenced screening instrument for preschool children: The Early Childhood Inventory-4. Journal of the American Academy of Child & Adolescent Psychiatry, 41(5), 604–612. https://doi.org/10.1097/00004583-200205000-00018

70.

Stearns

D. C.

Parrott

W. G.

(2012). When feeling bad makes you look good: Guilt, shame. Cognition and Emotion, 26(3), 37–41. https://doi.org/10.1080/02699931.2012.675879

71.

Steenkamp

J.-B. E. M.

Baumgartner

(1998). Assessing measurement invariance in cross-national consumer research. Journal of Consumer Research, 25(1), 78–107. https://doi.org/10.1086/209528

72.

Stipek

(1998). Differences between Americans and Chinese in the circumstances evoking pride, shame, and guilt. Journal of Cross-Cultural Psychology, 29, 616–629. https://doi.org/10.1177/0022022198295002

73.

Stipek

Recchia

McClintic

Lewis

(1992). Self-evaluation in young children. Monographs of the Society for Research in Child Development, 57(1), Serial No 226. https://doi.org/10.2307/1166190

74.

Stipek

Weiner

(1989). Testing some attribution-motion relations in the people’s republic of China. Journal of Personality and Social Psychology, 56, 109–116. https://doi.org/10.1037/0022-3514.56.1.109

75.

Stuewig

Tangney

J. P.

Heigel

Harty

Mccloskey

Heath

(2010). Shaming, blaming, and maiming : Functional links among the moral emotions, externalization of blame, and aggression. Journal of Research in Personality, 44(1), 91–102. https://doi.org/10.1016/j.jrp.2009.12.005

76.

Sznycer

(2019). Forms and functions of the self-conscious emotions. Trends in Cognitive Sciences, 23(2), 143–157. https://doi.org/10.1016/j.tics.2018.11.007

77.

Sznycer

Al-Shawaf

Bereby-Meyer

Curry

O. S.

De Smet

Ermer

Kim

N. P.

Lopez Seal

M. F.

McClung

J. O. J.

Ohtsubo

Quillien

Schaub

Sell

van Leeuwen

Cosmides

Tooby

(2017). Cross-cultural regularities in the cognitive architecture of pride. Proceedings of the National Academy of Sciences, 114(8), 1874–1879. https://doi.org/10.1073/pnas.1614389114

78.

Sznycer

Xygalatas

Alami

X.-F.

Ananyeva

K. I.

Fukushima

Hitokoto

Kharitonov

A. N.

Koster

J. M.

Onyishi

C. N.

Onyishi

I. E.

Romero

P. P.

Takemura

Zhuang

J.-Y.

Cosmides

Tooby

(2018). Invariances in the architecture of pride across small-scale societies. Proceedings of the National Academy of Sciences, 115(33), 8322–8327. https://doi.org/10.1073/pnas.1808418115

79.

Tangney

J. P.

Miller

R. S.

Flicker

Barlow

D. H.

(1996). Are shame, guilt, and embarrassment distinct emotions? Journal of Personality and Social Psychology, 70(6), 1256–1269. https://doi.org/10.1037/0022-3514.70.6.1256

80.

Tangney

J. P.

Stuewig

Martinez

A. G.

(2014). Two faces of shame: Understanding shame and guilt in the prediction of JAil inmates’ recidivism. Psychological Science, 25(3), 799–805. https://doi.org/10.1177/0956797613508790

81.

Tangney

J. P.

Stuewig

Mashek

(2007). Moral emotions and moral behavior. Annual Review of Psychology, 58, 345–372. https://doi.org/10.1146/annurev.psych.56.091103.070145.Moral

82.

Tangney

J. P.

Wagner

Gramzow

Flicker

Harman

Marschall

Maxfield

(1992). Proneness to shame, proneness to guilt, and psychopathology. Journal of Abnormal Psychology, 101(3), 469–478. https://doi.org/10.1037/0021-843X.101.3.469

83.

Thomaes

Bushman

B. J.

Nezlek

J. B.

William

(2011). Turning shame inside-out: “Humiliated fury” in young adolescents. Emotion, 11(4), 786–793. https://doi.org/10.1037/a0023403

84.

Tracy

J. L.

Robins

R. W.

(2004a). Putting the self into self-conscious emotions: A theoretical model. Psychological Inquiry, 15(2), 103–125. https://doi.org/10.1207/s15327965pli1502

85.

Tracy

J. L.

Robins

R. W.

(2004b). Show your pride: Evidence for a discrete emotion expression. Psychological Science, 15(3), 194–197. https://doi.org/10.1111/j.0956-7976.2004.01503008.x

86.

Tracy

J. L.

Robins

R. W.

(2007). The psychological structure of pride: A tale of two facets. Journal of Personality and Social Psychology, 92(3), 506–525. https://doi.org/10.1037/0022-3514.92.3.506

87.

Vandenberg

R. J.

Lance

C. E.

(2000). A review and synthesis of the measurement invariance literature: Suggestions, practices, and recommendations for organizational research. Organizational Research Methods, 3(1), 4–69. https://doi.org/10.1177/109442810031002

88.

Veiga

Leng

De Cachucho

Ketelaar

Kok

J. N.

Knobbe

Neto

Rieffe

(2017). Social competence at the playground: Preschoolers during recess. Infant and Child Development, 26(1), e1957. https://doi.org/10.1002/icd.1957

89.

Veiga

Oosterveld

Fernandes

Rieffe

(2019). Validation of the Portuguese emotion awareness questionnaire for children and adolescents. European Journal of Developmental Psychology, 16(2), 215–224. https://doi.org/10.1080/17405629.2017.1344124

90.

Williams

L. A.

DeSteno

(2008). Pride and perseverance: The motivational role of pride. Journal of Personality and Social Psychology, 94(6), 1007–1017. https://doi.org/10.1037/0022-3514.94.6.1007

91.

Yoshioka

M. R.

Choi

D. Y.

(2005). Culture and interpersonal violence research: Paradigm shift to create a full continuum of domestic violence services. Journal of Interpersonal Violence, 20(4), 513–519. https://doi.org/10.1177/0886260504267758

92.

Zahn-Waxler

Radke-Yarrow

Wagner

Chapman

(1992). Development of concern for others. Developmental Psychology, 28(1), 126–136. https://doi.org/10.1037/0012-1649.28.1.126