Use of the Brief Shame and Guilt Questionnaire in Deaf and Hard of Hearing Children and Adolescents

Abstract

No assessment tools are available to measure shame and guilt in children who are deaf or hard of hearing (DHH), while these self-conscious emotions might play a role in the frequently noted social and behavioral problems in this group. Therefore, the aim of this study was to validate the Brief Shame and Guilt Questionnaire (BSGQ) in DHH children. In addition, we examined associations of shame and guilt with social anxiety, self-esteem, delinquency, and psychopathic behaviors. A sum of 225 hearing (M_age = 11.62 years) and 108 DHH (M_age = 11.82 years) participants completed the self-report BSGQ. Multigroup confirmatory factor analysis confirmed the two-factor structure (i.e., shame and guilt) of the BSGQ in the DHH group. Measurement invariance was established across both groups. However, the DHH group reported lower levels of self-conscious emotions in comparison with the hearing group. The BSGQ showed good concurrent validity, where shame was associated with higher levels of social anxiety and lower levels of self-esteem, and guilt was associated with lower levels of delinquency and psychopathic behavior in both groups. Future research should investigate the potential behavioral consequences of lower reported levels of self-conscious emotions in DHH youth.

Keywords

hearing loss self-conscious emotions self-report questionnaire validation confirmatory factor analysis

Shame and guilt occur when failing to meet a certain standard, rule, or goal. Yet shame relates to an unwanted identity, while guilt emerges when causing harm to someone else (Olthof, Ferguson, Bloemers, & Deij, 2004). Consequently, shame is associated with more internalizing problems, such as social anxiety and low self-esteem (Gruenewald, Kemeny, Aziz, & Fahey, 2004; Hedman, Strom, Stunkel, & Mortberg, 2013), while guilt is associated with less externalizing behaviors, such as delinquency and psychopathy (Huesmann, Eron, & Dubow, 2002; Stuewig & McCloskey, 2005).

The development of shame and guilt depends on opportunities for social and emotional learning in the context of the social environment (Rieffe, Netten, Broekhof, & Veiga, 2015). Therefore, these emotions may develop less well in children whose access to the social environment is limited by communication challenges, such as hearing loss. Deaf and hard of hearing (DHH) children show more antisocial behaviors than their hearing peers (Coll, Cutler, Thobro, Haas, & Powell, 2009; Theunissen, Rieffe, Kouwenberg, et al., 2014; Theunissen, Rieffe, Netten, Briaire, Soede, Schoones, et al., 2014), and this may be related to lower levels of guilt in this particular group. Albeit lower levels of shame/guilt are found in a study with DHH toddlers (Ketelaar, Wiefferink, Frijns, Broekhof, & Rieffe, 2015), to the best of our knowledge, no studies have yet examined these emotions and their relations with antisocial behavior and psychopathology in DHH youth. However, there are no questionnaires available to assess self-conscious emotions in DHH youth. Therefore, the central aim of this study is to validate the Brief Shame and Guilt Questionnaire (BSGQ) in DHH children (Novin & Rieffe, 2015).

Shame and Guilt

Shame and guilt belong to a special class of emotions, known as self-conscious emotions. Both emotions require self-evaluative processes that occur when failing to meet a certain social standard, rule, or goal (Lewis, 2000; Tracy & Robins, 2004). However, even though shame and guilt are associated, a broad body of literature has emphasized that these emotions are distinct in terms of situational antecedents, appraisals regarding the cause, and subsequent action tendencies (Lewis, 2000; Olthof, Schouten, Kuiper, Stegge, & Jennekens-Schinkel, 2000).

Shame arises in response to an event in which one makes a negative self-evaluation and fears being negatively evaluated by important others (Olthof, 2012; Olthof et al., 2004). In the case of shame, this negative self-evaluation contains a global and stable cause (e.g., “I am an incompetent person”; Tracy & Robins, 2006). Shame elicits feelings of worthlessness and an urge to escape from the evoking social situation, which result in avoidant and withdrawn behaviors (Lindsay-Hartz, 1984; Tangney & Dearing, 2002).

Guilt is experienced when an individual feels responsible for harm caused to another person. In the case of guilt, the accompanying negative self-evaluation focuses on the specific behavior, which is attributed to a specific and unstable cause (e.g., “I did not handle this well”; Tracy & Robins, 2006). Regret over this specific behavior then motivates the individual to attempt to repair the relationship, for example by confessing, apologizing, or restoring the situation (Lindsay-Hartz, 1984; Tangney & Dearing, 2002; Tangney, Wagner, & Gramzow, 1992).

Consequently, shame and guilt are differently related to various psychological and behavioral problems. Higher levels of shame involve degrading and devaluing the self, which promote risk for low self-esteem (Tangney & Dearing, 2002; Tangney, Stuewig, Mashek, & Hastings, 2011). They also involve increased concern about others’ negative judgments, which is also characteristic for individuals with social anxiety (Fergus, Valentiner, McGrath, & Jencius, 2010). Yet higher levels of guilt are unrelated to indices of low self-worth and social withdrawal. Instead, higher levels of guilt are related to lower levels of externalizing, norm-violating behaviors, like delinquency and psychopathic behaviors, and help prevent the individual from harming other people (Rebellon, Manasse, Agnew, Van Gundy, & Cohn, 2016; Stuewig & McCloskey, 2005).

Shame and Guilt in Children Who Are Deaf or Hard of Hearing

The distinct contributions of shame and guilt to psychopathology and behavioral problems have been observed in non-DHH children and adolescents (Ferguson, Stegge, Miller, & Olsen, 1999; Stuewig et al., 2015). Yet, to date, the role of self-conscious emotions in the frequently noted social difficulties and problem behavior of DHH children appears to have been overlooked (Theunissen, Rieffe, Netten, Briaire, Soede, Schoones, et al., 2014). DHH children are found to have higher levels of norm-violating behaviors, such as psychopathy and conduct disorder (Coll et al., 2009; Theunissen, Rieffe, Kouwenberg, et al., 2014; Theunissen, Rieffe, Netten, Briaire, Soede, Schoones, et al., 2014).

The vast majority of DHH children are born to hearing parents, and this poses a challenge to the development of high-quality communication (Marschark & Wauters, 2008). DHH children not only have fewer means to engage in conversations with their (mainly hearing) caregivers and peers but they also miss out on overhearing others’ conversations or other kinds of social interactions, resulting in fewer opportunities for social learning. In turn, this provides DHH children with fewer opportunities to acquire a proficient emotional competence, including self-conscious emotions (Rieffe et al., 2015).

Self-conscious emotions arise in light of social standards and expected negative evaluations by others. Yet social standards are learnt implicitly, through social learning, thus more difficult to pick up from a social environment to which one has less access, as is the case for DHH children. Additionally, DHH children receive less specific feedback on their own behavior by their caregivers (Rieffe et al., 2015). An extra disadvantage for DHH children is their impaired Theory of Mind, which is the ability to take others’ perspectives in daily situations (Ketelaar, Rieffe, Wiefferink, & Frijns, 2012; Netten et al., 2015), which could hamper the DHH children in anticipating negative evaluations by others. Taken together, these findings suggest that DHH children have fewer means for developing a thorough understanding of self-conscious emotions, as compared with their hearing peers. This supposition is supported by results from one recent observation study involving DHH toddlers, which found lower levels of shame and guilt expression in DHH toddlers than in a hearing control group, in response to shame and guilt inducing events (Ketelaar et al., 2015). But to the best of our knowledge, no other studies have yet investigated experiences of shame and guilt in the DHH population. This could be explained by a lack of assessment tools that are appropriate for measuring shame and guilt in DHH children and adolescents.

Self-Reports Suitable for DHH Children and Adolescents

Administration of self-report questionnaires in DHH children requires several special considerations. First, DHH children have a higher incidence of language delays (Marschark & Wauters, 2008), so simple grammar and syntax must be used for each item, to avoid misinterpretation. Second, item content must be uniformly appropriate for hearing and DHH children alike. For example, if an item asks participants to imagine that they failed a foreign language listening test, DHH children would interpret this differently from hearing children because of the impact of their hearing loss. Third, the use of hypothetical situations requires less abstract thinking and less sophisticated linguistic capacities compared with self-reports where participants are asked to rate the applicability of various statements about their general tendency to experience certain feelings, cognitions, or behaviors. Therefore, researchers have advocated the use of scenario-based self-reports to measure and differentiate shame and guilt in children successfully (e.g., Tangney, 1996). Fourth, although there is no difference in performance between term-based response scales (i.e., I would feel not/a little/very guilty) and correlate-based response scales (i.e., I would want to apologize/my face would turn red; Olthof et al., 2000), DHH children may be less familiar with the correlate-based responses as these often use symbolic language. Therefore, the response scale should be term-based. And fifth, translations in sign language should be made available, since reliabilities for self-report questionnaires have been found to increase when items are presented in a child’s native language, or for DHH children in their preferred mode of communication (Cornes, Rohan, Napier, & Rey, 2006).

Both the Test of Self-Conscious Affect for Children (Tangney, Wagner, Burggraf, Gramzow, & Fletcher, 1990) and the Self-Conscious Emotions: Maladaptive and Adaptive Scale (Stegge & Ferguson, 1994) are widely used self-reports that use a scenario approach. The BSGQ is a simplified form of the Self-Conscious Emotions: Maladaptive and Adaptive Scale developed to address the needs of children with language impairments, such as children with hearing loss, autism, or language disorders (Novin & Rieffe, 2015). It is also available in Dutch Sign Language. The BSGQ places minimal demands on language capacities, and consists of 12 short descriptions of shame- or guilt-evoking scenarios, using simple grammar and syntax (Novin & Rieffe, 2015; see Table 1 for item content). All items are equally applicable to hearing and DHH children alike. Children are asked to imagine themselves in a described situation and rate the intensity of their anticipated feelings of shame or guilt (term-based response scale).

Table 1.

Questionnaire Items, Standardized Factor Loadings for the Hearing, and DHH Group Separately.

	Hearing	DHH
	n = 225	n = 108
Factor 1: Guilt
1. Your classmate is using the red pen the whole time. You also need the pen. You snatch away the pen.	.343	.546
3. You are riding your bike on the pavement. You are going really fast. Suddenly, a little girl is standing there and you bump into her.	.694	.819
5. You want to go home quickly. The little girl from next door drops her marbles. You do not help her, because you are in a hurry.	.561	.676
7. Your classmate worked a long time on a painting. But you do not watch out. You knock over a glass of water on his drawing. Everything spills over the painting. The painting is totally ruined.	.942	.925
8. Your classmate has not finished her essay on time. She asks you for help. You do not help her, because you do not feel like it.	.672	.703
10. There is only one cookie left in the cookie jar. You quickly put it in your mouth. Now your friend does not have a cookie.	.649	.618
Factor 2: Shame
2. You are walking in the middle of a busy shopping street. You trip. All your books and pens fall out of your bag on the street.	.774	.765
4. You get a very bad grade at school.	.572	.473
6. You are going to school. You have cut your own hair. You feel stupid.	.757	.869
9. You fall from your bike onto the pavement. People stop to watch. You leave quickly.	.770	.712
11. You are standing in front of the class. You have to give a talk. Everyone is looking at you. You forget what you wanted to say.	.673	.886
12. You are at your classmate’s house for the first time. You get a glass with chocolate milk. You trip on the carpet. The chocolate milk falls out of your hands.	.763	.821

Note. DHH = deaf or hard of hearing.

The BSGQ was previously validated in a hearing sample of Dutch children of 9- to 15-year-old children, confirming the two-factor structure and good reliability (i.e., Cronbach’s alphas: shame = .80, guilt = .78). In addition, the BSGQ showed good concurrent validity, with shame being uniquely associated with social anxiety and worry, and guilt being related to lower levels of conduct problems (Novin & Rieffe, 2015).

Given the positive characteristics of the BSGQ, we aimed to validate this self-report questionnaire in DHH children and adolescents from 9 to 15 years old. It is characteristic for this period in life that children prefer to spend the majority of their leisure time with peers (Brown, 2004). A need to belong and to be accepted by peers makes children more susceptible to social evaluation (Blakemore & Mills, 2014; Somerville, 2013; van Hoorn, van Dijk, Meuwese, Rieffe, & Crone, 2016). This motivates young adolescents to evaluate themselves through the eyes of others within social situations, paving the way for more frequent shame experiences (Reimer, 1996). Children at this age also gain increasing independence, and are gradually given more freedom. Without constant adult supervision, children become responsible for their own behavioral decisions (Wray-Lake, Crouter, & McHale, 2010). But in order to behave prosocially and make the right choices, children need a moral compass, to overcome the temptation to indulge in self-centered behaviors (e.g., stealing money, blaming others for their own mistakes). Feelings of guilt become increasingly important in this period of life, for the anticipation of guilt can serve as a motivator to behave according to the social standards (Krettenauer & Eichler, 2006; Lake, Lane, & Harris, 1995; Tangney, Stuewig, & Mashek, 2007).

The BSGQ is appropriate for measuring shame and guilt in children between 9 and 15 years old because children are the best informants on their own internal feelings states and they can meaningfully and reliably report them from the age of 8 years (Berti, Garattoni, & Venturini, 2000; Ferguson & Stegge, 1995). In addition, children older than 9 years are able to discriminate shame and guilt accurately (Olthof et al., 2000). To optimize suitability for DHH children, we provided a video translation in sign language for each item.

The Present Study

The central goal of this study was to examine the extent to which the BSGQ could successfully measure shame and guilt in a sample of DHH children and whether results from DHH children obtained on the BSGQ could be meaningfully compared with those of their hearing peers. To achieve this goal, we evaluated construct and concurrent validity. To examine the construct validity, we first assessed the hypothesized two-factor structure (i.e., shame and guilt) across both DHH and hearing children. Second, we assessed the reliability of the shame and guilt scales for each group separately. In the event that measurement invariance was established, we compared levels of shame and guilt between DHH and hearing participants. We predicted that DHH children would report lower levels of shame and guilt compared with their hearing peers, since a previous study indicated DHH children express less shame and guilt (Ketelaar et al., 2015) and DHH children are known to experience greater difficulty appreciating other people’s perspectives (Ketelaar et al., 2012; Netten et al., 2015; Rieffe, Dirks, van Vlerken, & Veiga, 2017).

To evaluate the concurrent validity of the BSGQ, children completed self-report questionnaires regarding social anxiety, self-esteem, and delinquency. Parents reported on their children’s levels of psychopathic behaviors. We predicted that higher levels of shame would be associated with more symptoms of social anxiety (Fergus et al., 2010) and lower self-esteem (Tangney et al., 2011), and we expected that higher levels of guilt would be associated with lower levels of delinquency and psychopathic behaviors (Huesmann et al., 2002; Stuewig & McCloskey, 2005; Tangney, Stuewig, & Martinez, 2014). We expected that these relationships would not differ between DHH and hearing children.

Method

Participants

Participants consisted of 225 hearing children (M_age = 11.62 years, SD = 1.37, 42.2% boys) and 108 DHH children (M_age = 11.82 years, SD = 1.46, 46.3% boys) between 9 and 15 years old. Independent t tests indicated that the hearing and DHH group did not differ in age, t(331) = −1.22, p = .223, intelligence, t(298) = 1.05, p = .293, and socioeconomic status, t(249) = .73, p = .469. In addition, a chi-square analysis revealed no differences in gender distribution, χ²(1, N = 333) = .49, p = .483.

DHH children were recruited through the distribution of leaflets about the study, which indicated a website where parents could go to register if children wanted to participate. Distribution of the leaflets took place at (a) ENT departments of hospitals, (b) speech and hearing centres, (c) special-needs schools providing education to DHH students, and (d) magazines and websites for the target population. All DHH participants were born to hearing parents. Hearing children were recruited from mainstream primary and secondary schools. Inclusion criteria for both groups were (a) no diagnosed developmental disabilities or learning difficulties, such as autism spectrum disorder, ADHD, and/or dyslexia; (b) normal intellectual functioning; and (c) living in the Netherlands or the Dutch speaking part of Belgium. In addition, DHH children were only included if they had an unaided hearing loss of at least 40 dB in both ears (i.e., moderate hearing loss) that was detected prelingually or perilingually. This criteria of >40 dB hearing loss is a standard set by the World Health Organization, and indicates an individual has frequent difficulties hearing normal speech, even at close distances. The Ethics committee of Leiden University granted permission for the study and all primary caregivers gave written consent before testing.

Materials

Intelligence and Socioeconomic Status

Nonverbal intelligence was assessed using two subscales of the Wechsler Intelligence Scale for Children–Third edition (Wechsler, 1991). In the first subtest, Block Design, children were given red and white colored square blocks and asked to arrange them to form geometric designs presented by the test leader in a two-dimensional image. In the second subtest, Picture Arrangement, children were given the task of arranging cartoon pictures from left to right in chronological order. Raw scores for both subtests were converted to norm scores corrected for age. The mean score of these two norm scores was used to examine group differences (see participants).

Socioeconomic status was assessed by requesting that parents indicate maternal and paternal educational level (1 = no/primary education, 2 = lower general secondary education, 3 = higher general secondary education, 4 = college/university) and net household income (1 ≤ €15,000, 2 = €15,000-€30,000, 3 = €30,000-€ 45,000, 4 = €45,000-€60,000, 5 ≥ €60,000). Net household income was converted to a 4-point scale, and a mean score was calculated based on these three indicators. The mean score was used to examine group differences on socioeconomic status.

Questionnaires

The BSGQ for Children (Novin & Rieffe, 2015) consists of 12 emotion-eliciting scenarios. Children were instructed to imagine themselves being in a described scenario and asked to rate how ashamed or guilty they would feel on a 3-point scale (1 = not at all, 2 = a little, 3 = a lot). Six scenarios were designed to describe behaviors that would cause harm to another and elicit guilt (e.g., “There is one biscuit left in the biscuit tin. You quickly put it in your mouth. Now your friend does not have a biscuit”), and six vignettes were designed to describe incompetent behavior in the presence of others without causing any harm to another and elicit shame (e.g., “You get a very bad grade in school”). The content of the items is presented in Table 1. Mean scores were calculated per scale.

The Social Anxiety Questionnaire (Theunissen et al., 2012) consists of six descriptions of socially charged situations, such as “talking to someone I don’t know” and “entering a room with strangers.” Children were asked to report the intensity of their fear for the described situation (1 = no fear, 2 = a little fearful, 3 = a lot of fear). Data of 1 DHH child (<.01%) is missing due to a computer failure in administering this questionnaire. The internal consistency of this questionnaire was rated as good (see Table 2).

Table 2.

Psychometric Properties of Questionnaires on Psychological and Behavioral Problems.

	No. of items	N participants		Min-Max	Average interitem correlation		Cronbach’s α
	No. of items	H	DHH	Min-Max	H	DHH	H	DHH
Social anxiety	6	225	107	1-3	.45	.36	.83	.77
Self-esteem	5	225	103	1-3	.23	.29	.62	.67
Delinquency	9	225	108	1-3	.21	.14	.70	.62
Psychopathic behaviors	20	175	88	1-3	.16	.16	.77	.79

Note. H = hearing; DHH = deaf or hard of hearing.

To assess children’s global self-esteem, we used the corresponding scale of the Children’s Self-Confidence and Acceptance Scale (Rieffe et al., 2007; Theunissen, Rieffe, Netten, Briaire, Soede, Kouwenberg, et al., 2014). Children were asked to consider how well five general statements, concerning the self, applied to them (e.g., “I like myself”), and to rate each one on a 3-point scale (1 = not true, 2 = sometimes true, 3 = often true). Data were missing for five DHH children (4.6%) due to a computer failure in administering this questionnaire. The internal consistency of this scale was rated as sufficient (see Table 2).

The Delinquency Questionnaire (Baerveldt, Van Rossem, & Vermande, 2003; Theunissen, Rieffe, Kouwenberg, et al., 2014) is a self-report measure that includes statements about 10 minor delinquent offences (e.g., “I stole money from my parents”). Children were asked to report their engagement in these behaviors according to a 3-point scale: 1 = never, 2 = once or twice, 3 = three times or more. This questionnaire was rated as showing undesirable to reasonable reliability (see Table 2).

The Psychopathy Screening Device (Frick, Obrien, Wootton, & Mcburnett, 1994; Theunissen, Rieffe, Kouwenberg, et al., 2014) is a parent questionnaire that measures psychopathic behaviors of the child (e.g., the child blames others for his or her mistakes). Parents were asked to rate how much the statements applied to their child (1 = not true, 2 = sometimes true, 3 = certainly true). Parents of 20 DHH children (18.5%) and 50 hearing children (22.2%) did not complete or return the questionnaire. The reliability of the questionnaire was rated as good (see Table 2).

Of the 259 completed Psychopathy Screening Devices, 196 were completed by the mother (74.5%), 37 by the father (14.1%), 16 by mother and father together (6.1%), and 1 by an older brother (.4%). For 13 questionnaires, the respondent was unknown. A one-way analysis of variance, including the three main respondent groups (i.e., mother, father, both) indicated no effect for the type of respondent on the psychopathy measure, F(2, 248) = .21, p = .935.

Procedure

Self-report questionnaires were administered to children individually in a quiet room at their home or school. Children were seated in front of a computer screen and assured that all answers would be kept confidential and processed anonymously. To ensure the questionnaires would be appropriate for DHH children, only questionnaires were selected that were previously used in this population and in which no complex grammar was used (Theunissen et al., 2012; Theunissen, Rieffe, Kouwenberg, et al., 2014; Theunissen, Rieffe, Netten, Briaire, Soede, Kouwenberg, et al., 2014). For all participants, questions were presented one by one on the computer screen. Administration of the questionnaire was uniform between groups, except DHH participants also viewed a video clip in which a sign language interpreter provided a translation. DHH participants could repeat these video clips as often as desired. During administration of the questionnaires, a test leader was present for both hearing and DHH children to answer possible questions from participants. DHH children were only tested by test leaders who were proficient in sign language. No questions were asked regarding item content of the BSGQ. All children were given a small present (a comic book) after filling out the questionnaires to thank them for their participation.

Parents were sent (electronic) mail with the Psychopathy Screening Device and questions about their socioeconomic status. Parents were requested to return the questionnaires within 2 weeks after their child’s test session.

Statistical Analyses

To evaluate the underlying factorial structure, confirmatory factor analyses (CFAs) were conducted in R version 3.2.1 using packages lavaan (Rosseel, 2012) and semTools (semTools Contributors, 2015). To take into account the categorical nature of our indicators, robust mean- and variance-adjusted weighted least-squares estimation was used (Finney & DiStefano, 2013). This estimation technique performs adequately in small samples and little bias occurs in case of multivariate nonnormality (Flora & Curran, 2004).

The hypothesized two-factor model was tested with a CFA for the hearing and DHH group separately (see Figure 1). To test for measurement invariance of the BSGQ across both groups, we performed several multigroup CFA models. First, we examined the hypothesized model simultaneously in both groups without constraints. This so-called configural model indicates whether overall model structure is similar across groups (Jöreskog, 1971). Second, we tested for metric invariance by constraining factor loadings, so they were the same across groups. Metric invariance assumes that each item is interpreted and responded to in the same way by the respondents. Third, we tested for scalar invariance by constraining intercepts equal across groups. Scalar invariance assumes individuals with the same actual level of shame/guilt would report identical on related items in the questionnaire, regardless of their hearing status (Byrne, 2006, 2008; Milfont & Fischer, 2010; Vandenberg & Lance, 2000).

Figure 1.

Hypothesized model for the hearing and DHH group.

Model fit was assessed using the χ²/df ratio. Kline (2005) argues that a ratio of less than 3:1 indicates good model fit. In addition, the comparative fit index (CFI), the Tucker–Lewis Index (TLI) and the root mean square error of approximation (RMSEA) were reported. CFI and TLI values above .90 indicate acceptable fit and values above .95 represent good fit (Hu & Bentler, 1999). For RMSEA, values below .05 suggest good fit and values up to .08 indicate reasonable model fit (Browne & Cudeck, 1989). Measurement invariance was evaluated comparing the nested models using Δχ² and ΔCFI with a cutoff point of <0.005 (Byrne, 2006; Chen, 2007).

Reliability analyses were conducted in IBM SPSS version 23. Internal consistency reliabilities for the BSGQ were examined using Cronbach’s alpha. The following ranges for evaluating Cronbach’s alpha were used: <.60 is unacceptable; ≥.60 is undesirable; >.65 is minimally acceptable, >.70 is good; and >.80 is very good (DeVellis, 2003). In addition, average interitem correlations were calculated. According to Clark and Watson (1995), average interitem correlations should fall within a .15 to .50 range.

To test whether DHH children differed from hearing children in levels of shame and guilt, Mann–Whitney U tests were conducted. In addition, r was reported as an index for effect size for which an effect size of .10 is considered small, ≥.30 is medium, and ≥.50 is large (Rosenthal, 1991).

Concurrent validity was evaluated using Spearman rank order correlation coefficients to assess links of shame and guilt with delinquency, psychopathic behaviors, social anxiety, and self-esteem. In addition, we assessed these links using partial correlations in which the other self-conscious emotion was controlled for. To find out whether correlations differed in strength between hearing and DHH participants, Fisher r-to-z transformations were carried out.

Results

Construct Validity

The hypothesized two-factor model resulted in adequate to good fit indices in both the hearing group, χ²/df = 1.97, CFI = .954, TLI = .943, RMSEA = .066, and the DHH group, χ²/df = 1.45, CFI = .975, TLI = .969, RMSEA = .065. Standardized factor loadings ranged from .343 to .942 (see Table 1). The correlation between shame and guilt was .72 for the hearing group and .77 for the DHH group.

The configural model confirmed that the hypothesized model fits well in both groups, χ²/df = 1.69, CFI = .965, TLI = .956, and RMSEA = .065. Testing metric invariance did not yield a significantly higher χ² value compared with the configural model, p = .208. Moreover, the ΔCFI decreased .002 points providing support for full metric invariance (see Table 3). Testing scalar invariance did not result in a substantial increase in the χ² value, p = .396. In addition, a ΔCFI value of less than .001 indicated that constraining intercepts did not lead to a decrease in model fit. Therefore, full scalar invariance can be assumed (see Table 3).

Table 3.

Fit Indices for the Multigroup Models of the Two-Factor Model of the Brief Shame–Guilt Questionnaire.

	Model fit indices						Indices of model fit differences
	χ²	df	χ²/df	CFI	TLI	RMSEA	ΔCFI	Δχ²	Δdf	p
Multigroup models
Configural model	179.236*	106	1.69	.965	.956	.065	—	—	—
Metric invariance	192.526*	116	1.66	.963	.958	.063	.002	13.290	10	.208
Scalar invariance	203.048*	126	1.61	.963	.961	.061	<.001	10.522	10	.396

Note. TLI = Tucker–Lewis Index; df = degrees of freedom; CFI = comparative fit index; RMSEA = root mean square error of approximation.

p < .001.

Reliability

The psychometric properties of the BSGQ are shown in Table 4. The self-conscious emotion scales showed good reliability with Cronbach’s alpha values ranging from .69 to .83. The interitem correlations were rated as acceptable to good (i.e., range = .28-.45).

Table 4.

Internal Consistency, Mean Scores, and Standard Deviations of the BSGQ Per Group.

	No. of items	N participants		Average interitem correlation		Cronbach’s α		Mean scores (SD)
	No. of items	H	DHH	H	DHH	H	DHH	H	DHH
BSGQ
Shame*	6	225	108	0.38	0.45	0.79	0.83	2.34 (0.49)	2.17 (0.55)
Guilt*	6	225	108	0.28	0.39	0.69	0.79	2.35 (0.41)	2.13 (0.49)

Note. H = hearing; DHH = deaf or hard of hearing; BSGQ = Brief Shame and Guilt Questionnaire. An asterisk indicates group differences at p ≤≤ .01 as evidenced by a Mann–Whitney U test.

Group Differences

The mean scores and standard deviations of the BSGQ are shown in Table 4. Levels of guilt and shame were compared between the hearing and DHH group with two Mann–Whitney U tests. DHH children reported lower levels of shame and guilt, as compared with their hearing peers (U = 10029, z = −2.59, p = .010, r = .14, and, U = 8914.5, z = −3.96, p < .001, r = −.22, respectively).

Concurrent Validity

Table 5 shows the outcomes of the correlations of shame and guilt with social anxiety, self-esteem, delinquency, and psychopathic behaviors as dependent variables. Outcomes indicate shame correlated positively with social anxiety, r(332) = .39, p < .001, and negatively with self-esteem, r(328) = −.13, p = .021, including when guilt was controlled for, r(332) = .31, p < .001, and r(328) = −.15, p = .006, respectively. Shame was unrelated to delinquency, r(333) = −.02, p = .789, and psychopathic behaviors, r(263) = −.01, p = .905.

Table 5.

Bivariate and Partial Spearman Correlations for Shame and Guilt With Social Anxiety, Self-Esteem, Delinquency, and Psychopathy Collapsed Over Group.

	Shame				Guilt
	Bivariate correlations		Partial correlations		Bivariate correlations		Partial correlations
	r	95% CI	r	95% CI	r	95% CI	r	95% CI
Social anxiety	.39 ***	[.29, .48]	.31 ***	[.21, .40]	.27 ***	[.16, .36]	.09	[−.02, .19]
Self-esteem	−.13 *	[−.23, −.02]	−.15 **	[−.26, −.04]	.01	[−.10, .11]	.08	[−.03, .19]
Delinquency	−.02	[−.12, .09]	.09	[−.02, .20]	−.18 ***	[−.28, −.08]	−.20 ***	[−.30, −.10]
Psychopathic behaviors	−.01	[−.13, .11]	.08	[−.04, .20]	−.15 *	[−.27, −.03]	−.17 **	[−.29, −.05]

Note. H = hearing; DHH = deaf or hard of hearing; CI = confidence interval. The strengths of the correlations were examined using Fisher r-to-z transformations and there were no differences found between the Hearing and DHH group.

p < .05. **p ≤.01. ***p ≤.001.

In addition, guilt was positively correlated with higher levels of social anxiety, r(332) = .27, p < .001, lower levels of delinquency, r(333) = −.18, p = .001, and psychopathic behaviors, r(333) = −.15, p = .013. However, guilt was unrelated to self-esteem, r(328) = .01, p = .915. After controlling for shame, the negative correlation of guilt with delinquency, r(333) = −.20, p < .001, and psychopathic behaviors, r(263) = −.17, p = .005, remained. However, guilt was no longer found to be associated with social anxiety, r(332) = .09, p = .104.

We tested for group differences in the strength of correlations between shame and guilt with social anxiety, self-esteem, delinquency, and psychopathic behaviors. Using Fisher r-to-z transformation, a z value score was calculated to assess whether the correlation coefficients differed between hearing and DHH children. The strength of these relationships did not differ between hearing and DHH children. Therefore, only the overall correlations (where both groups were combined) are displayed in Table 5.

Discussion

The aim of the present study was to validate the self-report BSGQ for DHH children. While administration of self-report questionnaires in DHH children contains many challenges due to the impact of hearing loss on language development and communication, we found full support for the two-factor model with shame and guilt as separate constructs in both the DHH and hearing group. Shame and guilt can be successfully measured in DHH children by using the BSGQ, and their scores on the BSGQ can be reliably compared with those of hearing children. Since children were asked to rate the intensity of their anticipated shame and guilt experiences, these results indicated that DHH children were as able as hearing children to distinguish between shame and guilt verbally. In addition, the reliabilities for both the shame and guilt scales for the DHH children were rated as very good or good (i.e., .83 and .79, respectively). These positive results for construct validity and the psychometric properties of the BSGQ in DHH children could be achieved based on simple item content formulation and the availability of video clips with a sign language interpretation. The video clips were frequently accessed by DHH children who indicated sign language as their preferred mode of communication. We recommend this procedure for developing questionnaires for DHH children who prefer sign language, because they can be tested in a standardized manner while minimizing risk that they will misinterpret item content (Enns & Herman, 2011). However, since we did not test the effectivity of the video clips in sign language for the DHH population directly, this could be tested in a follow-up study.

Notably, DHH children reported lower levels of shame and guilt compared with their hearing peers. Self-conscious emotions fulfill a key social function by motivating a broad range of appropriate behaviors (Giner-Sorolla, 2012). Therefore, a lower intensity in the experience of these particular emotions may have detrimental effects on children’s social and emotional development and functioning. Maintaining relationships could be more challenging for those who experience less guilt. If one does not experience guilt after harming another, one will be less inclined to display reparative behaviors such as apologizing or helping repair damage (Lindsay-Hartz, 1984; Tangney & Dearing, 2002). Expression of guilt provides the receiver with crucial information as it reflects awareness of the harm done, and intention to avoid repeating that behavior in the future. This makes it easier to forgive the other for the misconduct, and helps reinstate the relationship (Giner-Sorolla, 2012). Lower levels of guilt provide less motivation to display appropriate behaviors (Krettenauer & Eichler, 2006; Lake et al., 1995), and lower levels of guilt found in DHH children may explain the higher incidence of problem behaviors in DHH adolescents (Coll et al., 2009; Theunissen, Rieffe, Kouwenberg, et al., 2014; Theunissen, Rieffe, Netten, Briaire, Soede, Schoones, et al., 2014). Clearly, more research is needed to understand the implications of lower levels of reported shame and guilt for social–emotional abilities and problem behaviors in DHH children. Validation of the BSGQ in DHH children makes it possible to begin to study these interrelationships in this population now.

Shame is an overwhelming emotion accompanied by a negative evaluation about the global self, causing individuals to feel incompetent and bad about themselves (Lewis, 2000; Tangney et al., 1992). As expected, we found that children with higher levels of shame had lower self-esteem and more social anxiety. Although guilt was also related to more social anxiety, this association disappeared when we controlled for shame, which aligns with other studies (Gilbert, 2000; Hedman et al., 2013). While we stressed that shame and guilt have distinct features, they also share core characteristics. This is illustrated by the positive correlation we found between shame and guilt (i.e., correlation = .55; p < .001), which is congruent with other studies (Olthof, 2012; Tangney et al., 1992). Previous studies have emphasized the need to control for the shared variance between shame- and guilt-proneness in assessing its relationship with emotional functioning (Spruit, Schalkwijk, Vugt, & Stams, 2016). Future studies could more closely examine the extent to which covariance in shame and guilt affect predictive value for behavioral measures.

Guilt discourages socially inappropriate behavior, and this claim is supported in this study by associations of higher levels of guilt with lower levels of delinquency and psychopathy. This aligns with previous studies stressing the adaptive function of “shame-free” guilt (Spruit et al., 2016). In contrast, the relation between shame and delinquent behavior has been debated in the literature. Some claim that shame is a painful emotion that occurs in light of a transgression and motivates people to prevent experiencing this emotion in the future, and as such, shame inhibits antisocial behaviors (Tangney & Dearing, 2002; Tangney et al., 2007). Others claim that the pain of shame causes individuals to externalize blame, to regain a sense of control over their situation. This has been related to externalizing behaviors, such as aggression and delinquency (Spruit et al., 2016; Stuewig, Tangney, Heigel, Harty, & McCloskey, 2010). In a recent meta-analysis, Spruit et al. (2016) found evidence for neither an inciting nor inhibiting role for shame in delinquent behaviors, while guilt was related to less delinquency. This aligns with our findings, as shame was found to be unrelated to both psychopathy and delinquency, whereas guilt was negatively correlated with both norm-violating behaviors.

This study does have several limitations that need to be addressed. First, the internal consistencies of two scales were lower than the expected value of .70 (i.e., for self-esteem in both groups and delinquency in the DHH group). For this validation study, it was important to select questionnaires that had been used previously in a DHH population, and could show the unique contribution of self-conscious emotions to social–emotional functioning and problem behaviors. This resulted in limited options, stressing that more validation studies for instruments addressing this particular population are needed. Moreover, existing questionnaires may benefit from additional items. This is especially true for the self-esteem scale, which consisted of only five items. These could be developed in future studies. Nevertheless, we did find the predicted relationships for shame and guilt using these questionnaires. Second, our sample consisted of hearing and DHH with average intelligence and no diagnosed developmental disabilities. Our results can therefore not be generalized to children with intelligence below the normal range or with a diagnosed disability (e.g., attention hyperactivity disorder or autism spectrum disorder). Third, common method variance probably influenced our study results. On the one hand, this could have inflated correlations between the study variables (e.g., self-reports with a 3-point scale), while a difference in response format (i.e., how guilty/ashamed do you feel?) could cause a differentiation between shame and guilt based on the measurement method rather than the underlying constructs. However, in this study, the relations of shame and guilt with social anxiety, self-esteem, and delinquency were congruent with prior studies. In addition to self-reports, we also assessed psychopathy through parent report, minimizing the likelihood of common method variance. The relations of psychopathy with shame- and guilt-proneness were also consistent with prior studies (Tangney et al., 2007). Fourth, there can be a considerable overlap in shame and guilt regarding guilt-evoking situations, which cannot be completely ruled out in our measure. Future studies could also take this into account by controlling for shame also in the guilt-evoking situations and vice versa. Fifth, the data gathered in this study are all correlational. This makes it impossible to draw conclusions about causal relationships. Sixth, we did not test convergent validity between the BSGQ and other validated measures of shame and guilt.

Future studies could attempt to discover and analyze the longitudinal relationships between self-conscious emotions and the social and behavioral difficulties in DHH adolescents. The BSGQ could be used to track the development of shame and guilt in DHH individuals from late childhood to middle adolescence, a period in which the anticipation of shame and guilt experiences are known to influence behavior choices (Olthof, 2012; Stuewig et al., 2015). In addition, the questionnaire could help determine whether a lower intensity of guilt contributes to a heightened level of proactive aggression or problems in maintaining friendships in DHH children (Gilman, Easterbrooks, & Frey, 2004; Keilmann, Limberger, & Mann, 2007; Theunissen, Rieffe, Kouwenberg, et al., 2014; Wolters, Knoors, Cillessen, & Verhoeven, 2011). The role of shame in the development of psychopathology is not yet clear. Although shame does seem to contribute to more internalizing symptoms (Gruenewald et al., 2004; Tangney et al., 1992), the protective role in the development of antisocial behaviors cannot be confirmed in this study (Olthof, 2012). Future research could further examine the longitudinal relationships between these variables to further unravel the protective or possible harmful effect of shame. Validation of the BSGQ in DHH adolescents paves the way for future studies to begin to unravel the mystery of the role of self-conscious emotion in the social and emotional development of DHH adolescents.

Footnotes

Acknowledgements

The authors thank all participating children, parents, and schools. In addition, we thank Jennifer Schoerke for correcting our English.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The project was supported by the Innovational Research Incentives Scheme (a VIDI grant) by The Netherlands Organization for Scientific Research (NWO), no. 452-07-004 to Carolien Rieffe.

References

Baerveldt

Van Rossem

Vermande

(2003). Pupils’ delinquency and their social networks: A test of some network assumptions of the ability and inability models of delinquency. Dutch Journal of Social Sciences, 39, 107-125.

Berti

A. E.

Garattoni

Venturini

(2000). The understanding of sadness, guilt, and shame in 5-, 7-, and 9-year-old children. Genetic Social, and General Psychology Monographs, 126, 293-318.

Blakemore

S. J.

Mills

K. L.

(2014). Is adolescence a sensitive period for sociocultural processing? Annual Review of Psychology, 65, 187-207. doi:10.1146/annurev-psych-010213-115202

Brown

B. B.

(2004). Adolescents’ relationships with peers. In Lerner

R. M.

Steinberg

(Eds.), Handbook of adolescent psychology (2nd ed., pp. 363-394). Hoboken, NJ: Wiley.

Browne

M. W.

Cudeck

(1989). Single sample cross-validation indexes for covariance structures. Multivariate Behavioral Research, 24, 445-455. doi:10.1207/s15327906mbr2404_4

Byrne

B. M.

(2006). Structural equation modeling with EQS: Basic concepts, application, and programming. Mahwah, NJ: Lawrence Erlbaum.

Byrne

B. M.

(2008). Testing for multigroup equivalence of a measuring instrument: A walk through the process. Psicothema, 20, 872-882.

Chen

F. F.

(2007). Sensitivity of goodness of fit indexes to lack of measurement invariance. Structural Equation Modeling: A Multidisciplinary Journal, 14, 464-504.

Clark

L. A.

Watson

(1995). Constructing validity: Basic issues in objective scale development. Psychological Assessment, 7, 309-319.

10.

Coll

K. M.

Cutler

M. M.

Thobro

Haas

Powell

(2009). An exploratory study of psychosocial risk behaviors of adolescents who are deaf or hard of hearing: Comparisons and recommendations. American Annals of the Deaf, 154(1), 30-35.

11.

Cornes

Rohan

M. J.

Napier

Rey

J. M.

(2006). Reading the signs: Impact of signed versus written questionnaires on the prevalence of psychopathology among deaf adolescents. Australian & New Zealand Journal of Psychiatry, 40, 665-673. doi:10.1080/j.1440-1614.2006.01866.x

12.

DeVellis

R. F.

(2003). Scale development: Theory and applications (2nd ed., Vol. 26). Thousand Oaks, CA: Sage.

13.

Enns

C. J.

Herman

R. C.

(2011). Adapting the assessing British sign language development: Receptive skills test into American sign language. Journal of Deaf Studies and Deaf Education, 16, 362-374. doi:10.1093/deafed/enr004

14.

Fergus

T. A.

Valentiner

D. P.

McGrath

P. B.

Jencius

(2010). Shame- and guilt-proneness: Relationships with anxiety disorder symptoms in a clinical sample. Journal of Anxiety Disorders, 24, 811-815. doi:10.1016/j.janxdis.2010.06.002

15.

Ferguson

T. J.

Stegge

(1995). Emotional states and traits in children: The case of guilt and shame. In Tangney

J. P.

Fisher

K. W.

(Eds.), Self-conscious emotions: The psychology of shame, guilt, embarrassment, and pride (pp. 174-197). New York, NY: Guilford Press.

16.

Ferguson

T. J.

Stegge

Miller

E. R.

Olsen

M. E.

(1999). Guilt, shame, and symptoms in children. Developmental Psychology, 35, 347-357.

17.

Finney

S. J.

DiStefano

(2013). Nonnormal and categorical data in structural equation models. In Hancock

G. R.

Mueller

R. O.

(Eds.), A second course in structural equation modeling (2nd ed., pp. 439-492). Charlotte, NC: Information Age.

18.

Flora

D. B.

Curran

P. J.

(2004). An empirical evaluation of alternative methods of estimation for confirmatory factor analysis with ordinal data. Psychological Methods, 9, 466-491. doi:10.1037/1082-989x.9.4.466

19.

Frick

P. J.

Obrien

B. S.

Wootton

J. M.

Mcburnett

(1994). Psychopathy and conduct problems in children. Journal of Abnormal Psychology, 103, 700-707. doi:10.1037//0021-843x.103.4.700

20.

Gilbert

(2000). The relationship of shame, social anxiety and depression: The role of the evaluation of social rank. Clinical Psychology & Psychotherapy, 7, 174-189. doi:10.1002/1099-0879(200007)7:3<174::Aid-Cpp236>3.0.Co;2-U

21.

Gilman

Easterbrooks

S. R.

Frey

(2004). A preliminary study of multidimensional life satisfaction among deaf/hard of hearing youth across environmental settings. Social Indicators Research, 66, 143-164. doi:10.1023/B:Soci.0000007495.40790.85

22.

Giner-Sorolla

(2012). Judging passions: Moral emotions in persons and groups. New York, NY: Psychology Press.

23.

Gruenewald

T. L.

Kemeny

M. E.

Aziz

Fahey

J. L.

(2004). Acute threat to the social self: Shame, social self-esteem, and cortisol activity. Psychosomatic Medicine, 66, 915-924. doi:10.1097/01.psy.0000143639.61693.ef

24.

Hedman

Strom

Stunkel

Mortberg

(2013). Shame and guilt in social anxiety disorder: Effects of cognitive behavior therapy and association with social anxiety and depressive symptoms. PLoS ONE, 8(4), e61713. doi:10.1371/journal.pone.0061713

25.

L. T.

Bentler

P. M.

(1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6(1), 1-55. doi:10.1080/10705519909540118

26.

Huesmann

L. R.

Eron

L. D.

Dubow

E. F.

(2002). Childhood predictors of adult criminality: Are all risk factors reflected in childhood aggressiveness? Criminal Behaviour and Mental Health, 12, 185-208.

27.

Jöreskog

K. G.

(1971). Simultaneous factor analysis in several populations. Psychometrika, 36, 409-426. doi:10.1007/Bf02291366

28.

Keilmann

Limberger

Mann

W. J.

(2007). Psychological and physical well-being in hearing-impaired children. International Journal of Pediatric Otorhinolaryngology, 71, 1747-1752. doi:10.1016/j.ijporl.2007.07.013

29.

Ketelaar

Rieffe

Wiefferink

C. H.

Frijns

J. H. M.

(2012). Does hearing lead to understanding? Theory of mind in toddlers and preschoolers with cochlear implants. Journal of Pediatric Psychology, 37, 1041-1050. doi:10.1093/jpepsy/jss086

30.

Ketelaar

Wiefferink

C. H.

Frijns

J. H.

Broekhof

Rieffe

(2015). Preliminary findings on associations between moral emotions and social behavior in young children with normal hearing and with cochlear implants. European Child & Adolescent Psychiatry, 24, 1369-1380. doi:10.1007/s00787-015-0688-2

31.

Kline

R. B.

(2005). Principles and practice of structural equation modeling (2nd ed.). New York, NY: Guilford Press.

32.

Krettenauer

Eichler

(2006). Adolescents’ self-attributed moral emotions following a moral transgression: Relations with delinquency, confidence in moral judgment and age. British Journal of Developmental Psychology, 24, 489-506. doi:10.1348/026151005x50825

33.

Lake

Lane

Harris

P. L.

(1995). The expectation of guilt and resistance to temptation. Early Development and Parenting, 4, 63-73. doi:10.1002/edp.2430040203

34.

Lewis

(2000). Self-conscious emotions: Embarrassment, pride, shame, and guilt. In Lewis

Haviland-Jones

J. M.

(Eds.), Handbook of emotions (2nd ed., pp. 623-636). New York, NY: Guilford Press.

35.

Lindsay-Hartz

(1984). Contrasting experiences of shame and guilt. American Behavioral Scientist, 27, 689-704.

36.

Marschark

Wauters

(2008). Language comprehension and learning by deaf students. In Marschark

Hauser

P. C.

(Eds.), Deaf cognition: Foundations and outcomes (pp. 309-350). New York, NY: Oxford University Press.

37.

Milfont

T. L.

Fischer

(2010). Testing measurement invariance across groups: Applications in cross-cultural research. International Journal of Psychological Research, 3, 111-121.

38.

Netten

A. P.

Rieffe

Theunissen

S. C. P. M.

Soede

Dirks

Briaire

J. J.

Frijns

J. H. M.

(2015). Low empathy in deaf and hard of hearing (pre)adolescents compared to normal hearing controls. PLoS ONE, 10(4), e0124102. doi:10.1371/journal.pone.0124102

39.

Novin

Rieffe

(2015). Validation of the Brief Shame and Guilt Questionnaire for Children. Personality and Individual Differences, 85, 56-59. doi:10.1016/j.paid.2015.04.028

40.

Olthof

(2012). Anticipated feelings of guilt and shame as predictors of early adolescents’ antisocial and prosocial interpersonal behaviour. European Journal of Developmental Psychology, 9, 371-388. doi:10.1080/17405629.2012.680300

41.

Olthof

Ferguson

Bloemers

Deij

(2004). Morality- and identity-related antecedents of children’s guilt and shame attributions in events involving physical illness. Cognition and Emotion, 18, 383-404. doi:10.1080/02699930341000077

42.

Olthof

Schouten

Kuiper

Stegge

Jennekens-Schinkel

(2000). Shame and guilt in children: Differential situational antecedents and experiential correlates. British Journal of Developmental Psychology, 18, 51-64. doi:10.1348/026151000165562

43.

Rebellon

C. J.

Manasse

M. E.

Agnew

Van Gundy

K. T.

Cohn

E. S.

(2016). The relationship between gender and delinquency: Assessing the mediating role of anticipated guilt. Journal of Criminal Justice, 44, 77-88. doi:10.1016/j.jcrimjus.2015.11.006

44.

Reimer

M. S.

(1996). “Sinking into the ground”: The development and consequences of shame in adolescence. Developmental Review, 16, 321-363. doi:10.1006/drev.1996.0015

45.

Rieffe

Dirks

van Vlerken

Veiga

(2017). The empathic mind in children with communication impairments: The case of children who are Deaf or Hard of Hearing (DHH); children with an Autism Spectrum Disorder (ASD); and children with Specific Language Impairments (SLI). In Slaughter

de Rosnay

(Eds.), Theory of mind development in context (pp. 106-120). Hove, England: Psychology Press.

46.

Rieffe

Netten

A. P.

Broekhof

Veiga

(2015). The role of environment in children’s emotion socialization: The case of deaf or hard-of-hearing (DHH) children. In Marschark

Knoors

H. E. T.

(Eds.), Educating deaf learners: Creating global evidence base (pp. 369-388). London, England: Oxford University Press.

47.

Rieffe

Terwogt

M. M.

Bosch

J. D.

Kneepkens

C. M. F.

Douwes

A. C.

Jellesma

F. C.

(2007). Interaction between emotions and somatic complaints in children who did or did not seek medical care. Cognition and Emotion, 21, 1630-1646. doi:10.1080/02699930701238495

48.

Rosenthal

(1991). Meta-analytic procedures for social research (2nd ed.). Newbury Park, CA: Sage.

49.

Rosseel

(2012). lavaan: An R package for structural equation modeling. Journal of Statistical Software, 48(2), 1-36.

50.

semTools Contributors. (2015). semTools: Useful tools for structural equation modeling. Retrieved from http://cran.r-project.org/package=semTools

51.

Somerville

L. H.

(2013). Special issue on the teenage brain: Sensitivity to social evaluation. Current Directions in Psychological Science, 22, 121-127. doi:10.1177/0963721413476512

52.

Spruit

Schalkwijk

van Vugt

Stams

G. J.

(2016). The relation between self-conscious emotions and delinquency: A meta-analysis. Aggression and Violent Behavior, 28, 12-20. doi:10.1016/j.avb.2016.03.009

53.

Stegge

Ferguson

T. J.

(1994). Self-conscious emotions: Maladaptive and adaptive scales (SCEMAS). Unpublished manuscript, Free University, Amsterdam, Netherlands.

54.

Stuewig

McCloskey

L. A.

(2005). The relation of child maltreatment to shame and guilt among adolescents: Psychological routes to depression and delinquency. Child Maltreat, 10, 324-336. doi:10.1177/1077559505279308

55.

Stuewig

Tangney

J. P.

Heigel

Harty

McCloskey

(2010). Shaming, blaming, and maiming: Functional links among the moral emotions, externalization of blame, and aggression. Journal of Research in Personality, 44, 91-102. doi:10.1016/j.jrp.2009.12.005

56.

Stuewig

Tangney

J. P.

Kendall

Folk

J. B.

Meyer

C. R.

Dearing

R. L.

(2015). Children’s proneness to shame and guilt predict risky and illegal behaviors in young adulthood. Child Psychiatry and Human Development, 46, 217-227. doi:10.1007/s10578-014-0467-1

57.

Tangney

J. P.

(1996). Conceptual and methodological issues in the assessment of shame and guilt. Behaviour Research and Therapy, 34, 741-754. doi:10.1016/0005-7967(96)00034-4

58.

Tangney

J. P.

Dearing

R. L.

(2002). Shame and guilt. New York, NY: Guilford Press.

59.

Tangney

J. P.

Stuewig

Martinez

A. G.

(2014). Two faces of shame: The roles of shame and guilt in predicting recidivism. Psychological Science, 25, 799-805. doi:10.1177/0956797613508790

60.

Tangney

J. P.

Stuewig

Mashek

Hastings

(2011). Assessing jail inmates’ proneness to shame and guilt: Feeling bad about the behavior or the self? Criminal Justice and Behavior, 38, 710-734. doi:10.1177/0093854811405762

61.

Tangney

J. P.

Stuewig

Mashek

D. J.

(2007). Moral emotions and moral behavior. Annual Review of Psychology, 58, 345-372. doi:10.1146/annurev.psych.56.091103.070145

62.

Tangney

J. P.

Wagner

Gramzow

(1992). Proneness to shame, proneness to guilt, and psychopathology. Journal of Abnormal Psychology, 101, 469-478. doi:10.1037//0021-843x.101.3.469

63.

Tangney

J. P.

Wagner

P. E.

Burggraf

S. A.

Gramzow

Fletcher

(1990). The Test of Self-Conscious Affect for Children (TOSCA-C). Fairfax, VA: George Mason University.

64.

Theunissen

S. C. P. M.

Rieffe

Kouwenberg

De Raeve

Soede

Briaire

J. J.

Frijns

J. H. M.

(2012). Anxiety in children with hearing aids or cochlear implants compared to normally hearing controls. Laryngoscope, 122, 654-659. doi:10.1002/lary.22502

65.

Theunissen

S. C. P. M.

Rieffe

Kouwenberg

De Raeve

L. J. I.

Soede

Briaire

J. J.

Frijns

J. H. M.

(2014). Behavioral problems in school-aged hearing-impaired children: The influence of sociodemographic, linguistic, and medical factors. European Child & Adolescent Psychiatry, 23, 187-196. doi:10.1007/s00787-013-0444-4

66.

Theunissen

S. C. P. M.

Rieffe

Netten

A. P.

Briaire

J. J.

Soede

Kouwenberg

Frijns

J. H. M.

(2014). Self-esteem in hearing-impaired children: The influence of communication, education, and audiological characteristics. PLoS ONE, 9(4), e94521. doi:10.1371/journal.pone.0094521

67.

Theunissen

S. C. P. M.

Rieffe

Netten

A. P.

Briaire

J. J.

Soede

Schoones

J. W.

Frijns

J. H. M.

(2014). Psychopathology and its risk and protective factors in hearing-impaired children and adolescents: A systematic review. JAMA Pediatrics, 168, 170-177. doi:10.1001/jamapediatrics.2013.3974

68.

Tracy

J. L.

Robins

R. W.

(2004). Putting the self into self-conscious emotions: A theoretical model. Psychological Inquiry, 15, 103-125. doi:10.1207/s15327965pli1502_01

69.

Tracy

J. L.

Robins

R. W.

(2006). Appraisal antecedents of shame and guilt: Support for a theoretical model. Personality & Social Psychology Bulletin, 32, 1339-1351. doi:10.1177/0146167206290212

70.

van Hoorn

van Dijk

Meuwese

Rieffe

Crone

E. A

. (2016). Peer influence on prosocial behavior in adolescence. Journal of Research on Adolescence, 26, 90-100.

71.

Vandenberg

R. J.

Lance

C. E.

(2000). A review and synthesis of the measurement invariance literature: Suggestions, practices, and recommendations for organizational research. Organizational Research Methods, 3, 4-70. doi:10.1177/109442810031002

72.

Wechsler

(1991). The Wechsler Intelligence Scale for Children (3rd ed.). San Antonio, TX: Psychological Corporation.

73.

Wolters

Knoors

H. E.

Cillessen

A. H.

Verhoeven

(2011). Predicting acceptance and popularity in early adolescence as a function of hearing status, gender, and educational setting. Research in Developmental Disabilities, 32, 2553-2565. doi:10.1016/j.ridd.2011.07.003

74.

Wray-Lake

Crouter

A. C.

McHale

S. M.

(2010). Developmental patterns in decision-making autonomy across middle childhood and adolescence: European American parents’ perspectives. Child Development, 81, 636-651.