A Validation Study of the Psychometric Properties of the Other As Shamer Scale

Abstract

External shame is associated to social and emotional difficulties, which are predictors of psychological disorders. The aim of the present study was to investigate the psychometric properties of the shorter Italian version of the Other As Shamer Scale (OAS-2), a measure of external shame. Confirmatory factor analysis was used in a sample of 612 (54.2% male) Italian undergraduate students. Evidence for this model provided support for the assumption that the one-factor model was similar across the gender groups. The Italian OAS-2 fully replicated the one-factor structure of the original OAS. Cronbach’s alpha index, as well as test–retest stability provided satisfactory results. The OAS-2 showed good convergent and divergent validity, being highly correlated with measures of internal shame, trait anxiety, and depression. The OAS-2 can be considered a useful instrument for measuring external shame when time constraints impede the use of the 18-item OAS. Implications of the results are discussed and recommendations for future work with the scale are made.

Keywords

shame OAS depression anxiety emotion

Introduction

Construct of External Shame

Shame is a “self-conscious” emotion that has a significant impact on an individual’s sense of self and well-being (Kim, Thibodeau, & Jorgensen, 2011; Tangney & Dearing, 2003; Tracy, Robins, & Tangney, 2007). According to the Evolutionary and Biopsychosocial Model of shame (Gilbert, 1998, 2003, 2007, 2010), shame appears to have evolved as a system that regulates psychobiological responses to social rank. This emotion corresponds to involuntary perceived subordination, which carries risks of leading to social exclusion, put-down, rejection, or even persecution. Thus, it is considered a marker or alerter of potential social damage.

Consistent with this conceptualization, there are two apparently distinct elements of the shame experience, namely “internal shame” and “external shame” (Gilbert, 1998; Gilbert & Procter, 2006; Smith, Webster, Parrott, & Eyre, 2002). In internal shame, the focus is on the “bad self,” involving a negative view of self as seen through the individual’s own eyes, whereas in external shame, the focus is on the social world, on how we experience ourselves in the minds of others, and involves the belief that others view the self negatively. Moreover, increasing evidence shows that this distinction between internal and external shame is valid and important and relates to psychopathology in slightly different ways (Kim et al., 2011).

Processes such as harsh self-criticism (Gilbert, 2007) and perfectionism (Ashby, Rice, & Martin, 2006; Blatt, 1995; McCranie & Bass, 1984; Sandquist, Grenyer, & Caputi, 2009) maintain internal shame. Self-criticism is a form of hostility, disgust, and disregard aimed at oneself that blocks developing a sense of protection and safety toward oneself (Gilbert, Baldwin, Irons, Baccus, & Palmer, 2006; Whelton & Greenberg, 2005). Differently, external shame is often associated with a complex psychology of social attractiveness and monitors how we exist for others and their judgments about us (Gilbert, 1998, 2003, 2007). We tend to became vulnerable to external shame, embracing the perception that others hold negative thoughts and beliefs regarding the self to the extent that we felt neglected, devalued, and/or abused.

A growing body of recent scientific studies underlines the urgent need to focus on external shame, as it appears to be more strongly associated with social difficulties such as ostracism and social criticism which adversely affect the representation of the self (Matos, Pinto-Gouveia, Gilbert, Duarte, & Figueiredo, 2015). The perception of being seen negatively by others is likely to lead the individual to receive potential invalidating social signals that inhibit the activation of positive affect linked to the sense of caring and of “being loved,” and ultimately to a better physical and mental well-being (Cacioppo & Patrick, 2008; Cozolino, 2014). In this manner, individuals get trapped in a vicious circle which reinforces external shame and gives rise to the use of maladaptive strategies for managing shame (Gilbert, 2007).

Recently, many studies have shown that social isolation and repeated experiences of social refusal are factors related to psychological and physical disturbances such as obesity and diabetes type 2, through their activation of the anterior cingulate cortex and the amygdala (Cacioppo & Hawkley, 2009; Cacioppo & Patrick, 2008; Dunbar & Shultz, 2007; Eisenberger & Cole, 2012; Innamorati et al., 2014; Miller, Chen, & Cole, 2009). Intervening on external shame, then, becomes important because it allows the individual to regulate the management of interpersonal relationships better by positively influencing the mental and physical health through neuro-physiological mediators such as the sympathetic nervous system and the hypothalamic-pituitary-adrenal circuit (Eisenberger & Cole, 2012; Miller et al., 2009).

There is also evidence that suggests gender differences with regard to shame, with women reporting higher levels of shame proneness (Benetti-McQuoid & Bursik, 2005; Galhardo, Cunha, Pinto-Gouveia, & Matos, 2013; Vagos, da Silva, Brazão, Rijo, & Gilbert, 2016). Also, as suggested by Mills, Arbeau, Lall, and De Jaeger (2010), girls showed more shame than boys between preschool age and school age. This trend is also evident during adolescence (Roos, Hodges, & Salmivalli, 2014). Contradictorily, adolescent boys reported higher score on Emptiness factor of the Other As Shamer Scale (OAS) than girls (Vagos et al., 2016). Finally, a recent large meta-analysis reported no gender differences in shame experiences (Else-Quest, Higgins, Allison, & Morton, 2012), suggesting that blanket stereotypes about females’ higher emotionality are probably erroneous.

Measurement of External Shame

The scale most currently used for measuring external shame was Other As Shamer Scale (OAS; Allan, Gilbert, and Goss 1994; Goss, Gilbert, and Allan 1994), developed from Cook (1996) Internal Shame Scale (ISS).Thus, the focus of the OAS (Allan et al., 1994; Goss et al., 1994) is on beliefs about how “others see/judge me.”

In the original study of the psychometric properties of the scale on a sample of 156 undergraduate students, exploratory factor analysis (EFA) of the 18-item OAS was made up of three dimensions: (a) inferiority, composed of items related to being seen as inferior; (b) emptiness, consisting in items related to being seen as empty; and (c) mistake, consisting of items relating to how others behave when they see their own mistakes (Goss et al., 1994). The inferiority factor, which was the core experience of self-reported external shame, accounted for the largest proportion of the total variance, whereas the emptiness factor was more strongly associated with measures of psychopathology, including depression (Goss et al., 1994). Using confirmatory factor analysis (CFA), these three dimensions were confirmed as related first-order factors with one higher order factor within a hierarchical model, the best-fitting model in a sample of 687 nonclinical Italian subjects (Balsamo et al., 2013). This model would support the use of both total score as an overall index of external shame, and three subfactor scores for measuring different related components of external shame. The use of the OAS’s three subfactor scores could aid in the assessment of external shame and what individuals are actually concerned with when being judged. Furthermore, evidence of a strong cross-sectional association between shame and depressive symptoms was supported by different depression self-report questionnaires (Allan et al., 1994; Balsamo, Macchia, et al., 2015; Goss et al., 1994; Kim et al., 2011; Matos et al., 2015).

Recently, a shorter form of the OAS, Other As Shamer Scale–2 (OAS-2; Matos et al., 2015), was developed. Eight items were selected from the long 18-item OAS using a two-step methodology: (a) items with high intercorrelations to maximize the scale structure, and (b) items of the construct of external shame based on the expert’s ratings, to select the most representative items. Indeed, a group of 14 experts evaluated the eight items that best represented the construct of external shame as theoretically defined (Gilbert, 1998, 2007). This set of items became the basis for testing a model in Portuguese undergraduate students (n = 312) and the community sample (n = 378) by using CFA. Results confirmed the previous proposed OAS one-factor structure (Matos et al., 2015; Matos, Pinto-Gouveia, & Costa, 2013). Furthermore, the OAS-2 revealed good internal consistency (.82), similar to the longer version, as well as good concurrent and divergent validity, being highly correlated with the OAS (r = .91). The OAS and OAS-2 resulted to have similar significant correlations with measures of internal shame, psychopathology, and anger, with no significant difference between them. These results seem to show that the OAS-2 is an economic, valid, and reliable measure of external shame, and can be used in place of the longer form of the questionnaire. However, to date, no further studies have investigated its psychometric properties in other adult samples.

Aims of the current study

The current study investigated the construct validity of the Italian version of OAS-2 through confirmatory methodology in a large sample of Italian undergraduates, as well as other psychometric properties (reliability, and convergent and discriminant validity). Moreover, as there are valid theoretical and empirical studies which showed mixed, controversial findings on gender differences regarding shame, measurement invariance across gender was also tested in our sample of undergraduate students, along with between-gender comparisons. Our aim was to analyze the clinical significance of the resulting OAS-2 factors, by investigating the relationships with several constructs that tend to relate to external shame. In particular, in line with results from a previous study using the Italian OAS long form in Italian sample (Balsamo, Macchia, et al., 2015) and another one using OAS-2 in a Portuguese sample (Matos et al., 2015), we expected positive and significant relationships with depression, internal shame, and trait anxiety measures. In addition to the findings from these previous studies, the associations of external shame with both cognitive and somatic facets of anxiety were also explored. Given that the pain of external shame is rooted in the recognition that others view the self negatively; thus, it is more likely to be linked to the cognitive dimension of anxiety than the somatic dimension. Thus, we expected that cognitive dimension of trait anxiety resulted to be highly correlated to OAS-2 somatic dimension.

Method

Participants

Six hundred twelve Italian students (n = 280 female, 45.8%; n = 332 male, 54.2%) participated in the study, including high school students (12.6%), undergraduates (79.7%), and graduate students (7.7%). Their mean age was 21.27 years (SD = 2.44, range = 18-28). Participants’ mean years of education was 12.62 (SD = 1.99).

Procedure

The respondents completed a set of structured questionnaires on a voluntary and anonymous basis. Questionnaires were presented in a randomized order across participants. No honorarium or compensation was given for completing the assessments. All participants provided written informed consent. Parental consent was requested for students under 18 years of age; the consent forms were signed by one of the two parents and brought back by the student. The study was approved by the school board of each school and university ethical committee involved in the study.

Materials

Other As Shamer Scale-2 (OAS-2)

The OAS-2 (Matos et al., 2015) is a short self-report measure of external shame, which includes eight items drawn from the Italian version (Balsamo, Macchia, et al., 2015) of the original 18-item OAS (Goss et al., 1994). This shorter measure of shame, which reflects the original one, was developed by Matos et al. (2015) because of its advantage of being easier and quicker to administer, especially in clinical settings. The Italian OAS-2 was developed by drawing the same eight items indicated by Matos et al. (2015) from the original Italian OAS. Respondents are asked to rate the frequency with which they make certain evaluations on a 5-point Likert-type scale, ranging from 0 (never) to 4 (almost always). The total score, calculated by summing up item scores, ranges from 0 to 32, with higher scores indicating greater external shame. In their study, Matos et al. (2015) found this scale to have a Cronbach’s alpha of .82, and similar associations with psychopathology variables to those of the original version. In this study, the Cronbach’s alpha of the OAS-2 Italian version was .89.

The Experience of Shame Scale (ESS)

The ESS (Andrews, Qian, & Valentine, 2002) is a 25-item self-report measure of internal shame. The measure investigated character (e.g., shame of personal habits, manner with others), behavioral (e.g., shame about doing something wrong, saying something stupid), and bodily areas of shame (e.g., feeling ashamed of body or any part of it) over the past month. Examples of items are, respectively, “Have you felt ashamed of the sort of person you are?” (Character); “Have you tried to cover up or conceal any of your personal habits?” (Behavioral), and “Have you felt ashamed of your body or any part of it?” (Bodily). Participants rate each item on a scale ranging from 1 (not at all) to 4 (very much), with higher scores indicating greater shame. Coefficient alpha for this study was .95.

Teate Depression Inventory (TDI)

The TDI (Balsamo & Saggino, 2013, 2014) is a 21-item self-report instrument designed to assess Major Depressive Disorder as specified by the latest editions of the Diagnostic and Statistical Manual of Mental Disorders (American Psychiatric Association, 2000, 2013). It was developed via Rasch logistic analysis of responses (Balsamo, Giampaglia, & Saggino, 2014), within the framework of Item Response Theory (Andrich, 1995; Rasch, 1960), to overcome inherent psychometric weaknesses of existing measures of depression, including the Beck Depression Inventory-II (Balsamo & Saggino, 2007). Each item is rated on a 5-point Likert-type scale, ranging from 0 (always) to 4 (never). Growing literature suggests that the TDI has strong psychometric properties in both clinical and nonclinical samples, including an excellent Person Separation Index, no evidence of bias due to item-trait interaction, good discriminant and convergent validity, and control of major response sets (Balsamo, Carlucci, Sergi, Murdock, & Saggino, 2015; Balsamo, Innamorati, Van Dam, Carlucci, & Saggino, 2015; Balsamo et al., 2013; Innamorati et al., 2013). In the present sample, Cronbach’s alpha was .91.

State–Trait Inventory for Cognitive and Somatic Anxiety (STICSA)

The STICSA (Ree, French, MacLeod, & Locke, 2008; for the Italian version, see Balsamo et al., 2016; Balsamo, Innamorati, et al., 2015) is a 21-item measure designed to assess cognitive (e.g., “I feel agonized over my problems,” “I think that others won’t approve of me”) and somatic (e.g., “My heart beats fast,” “My muscles are tense”) symptoms, both on Trait and State variations. In the Trait Anxiety subscale, the individual rates how often a statement is true in general, whereas in the State Anxiety subscale, she or he rates how she or he feels at the moment of assessment. In total, the overall scale is made up of four subscales: State–Somatic (SS), Trait–Somatic (TS), State–Cognitive (SC), and Trait–Cognitive (TC). Each statement is rated on a 4-point Likert-type scale from 1 (not at all) to 4 (very much so). In this study, we administered the Trait subscale. In this study, coefficient alphas were .88 for total STICSA–Trait score, and .78 to .87 for TS and TC, respectively.

Statistical Procedure

To test whether the factor structure of the Italian OAS-2 replicated the one-factor model proposed by the previous study in Portuguese adult (Matos et al., 2015) and adolescent sample (Vagos et al., 2016), we conducted a CFA using the statistical package MPLUS v7 (Muthén & Muthén, 2012). To test the multivariate normality of the data collected in our sample, the Mardia’s test of multivariate skewness and kurtosis test was used (DeCarlo, 1997; Von Eye & Bogat, 2004). If this test shows a significant deviation from normality, it is recommended to use the Maximum Likelihood Robust (MLR) as appropriate estimator.

Based on the previous studies which assess the one-factor measurement model underlying the OAS-2, also due to the small number of OAS-2 items, we tested for unidimensional latent structure in our sample. Model fit was assessed by means of the following goodness-of-fit indexes: (a) the Satorra–Bentler chi-square (SBχ²) statistic and its degrees of freedom, (b) the root mean square error of approximation (RMSEA) and its 90% confidence interval (90% CI), (c) the comparative fit index (CFI), and (d) the standardized root mean square residuals (SRMR). CFI values of .90 and above were considered to reflect adequate fit, while values of .95 and above were considered to indicate excellent fit. RMSEA and SRMR values of .08 or less were considered to reflect an adequate fit, while values of .05 or less were considered to reflect good fit (Schermelleh-Engel, Moosbrugger, & Müller, 2003).

As various valid theoretical and empirical studies present different statements concerning gender differences in the experience of shame, we conducted a multiple-group CFA to assess measurement invariance of the OAS-2 one-factor model across gender groups. Configural invariance is established if the OAS-2 items exhibit the same configuration of salient (nonzero) and nonsalient (zero or near zero) factor loadings across male and female groups. Subsequently, following the “step-up” approach to invariance evaluation (Brown, 2015), we also tested for metric and scalar invariance. In the metric invariance, only the factor loadings are equal across groups but the intercepts are allowing to differ between groups. This is useful to test whether respondents across groups attribute the same meaning to the latent construct under study. Differently, in the scalar invariance, all the loadings and intercepts are constrained to be equal and implies that the meaning of the construct, and the levels of the underlying items are equal across males and females (Van de Schoot, Lugtig, & Hox, 2012). To proceed to the evaluation of groups’ equivalence, we further tested for partial scalar invariance (Brown, 2015). The difference between the invariance models was assessed computing the MLR chi-square difference test (Muthén, & Muthén, 2005).

In addition, internal consistency was examined using the Cronbach’s alpha coefficient and the item-total correlations. A t test for independent sample was computed to assess gender effects on the OAS-2 total score. To assess the convergent validity of the OAS-2, correlations with measures of internal shame, and psychopathological measures (depression and trait anxiety) were examined through Pearson’s correlation coefficients.

Results

Results from the Mardia’s multivariate normality test showed that the data collected in our sample had a low probability of having been drawn from a multinormal population (Mardia’s = 121.428, p < .001; DeCarlo, 1997). Therefore, we considered the MLR as an appropriate estimator. The one-factor model tested showed low and unsatisfying model fit according to all goodness-of-fit indexes, SBχ² (df) = 134.04 (20), p < .001; RMSEA = .10; 90% CI RMSEA [.081, .112]; CFI= .93, SRMR = .04.

Given the CFA modification indices (MI) suggested a further refining of the measurement model (MI > 30), this model was achieved after allowing covariance between error terms of the Items 1 and 2, and the Items 5 to 7 to be freely estimated. Generally, MI are allowed to improve models fit when the modifications are few, well represented on the same latent construct (Brown, 2015), as well as theoretically and practically plausible (e.g., MacCallum, 1995). Based on detailed examination of the item’s content, Items 1 (“I feel other people see me as not good enough”) and 2 (“I think that other people look down on me”) and Items 5 (“Other people see me as small and insignificant”) and 7 (“Other people see me as somehow defective as a person”) showed similar phrasing and meaning, proving to be theoretically related and also easily misunderstanding.

Figure 1 shows loading/path coefficients of the one-factor model in which all items were forced to load on a single latent factor. All items loaded considerably (>.45) on the one factor. All goodness-of-fit statistics fit indexes for the refined one-factor model had values indicating good model fit: RMSEA = .06; 90% CI RMSEA = [.050, .084]; CFI = .97; SRMR = .028. However, the χ² goodness-of-fit test, SBχ² (df) = 67.02 (18), was significant at p < .001.

Figure 1.

Path coefficients of the OAS-2 one-factor model (N = 612).

The standardized regression weight (ranged from .52 to .81), and the square multiple correlations (R² ranged from .27 to .65) were medium and all statistically significant at p < .001, suggesting an adequate factorial structure of the OAS-2 (Table 1). Cronbach’s α, that is internal consistency reliability of the OAS-2 total score, was high (.89), and item-total correlations ranged from .30 to .59 (item-total mean equal to .51). Regarding gender differences, an independent t test showed no significant differences between males and females (OAS-2 t₆₁₂ = .619, p = .536).

Table 1.

Standardized Regression Weight, R² for the OAS-2 Items (N = 612).

Item	Standardized regression weight	R ²
OAS-2-1	.552	.272
OAS-2-2	.747	.559
OAS-2-3	.713	.508
OAS-2-4	.787	.614
OAS-2-5	.808	.653
OAS-2-6	.757	.573
OAS-2-7	.751	.564
OAS-2-8	.748	.559

Note. OAS-2 = Other As Shamer Scale-2; R² = squared multiple correlation.

p < .001.

Furthermore, configural invariance of the OAS-2 was evaluated by estimating a model where all parameters (fixed and nonfixed) were freely estimated across the male and female groups of our sample. A simultaneous test of equal form was conducted to test the presence of an identical factor structure across gender groups. All fit indices indicated a good model fit, SBχ² (df) = 103.99 (36); RMSEA = .07; 90% CI RMSEA = [.061, .096]; CFI = .97, and the fixed and free parameters were equivalent across gender. Concerning the measurement invariance, full metric invariance for gender was found in our sample (Δχ² = 7.53, df = 7, p > .37). With the support of metric invariance model, scalar invariance was tested. A χ² difference test was performed comparing the scalar invariance model and the metric invariance model. Because the χ² difference was statistically significant at a p = .001 (Δχ² = 23.65, with 2 df), scalar invariance was not supported. Results showed that only partial scalar invariance was achieved after allowing the intercepts of Items 1, 3, and 4 to be freely estimated across male and female groups (Δχ² = 8.05, df = 5, p > .15).

Table 2 presents the descriptive statistics of all used measures in this study and the correlations between the OAS-2 and measures of internal shame (ESS), depression (TDI), and anxiety trait (STICSA–Trait). Concerning the concurrent validity, the OAS-2 presented strong and positive correlations with the internal shame, as measured by the ESS. With regard to the associations with depression and anxiety, the OAS-2 was moderately and positively correlated with the TDI, similar to what was found with the Italian 18-item OAS (Balsamo, Macchia, et al., 2015). Similar associations were found with trait anxiety, both in the cognitive, and to a lesser extent, somatic dimensions.

Table 2.

Descriptive Statistics and Zero-Order Correlations Between OAS-2, ESS, TDI, and STICSA (N = 612).

	M	SD	OAS-2
OAS-2	8.96	6.43	—
ESS^a	23.55	14.46	.60**
TDI	30.75	13.27	.46**
STICSA–Trait	38.72	11.09	.49**
STICSA–Trait, Somatic	19.09	6.32	.36**
STICSA–Trait, Cognitive	19.63	6.11	.51**

Note. OAS-2 = Other As Shamer Scale-2; ESS = Experience of Shame Scale; TDI = Teate Depression Inventory; STICSA = State–Trait Inventory for Cognitive and Somatic Anxiety.

n = 351.

p < .001.

Discussion

Growing evidence suggests that external shame should be distinguished from internal shame both conceptually and empirically (Kim et al., 2011). Indeed, the two different forms of shame are related to psychopathology in slightly different ways. In particular, external shame, defined as self–other perception in which the self is negatively evaluated by others, is significantly associated with a wide range of mental health problems, particularly depressive symptoms (Allan et al., 1994; Cheung, Gilbert, & Irons, 2004; Cook, 1996; Gilbert, Cheung, Irons, & McEwan, 2005; Kim et al., 2011).

Based on Gilbert’s (1992, 1997) view of shame as an emotion corresponding to perceived low social rank, the OAS is currently a widely used self-report questionnaire measuring external shame. Recently, a shorter version of this scale, called OAS-2, was proposed by (Matos et al., 2015). The aim of the present study was to investigate dimensionality and psychometric properties of the OAS-2 in a large heterogeneous sample of Italian students, to replicate the model presented in the recent literature.

CFA results replicated the previously proposed OAS one-factor structure (Matos et al., 2015) in our sample. Indeed, the χ² goodness-of-fit test was significant, as expected, due to its sensitivity to the large sample size and multivariate normal distribution violations (Hu & Bentler, 1998; Kahn, 2006; Kline, 2011; Maruyama, 1997; Tanaka, 1993), while all goodness-of-fit indexes suggested the good fit of the one-factor model for the data.

As argued by several authors (Diamantopoulos, Siguaw, & Siguaw, 2000; Fan, Thompson, & Wang, 1999; Hu & Bentler, 1998; Rigdon, 1996; Schermelleh-Engel et al., 2003), the fit indexes RMSEA and CFI do not depend on sample size, as strongly as χ². In addition, they are sensitive to structural model misspecifications and to the number of estimated parameters in the model. Thus, these indexes should strongly be considered.

Specifically, in this study, RMSEA and the CFI noncentrality-based indexes, which represent an “error of approximation,” indicated that the one-factor model fit reasonably well in the population; the SRMR index suggested that fit of the model was good at an absolute level (SRMR ≤ .05) and in relation to a null solution; CFI > .95 and RMSEA < .08 indicated a good parsimony for the tested model.

Similar to that reported in Matos et al. (2015), all standardized indicators had statistical support and confirmed this model as a plausible solution to explain the factor structure of the OAS-2.

Furthermore, both configural and metric invariance analyses across gender suggested that the proposed one-factor structure and pattern of factor loadings of the OAS-2 were similar for the groups of men and women in this study.

Nevertheless, only partial scalar invariance was achieved suggesting that male–female differences in the means of observed items tapping external shame did not appear to stem from differences in the means of the latent construct of external shame.

In other words, there is a consistency between gender differences in latent means and gender differences in observed means. Shame experiences may perform differently on some items in external shame self-report measure, probably in part due to the presence of item bias (Dimitrov, 2010). This finding is in line with Vagos et al. (2016), showing some differences on shame experiences may be located at item level. Summing up, the conceptualization of the external shame construct, as assessed with the OAS-2, seems to be partially similar for both gender groups.

This finding is consistent with a large part of the current literature. Specifically, no significant gender differences in either the long or short versions of the OAS were found in an adult Portuguese sample through independent-samples t tests (Matos et al., 2015; Vagos et al., 2016), as well as measurement models of the OAS–Adolescent version (OAS-A; Figueira, 2010) were partially invariant across boys and girls, who presented similar levels of shame, in three Portuguese adolescent samples presenting diverse degrees of behavioral problems’ severity (Vagos et al., 2016).

Regarding reliability, the item-total correlations further confirmed the adequacy of these items. These values were slightly higher than in the Italian 18-item OAS (Balsamo, Macchia, et al., 2015). Also, Cronbach’s α of the OAS-2 (.89) total score was close to that of the longer version (.87) found in another Italian sample of 687 nonclinical adults (Balsamo, Macchia, et al., 2015), and higher than that reported in a Portuguese sample of 690 nonclinical adults, both for the short (.82) and long version (.89) (Matos et al., 2015; Matos et al., 2013). This added extra support for the reliability of the OAS-2 as a shorter measure of external shame.

The concurrent and divergent validity analyses corroborated our hypotheses with OAS-2 significantly correlating with internal shame and depression measures, as well as distinct dimensions of cognitive and somatic trait anxiety. Such associations are similar in magnitude and directions to those found in the original OAS, as well as in the Italian version (Allan et al., 1994; Balsamo, Macchia, et al., 2015), suggesting that individuals with higher levels of external shame, as measured by the OAS-2, also experience feelings of internal shame, depression, and anxiety symptoms.

As expected, higher positive correlation of OAS-2 with cognitive dimension of trait anxiety compared with somatic dimension were reported in our sample. As external shame is focused on monitoring how we exist for others and their judgments of us (Gilbert, 1998, 2003, 2007), the cognitive dimension of anxiety presented higher correlation with the OAS-2 than the somatic dimension. Indeed, while the somatic dimension includes self-reported symptoms such as hyperventilation, sweating, trembling, and palpitations, cognitive dimension reflects symptoms that are associated more directly with thought processes, including worry, intrusive thoughts, and lack of concentration (Ree et al., 2008).

With regard to the correlation between internal and external shame, we found the correlation between OAS-2 and ESS (.62, p < .001) very similar to that (.54, p < .001) obtained in the Portuguese sample by Matos et al. (2015). Nonetheless, Kline (2000) cautioned against high correlations between scales measuring similar but not the same constructs because these could be explained by the redundancy between scales. This might alter the content validity of the scale and limit the usefulness of a short version of the OAS in the distinctiveness of the internal and external shame. Therefore, additional evidence of reliability and validity would be necessary.

These findings support convergent and divergent validity concerning important psychopathological symptoms and emotional difficulties, in line with existing literature reporting a strong link between external shame and clinically significant indicators (Cheung et al., 2004; Cunha, Matos, Faria, & Zagalo, 2012; Cunha, Xavier, & Vitória, 2014; Gilbert & Miles, 2000; Gratz, Rosenthal, Tull, Lejuez, & Gunderson, 2010; Kim et al., 2011; Luoma, Kohlenberg, Hayes, & Fletcher, 2012; Matos & Pinto-Gouveia, 2010; Pinto-Gouveia, Matos, Castilho, & Xavier, 2014; Rüsch et al., 2007; Wei, Shaffer, Young, & Zakalik, 2005).

Comparing results found in Matos et al. (2015) in a Portuguese sample of adults, the results of the present study confirm the adequacy of a shorter version of the OAS, which proved to be an economic questionnaire to measure external shame. However, it should be noted that the OAS was shown to have different factors within it that allow us to assess different forms of external shame.

In contrast to the findings for the long version of the Italian OAS (Balsamo, Macchia, et al., 2015), the short version of the OAS may not illuminate different forms of external shame. While the hierarchical model of the OAS, tested by Balsamo, Macchia, et al. (2015) in a large nonclinical sample, supported the use of both total score, as an overall index of external shame, and three subfactor scores, for measuring different components of external shame, that is, Inferiority, Emptiness, and Mistake, the OAS-2 provides an unique score, due to its unidimensional nature. This implies a loss of potential information about the interpretation of possible different dimensions of external shame.

Indeed, although methodological advances in psychometric analyses indicate the importance of considering both brevity and reliability when using self-report measures (Archer, Tirrell, & Elkins, 2001; DeVellis, 2012; Scholte & De Bruyn, 2001), a broader part of literature has been critical of short forms for a number of reasons (Levy, 1968; Smith & McCarthy, 1995; Smith, McCarthy, & Anderson, 2000). Short forms may be inappropriate in clinical assessment procedures when the classification and treatment of individuals is at stake (Francis & Jackson, 2004). With respect to survey style studies, however, the case against short forms is much less substantial. What is required of short forms used in this way is clear evidence of their concurrent validity alongside the parent full-form of the scale and good reliability.

Thus, the decision to use the long or the short form of the OAS for assessing external shame should be based on the unique characteristics of the sample and the purpose for using the instrument. For research studies, the OAS-2 may be considered a useful and economic instrument for screening external shame when time constraints impede the use of the long version. In clinical settings, the use of the 18-item OAS is preferable, to investigate the construct as a whole and to distinguish different types of social fears.

An important limitation of the present study concerns the potential lack of generalizability of these results to clinical conditions. The recourse to a sample of undergraduate students raises an important issue of external validity and limits the usefulness of this study. More specifically, the confirmatory models and the correlational analyses among variables found in nonclinical samples might not be similar to the processes in clinical samples. Future studies should seek to replicate this investigation using diverse general population samples to enable stronger conclusions to be drawn, and testing the presence of item bias (through the differential item functioning analysis) in gender-invariant structure of the OAS-2. Moreover, future research should investigate the convergent validity of the OAS-2 through its correlation with the full-form version (OAS), to test if it could serve as good predictor of scores obtained by the OAS. Last, investigating the invariance of the OAS-2 structure among samples from different countries would be interesting.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

Author Biographies

Aristide Saggino, PhD, is a full professor of Psychometrics at the School of Medicine and Health Sciences of the University of di Chieti-Pescara, Italy. He has an extensive background in the areas of assessment, individual differences and cognitive behavioral psychotherapy. He has authored several books, book chapters and scientific papers in Italian and English journals.

Leonardo Carlucci, PhD, is a psychologist and a PostDoc researcher. He conducted his doctoral research on cognitive and psychometric aspects of religious fundamentalism at the University of di Chieti-Pescara, Italy. He has an extensive background in psychology of religion and fundamentalism, psychometric testing and personality assessment. He has authored several papers both in Italian and in English.

Maria Rita Sergi, PhD, conducted her doctoral research on psychometric and biological aspects of emotional intelligence at the Department of Neuroscience and Imaging, University of di Chieti-Pescara, Italy. She has skills and expertise in the areas of personality and clinical assessment. She has authored few papers in Italian and English journals.

Ines D’Ambrosio is graduated in clinical psychology and health, with a thesis in psychometrics at the University of Chieti-Pescara, Italy. She is a psychologist, BCaBA (Board Certified assistant Behavior Analyst) and is working with autistic children.

Beth Fairfield, PhD, is a cognitive psychologist and researcher at the University of Chieti-Pescara, Italy. She conducts research in the fields of memory and aging and has carried out extensive research regarding emotion-cognition interaction. She has authored over 30 peer-reviewed scientific papers in English in the areas of memory, aging, emotion and source monitoring.

Nicoletta Cera is a cognitive psychologist, PhD in Functional Neuroimaging. After having performed several fMRI studies on the aging related diseases and sexual behaviour, she is actually involved in the research Activities of the Human sexuality research Group at the university of Porto, Portugal. Her interests are mainly about the cerebral correlates of the male sexual behaviour and related pathologies.

Michela Balsamo, PhD, is an associate professor of Psychometrics at the School of Medicine and Health Sciences of the University of Chieti-Pescara, Italy. She has an extensive background in the area of assessment of depression and anxiety. She has authored over 30 peer-reviewed scientific papers both in English and in Italian in the areas of psychometrics and cognitive behavioral psychotherapy.

References

Allan

Gilbert

Goss

(1994). An exploration of shame measures—II: Psychopathology. Personality and Individual Differences, 17, 719-722.

American Psychiatric Association. (2000). Diagnostic and statistical manual of mental disorders (4th ed., text rev.). Washington, DC: Author.

American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders (5th ed.). Arlington, VA: American Psychiatric Publishing.

Andrews

Qian

Valentine

J. D.

(2002). Predicting depressive symptoms with a new measure of shame: The Experience of Shame Scale. British Journal of Clinical Psychology, 41, 29-42.

Andrich

(1995). Models for measurement, precision, and the nondichotomization of graded responses. Psychometrika, 60, 7-26.

Archer

R. P.

Tirrell

C. A.

Elkins

D. E.

(2001). Evaluation of an MMPI-A short form: Implications for adaptive testing. Journal of Personality Assessment, 76, 76-89.

Ashby

J. S.

Rice

K. G.

Martin

J. L.

(2006). Perfectionism, shame, and depressive symptoms. Journal of Counseling & Development, 84, 148-156.

Balsamo

Carlucci

Sergi

M.R.

Romanelli

D’Ambrosio

Fairfield

. . .Saggino

(2016). Una nuova misura dell’ansia di tratto e di stato: Lo State Trait Inventory of Cognitive and Somatic Anxiety (STICSA). Standardizzazione in un campione italiano [A new measure for trait and state anxiety: The State Trait Inventory of Cognitive and Somatic Anxiety (STICSA). Standardization in an Italian population]. Psicoterapia Cognitiva e Comportamentale, 22, 229-232.

Balsamo

Carlucci

Sergi

M. R.

Murdock

K. K.

Saggino

(2015). The mediating role of early maladaptive schemas in the relation between co-rumination and depression in young adults. PLoS ONE, 10(10), e0140177.

10.

Balsamo

Giampaglia

Saggino

(2014). Building a new Rasch-based self-report inventory of depression. Neuropsychiatric Disease and Treatment, 10, 153-165.

11.

Balsamo

Innamorati

Van Dam

N. T.

Carlucci

Saggino

(2015). Measuring anxiety in the elderly: Psychometric properties of the State Trait Inventory of Cognitive and Somatic Anxiety (STICSA) in an elderly Italian sample. International Psychogeriatrics, 27, 999-1008.

12.

Balsamo

Macchia

Carlucci

Picconi

Tommasi

Gilbert

Saggino

(2015). Measurement of external shame: An inside view. Journal of Personality Assessment, 97, 81-89.

13.

Balsamo

Romanelli

Innamorati

Ciccarese

Carlucci

Saggino

(2013). The State-Trait Anxiety Inventory: Shadows and lights on its construct validity. Journal of Psychopathology and Behavioral Assessment, 35, 475-486.

14.

Balsamo

Saggino

(2007). Test per l’assessment della depressione nel contesto italiano: un’analisi critica [Tests for depression assessment in Italian context: A critical review]. Psicoterapia Cognitiva e Comportamentale, 13, 167-199.

15.

Balsamo

Saggino

(2013). Il Teate Depression Inventory-Manuale di standardizzazione [Teate Depression Inventory, Manual]. Firenze, Italy: Hoegrefe.

16.

Balsamo

Saggino

(2014). Determining a diagnostic cut-off on the Teate Depression Inventory. Neuropsychiatric Disease and Treatment, 10, 987-995.

17.

Benetti-McQuoid

Bursik

(2005). Individual differences in experiences of and responses to guilt and shame: Examining the lenses of gender and gender role. Sex Roles, 53, 133-142.

18.

Blatt

S. J.

(1995). The destructiveness of perfectionism: Implications for the treatment of depression. American Psychologist, 50, 1003-1020.

19.

Brown

T. A.

(2015). Confirmatory factor analysis for applied research. New York, NY: Guilford Press.

20.

Cacioppo

J. T.

Hawkley

L. C.

(2009). Perceived social isolation and cognition. Trends in Cognitive Sciences, 13, 447-454.

21.

Cacioppo

J. T.

Patrick

(2008). Loneliness: Human nature and the need for social connection. New York, NY: W.W. Norton.

22.

Cheung

M.-P.

Gilbert

Irons

(2004). An exploration of shame, social rank and rumination in relation to depression. Personality and Individual Differences, 36, 1143-1153.

23.

Cook

D. R.

(1996). Empirical studies of shame and guilt: The Internalized Shame Scale. In Nathanson

(Ed.), Knowing feeling: Affect, script, and psychotherapy (pp. 132-165). New York, NY: W.W. Norton.

24.

Cozolino

(2014). The neuroscience of human relationships: Attachment and the developing social brain. New York, NY: W.W. Norton.

25.

Cunha

Matos

Faria

Zagalo

(2012). Shame memories and psychopathology in adolescence: The mediator effect of shame. International Journal of Psychology and Psychological Therapy, 12, 203-218.

26.

Cunha

Xavier

Vitória

(2014). Avaliação da auto-compaixão em adolescentes: adaptação e qualidades psicométricas da escala de auto-compaixão [Assessment of self-compassion in adolescents: Adaptation and psychometric properties of the SelfCompassion Scale]. Revista de Psicologia da Criança e do Adolescente, 4(2), 95-117.

27.

DeCarlo

L. T.

(1997). On the meaning and use of kurtosis. Psychological Methods, 2, 292-307.

28.

DeVellis

R. F.

(2012). Scale development: Theory and applications. Thousand Oaks, CA: Sage.

29.

Diamantopoulos

Siguaw

J. A.

Siguaw

J. A.

(2000). Introducing LISREL: A guide for the uninitiated. London, England: Sage.

30.

Dimitrov

D. M.

(2010). Testing for factorial invariance in the context of construct validation. Measurement and Evaluation in Counseling and Development, 43, 121-149.

31.

Dunbar

R. I.

Shultz

(2007). Understanding primate brain evolution. Philosophical Transactions of the Royal Society of London B: Biological Sciences, 362, 649-658.

32.

Eisenberger

N. I.

Cole

S. W.

(2012). Social neuroscience and health: Neurophysiological mechanisms linking social ties with physical health. Nature Neuroscience, 15, 669-674.

33.

Else-Quest

N. M.

Higgins

Allison

Morton

L. C.

(2012). Gender differences in self-conscious emotional experience: A meta-analysis. Psychological Bulletin, 138, 947-981.

34.

Fan

Thompson

Wang

(1999). Effects of sample size, estimation methods, and model specification on structural equation modeling fit indexes. Structural Equation Modeling: A Multidisciplinary Journal, 6, 56-83.

35.

Figueira

(2010). A vivência de vergonha nos adolescentes: Estudo da vergonha externa e sua relação com a comparação social e o auto-criticismo [The experience of shame in adolescentes: A study of external shame and its relation with social comparison and selfcriticism] (Master’s thesis). University of Coimbra, Portugal.

36.

Francis

L. J.

Jackson

C. J.

(2004). Which version of the Eysenck Personality Profiler is best? 6-, 12- or 20-items per scale. Personality and Individual Differences, 37, 1659-1666.

37.

Galhardo

Cunha

Pinto-Gouveia

Matos

(2013). The mediator role of emotion regulation processes on infertility-related stress. Journal of Clinical Psychology in Medical Settings, 20, 497-507.

38.

Gilbert

(1992). Depression: The evolution of powerlessness. Hove: Lawrence Erlbaum, New York: Guilford Press.

39.

Gilbert

(1997). The evolution of social attractiveness and its role in shame, humiliation, guilt and therapy. British Journal of Medical Psychology, 70(2), 113-147.

40.

Gilbert

(1998). What is shame? Some core issues and controversies. New York, NY: Oxford University Press.

41.

Gilbert

(2003). Evolution, social roles, and the differences in shame and guilt. Social Research: An International Quarterly of the Social Sciences, 70, 1205-1230.

42.

Gilbert

(2007). The evolution of shame as a marker for relationship security: A biopsychosocial approach. In Tracy

Robins

Tangney

(Eds.), The self-conscious emotions: Theory and research (pp. 283-309). New York, NY: Guilford Press.

43.

Gilbert

(2010). CBT Distinctive Features. Compassion focused therapy: Distinctive features. London, England: Routledge.

44.

Gilbert

Baldwin

M. W.

Irons

Baccus

J. R.

Palmer

(2006). Self-criticism and self-warmth: An imagery study exploring their relation to depression. Journal of Cognitive Psychotherapy, 20, 183-200.

45.

Gilbert

Cheung

Irons

McEwan

(2005). An exploration into depression-focused and anger-focused rumination in relation to depression in a student population. Behavioural and Cognitive Psychotherapy, 33, 273-283.

46.

Gilbert

Miles

J. N.

(2000). Sensitivity to Social Put-Down: Its relationship to perceptions of social rank, shame, social anxiety, depression, anger and self-other blame. Personality and Individual Differences, 29, 757-774.

47.

Gilbert

Procter

(2006). Compassionate mind training for people with high shame and self-criticism: Overview and pilot study of a group therapy approach. Clinical Psychology & Psychotherapy, 13, 353-379.

48.

Goss

Gilbert

Allan

(1994). An exploration of shame measures—I: The Other As Shamer Scale. Personality and Individual Differences, 17, 713-717.

49.

Gratz

K. L.

Rosenthal

M. Z.

Tull

M. T.

Lejuez

Gunderson

J. G.

(2010). An experimental investigation of emotional reactivity and delayed emotional recovery in borderline personality disorder: The role of shame. Comprehensive Psychiatry, 51, 275-285.

50.

Bentler

P. M.

(1998). Fit indices in covariance structure modeling: Sensitivity to underparameterized model misspecification. Psychological Methods, 3, 424-453.

51.

Innamorati

Imperatori

Balsamo

Tamburello

Belvederi Murri

Contardi

. . .Fabbricatore

(2014). Food Cravings Questionnaire–Trait (FCQ–T) discriminates between obese and overweight patients with and without binge eating tendencies: The Italian Version of the FCQ–T. Journal of Personality Assessment, 96, 632-639.

52.

Innamorati

Tamburello

Contardi

Imperatori

Tamburello

Saggino

Balsamo

(2013). Psychometric properties of the attitudes toward self-revised in Italian young adults. Depression Research and Treatment, 2013, Article 209216.

53.

Kahn

J. H.

(2006). Factor analysis in counseling psychology research, training, and practice: Principles, advances, and applications. The Counseling Psychologist, 34, 684-718.

54.

Kim

Thibodeau

Jorgensen

R. S.

(2011). Shame, guilt, and depressive symptoms: A meta-analytic review. Psychological Bulletin, 137, 68-96.

55.

Kline

(2000). The handbook of psychometric testing (2nd ed.). London, England: Routledge.

56.

Kline

R. B.

(2011). Principles and practice of structural equation modeling (4th ed.). New York, NY: Guilford Press.

57.

Levy

L. H.

(1968). Originality as role-defined behavior. Journal of Personality and Social Psychology, 9, 72-78.

58.

Luoma

J. B.

Kohlenberg

B. S.

Hayes

S. C.

Fletcher

(2012). Slow and steady wins the race: A randomized clinical trial of acceptance and commitment therapy targeting shame in substance use disorders. Journal of Consulting and Clinical Psychology, 80, 43-53.

59.

MacCallum

R. C.

(1995). Model specification: Procedures, strategies, and related issues. In Hoyle

R. H.

(Ed.), Structural equation modeling: Concepts, issues, and applications (pp. 16-36). Thousand Oaks, CA: Sage.

60.

Maruyama

G. M.

(1997). Basics of structural equation modeling. Thousand Oaks, CA: Sage.

61.

Matos

Pinto-Gouveia

(2010). Shame as a traumatic memory. Clinical Psychology & Psychotherapy, 17, 299-312.

62.

Matos

Pinto-Gouveia

Costa

(2013). Understanding the importance of attachment in shame traumatic memory relation to depression: The impact of emotion regulation processes. Clinical Psychology & Psychotherapy, 20, 149-165.

63.

Matos

Pinto-Gouveia

Gilbert

Duarte

Figueiredo

(2015). The Other As Shamer Scale–2: Development and validation of a short version of a measure of external shame. Personality and Individual Differences, 74, 6-11.

64.

McCranie

E. W.

Bass

J. D.

(1984). Childhood family antecedents of dependency and self-criticism: Implications for depression. Journal of Abnormal Psychology, 93, 3-8.

65.

Miller

Chen

Cole

S. W.

(2009). Health psychology: Developing biologically plausible models linking the social world and physical health. Annual Review of Psychology, 60, 501-524.

66.

Mills

R. S. L.

Arbeau

K. A.

Lall

D. I.

De Jaeger

A. E.

(2010). Parenting and child characteristics in the prediction of shame in early and middle childhood. Merrill-Palmer Quarterly, 56(4), 500-528.

67.

Muthén

B. O.

Muthén

L. K.

(2012). Mplus 7 base program. Los Angeles, CA: Author.

68.

Muthén

L. K.

Muthén

B. O.

(2005). Chi-square difference testing using the Satorra-Bentler scaled chi-square. Available from www.statmodel.com

69.

Pinto-Gouveia

Matos

Castilho

Xavier

(2014). Differences between depression and paranoia: The role of emotional memories, shame and subordination. Clinical Psychology & Psychotherapy, 21, 49-61.

70.

Rasch

(1960). Probabilistic models for some intelligence and attainment tests. Copenhagen: Danish Institute for Educational Research.

71.

Ree

M. J.

French

MacLeod

Locke

(2008). Distinguishing cognitive and somatic dimensions of state and trait anxiety: Development and validation of the State-Trait Inventory for Cognitive and Somatic Anxiety (STICSA). Behavioural and Cognitive Psychotherapy, 36, 313-332.

72.

Rigdon

E. E.

(1996). CFI versus RMSEA: A comparison of two fit indexes for structural equation modeling. Structural Equation Modeling: A Multidisciplinary Journal, 3, 369-379.

73.

Roos

Hodges

E. V.

Salmivalli

(2014). Do guilt- and shame-proneness differentially predict prosocial, aggressive, and withdrawn behaviors during early adolescence? Developmental Psychology, 50, 941-946.

74.

Rüsch

Lieb

Göttler

Hermann

Schramm

Richter

. . .Bohus

(2007). Shame and implicit self-concept in women with borderline personality disorder. The American Journal of Psychiatry, 164, 500-508.

75.

Sandquist

Grenyer

B. F. S.

Caputi

(2009). The relation of early environmental experience to shame and self-criticism: Psychological pathways to depression. In Proceedings of the 44th APS annual conference, Darwin, 30 September 25th-4 October 2009 (pp. 161-166). Australia: Australian Psychological Society.

76.

Schermelleh-Engel

Moosbrugger

Müller

(2003). Evaluating the fit of structural equation models: Tests of significance and descriptive goodness-of-fit measures. Methods of Psychological Research Online, 8(2), 23-74.

77.

Scholte

R. H.

De Bruyn

E. E.

(2001). The Revised Junior Eysenck Personality Questionnaire (JEPQ-R): Dutch replications of the full-length, short, and abbreviated forms. Personality and Individual Differences, 31, 615-625.

78.

Smith

G. T.

McCarthy

D. M.

(1995). Methodological considerations in the refinement of clinical assessment instruments. Psychological Assessment, 7, 300-308.

79.

Smith

G. T.

McCarthy

D. M.

Anderson

K. G.

(2000). On the sins of short-form development. Psychological Assessment, 12, 102-111.

80.

Smith

R. H.

Webster

J. M.

Parrott

W. G.

Eyre

H. L.

(2002). The role of public exposure in moral and nonmoral shame and guilt. Journal of Personality and Social Psychology, 83, 138-159.

81.

Tanaka

J. S.

(1993). Multifaceted conceptions of fit in structural equation models. In Bollen

K. A.

Long

J. S.

(Eds.), Testing structural equation models (Vol. 154, pp. 10-39). Newbury Park, CA: Sage.

82.

Tangney

J. P.

Dearing

R. L.

(2003). Shame and guilt. New York, NY: Guilford Press.

83.

Tracy

J. L.

Robins

R. W.

Tangney

J. P.

(2007). The self-conscious emotions: Theory and research. New York, NY: Guilford Press.

84.

Vagos

da Silva

D. R.

Brazão

Rijo

Gilbert

(2016). Dimensionality and measurement invariance of the Other As Shamer Scale across diverse adolescent samples. Personality and Individual Differences, 98, 289-296.

85.

Van de Schoot

Lugtig

Hox

(2012). A checklist for testing measurement invariance. European Journal of Developmental Psychology, 9, 486-492.

86.

Von Eye

Bogat

G. A.

(2004). Testing the assumption of multivariate normality. Psychology Science, 46, 243-258.

87.

Wei

Shaffer

P. A.

Young

S. K.

Zakalik

R. A.

(2005). Adult attachment, shame, depression, and loneliness: The mediation role of basic psychological needs satisfaction. Journal of Counseling Psychology, 52, 591-601.

88.

Whelton

W. J.

Greenberg

L. S.

(2005). Emotion in self-criticism. Personality and Individual Differences, 38, 1583-1595.

A Validation Study of the Psychometric Properties of the Other As Shamer Scale–2

Abstract

Keywords

Introduction

Construct of External Shame

Measurement of External Shame

Aims of the current study

Method

Participants

Procedure

Materials

Other As Shamer Scale-2 (OAS-2)

The Experience of Shame Scale (ESS)

Teate Depression Inventory (TDI)

State–Trait Inventory for Cognitive and Somatic Anxiety (STICSA)

Statistical Procedure

Results

Discussion

Footnotes

Declaration of Conflicting Interests

Funding

Author Biographies

References