Analysis of Action Units,Emotions,and Voice Units in High and Low Self-Critical Participants During Self-Criticizing

Abstract

Self-criticism is considered a transdiagnostic predictor of psychopathology, treatment, and persistence. Hence, diagnosing the various levels of self-criticism might help in its early screening, prevention and then easier treatment. Recently, self-criticism has even been associated with the reduced capacity to cope with the COVID-19 pandemic. The aim of our research was to analyze differences in action units, emotions, and voice units in individuals with higher versus lower levels of self-criticism. Our convenience sample consisted of high and low self-critical individuals who ranked in the upper (26 participants) and lower (26 participants) 15th percentiles of the Hated Self subscale, according to the Slovak norms for the Forms of Self-criticizing/attacking & Self-reassuring Scale. Participants were selected from a total sample of 262 and were recorded while criticizing themselves using the two-chair technique. The recordings were analyzed using iMotions and Praat software. The multilevel random slope and random intercept analyses and logistic multilevel regression analyses were done in Program R. Compared to low self-critical participants, high self-critical participants showed significantly less fear and more anger. High self-critical participants used the following action units significantly less often than low self-critical participants: AU5 (Eye Widen), AU10 (Upper Lip Raise), AU12 (Smirk). Conversely, high self-critics used AU6 (Cheek Raise), AU1 (Inner Brow Raise), AU4 (Brow Furrow) significantly more often. In our sample, there were no differences in pitch or intensity of voice between high and low self-critical participants. The findings of this study may contribute to better diagnosis of individuals with higher and lower levels of self-criticism, irrespective of self-rating instruments.

Introduction

Self-criticism

According to Whelton et al. (2007) self-criticism is a negative belief about oneself and manifests differently in each individual – it can occur at any stage in life but may be present throughout the person’s life. An increased level of self-criticism is associated with a higher probability of psychopathology occurring, proving difficult to treat, and persisting (Duarte et al., 2016; Gilbert & Procter, 2006; Letovancová & Halamová, 2016). According to Blatt and Zuroff (1992) self-criticism is accompanied by negative feelings such as depression, unworthiness, weakness, inferiority, failure, guilt, and helplessness. To this Kannan and Levitt (2013) add sadness, loneliness, and low positive feelings, while Gilbert et al. (2004) refer to inadequacy, contempt, aversion, disgust, and hatred. In extreme cases, individuals may also attempt to harm themselves or others (Blatt & Zuroff, 1992). According to Gilbert et al. (2004) there are two kinds of self-criticism: Inadequate Self and Hated Self. In cases of Inadequate Self, the individual considers their behavior to be inadequate according to social norms. In cases of Hated Self, the individual’s intention is to harm themselves. Self-critical individuals have problems in their relationships with others (Letovancová & Halamová, 2016) because they often suffer from depression and social anxiety (Gilbert et al., 2004), feelings of inferiority, or are more submissive to others (Blatt & Zuroff, 1992). In recent research, self-criticism has also been linked to coping with the COVID-19 pandemic because self-criticism, together with dependence on others, may cause a higher degree of loneliness and therefore such individuals may be less able to cope (Besser et al., 2022).

Emotions, Facial Expressions, and Action Units

Ekman is one of the leading researchers in the field of emotions and facial expressions (2014). His view is that every emotion consists of certain action units and so he invented the Facial Action Coding System (FACS, Ekman et al., 2002; Ekman & Friesen, 1978) which is also used in the present paper. Hjortsjö (1969) created the first 23 units of the FACS and other authors (Ekman & Friesen, 1978) further expanded them. The FACS was later updated (Ekman et al., 2002). The FACS is based on anatomical facial movements (composed of action units) and can therefore be used to describe these movements (Ekman & Friesen, 1978). Action units are according to Ekman and Friesen (1982)the minimal units that are anatomically separate and visually distinguishable. In addition, Ekman (2003) identified seven basic (primary) emotions (surprise, fear, joy, disgust, sadness, contempt, anger). Some facial expressions or emotions can also indicate a lie or socially desirable behavior (Ekman, 2003). Although manual coding of facial expressions may be reliable (even for complex facial expressions), it is much more time consuming than automatic coding (Girard et al., 2013). Just learning the FACS requires 100 hr of study (FACS, n.d.), not to mention the coding itself. Another potential disadvantage of manual facial recognition is attention span – people are not always able to fully concentrate on a long-lasting stimulus. Moreover, another potential disadvantage is subjectivity – for example the notion that women are more emotional than men (Barrett et al., 1998). Conversely, automated coding still needs to be closely watched for errors. Real-time coding is possible with the use of software (Anderson & McOwan, 2006), but there are a few disadvantages. Participants may divert their attention away from the stimulus, cover their face, or shift body positions, all of which are obstacles that manual coders are recommended to pay attention to (Kring & Sloan, 1991). They may lead to a deterioration in the software performance, or a manual inspection may be required as well. Also, if the videorecording is not high quality (e.g., the lighting conditions are bad), the software may struggle with the facial expressions (Wesley et al., 2012). Finally, in some studies, automated systems were comparable with manual coding of the FACS (Girard et al., 2013; Lewinski et al., 2014; Torre et al., 2011), which suggests they are very effective in research. Therefore, we chose to do automated coding over manual coding.

Voice Units

There is a lot of research on recognizing emotions from speech (Ali et al., 2015; Chuang & Wu, 2004; Kumbhakarn & Sathe-Pathak, 2015; Magdin et al., 2019; Rong et al., 2007). In the early days (1997–2007), basic acoustic parameters were the primary choice (Rong et al., 2007). They are based on extracted attributes that are related to pitch, intensity, and duration. Rong et al. (2007) further stated that the results of several studies are contradictory. For example, a higher mean pitch indicates a negative emotion in some studies and a positive emotion in others. Whereas Ali et al. (2015) argue that pitch is essential for accurate emotion classification, unlike other prosodic features. Without pitch, the accuracy is only about 20%, with pitch the classification accuracy is about 40%. People express their emotions not only through facial expressions, but also different vocal attributes (Dasgupta, 2017). The results of research by Busso et al. (2004) show that facial expressions and voice expressions complement each other and that when the two are combined the accuracy of the emotion recognition system is measurably improved.

Expressing Self-Criticism

Several studies have investigated the self-critical voice (Halamová et al., 2021; Whelton & Greenberg, 2005; Whelton & Henkelman, 2002). Negative self-evaluation consists of several aspects – appearance, performance, thoughts, emotions, etc. (Iancu et al., 2015). Several recent research studies have tried to find out what a self-critical monolog sounds like (Bailey et al., 2024; Whelton & Greenberg, 2005) and which facial expressions are involved (Bailey et al., 2023; Halamová et al., 2020; Whelton & Greenberg, 2005). The findings of both these recent studies were obtained from a general sample rather than a sample of high and low self-critical participants (Bailey et al., 2023, 2024). However, the differences between the two groups may help in diagnosis because, as we stated in the introduction, high levels of self-criticism are often associated with psychopathology (Duarte et al., 2016; Gilbert & Procter, 2006). The self-critical dialog (Bailey et al., 2024) was found to have a high intensity but low pitch. The research study was conducted on 12 commercially available videos of Emotion Focused Therapy (EFT, L. S. Greenberg, 2002) using Praat software (Boersma & Weenink, n.d.). Whereas Whelton and Greenberg (2005) used Specific Affect Coding System (Gottman et al., 1996) to detect emotions in low and high self-critical participants, which is a coding system in which the result is obtained based on verbal content, context, tone of voice, and facial and bodily expressions. They videorecorded the participants’ self-criticism and response to self-criticism, and their findings suggest that highly self-critical participants expressed more contempt and disgust than the low self-critical participants. Halamova et al. (2019) tried to establish which facial expressions in the Facial Action Coding System (FACS, Ekman et al., 2002; Ekman & Friesen, 1978) are exhibited during participants’ self-critical monolog. They used iMotions software to identify the expressions automatically (iMotions, 2020). The most frequent were “Mouth Open,”“Smile,”“Jaw Drop,”“Eye Closure,”“Eye Widen,”“Cheek Raise,”“Dimpler,”“Lip Press,” and “Brow Raise.” The authors concluded that people may be more contemptuous, frightened, and ashamed when exposed to self-criticism, regardless of level of self-criticism. In EFT theory, self-criticism is seen as a maladaptive type of anger turned inward (Pascual-Leone et al., 2013; Pascual-Leone & Greenberg, 2007). In highly self-critical individuals in particular, self-criticism is associated with self-hatred and self-contempt (Blatt & Zuroff, 1992; Whelton & Greenberg, 2005). Additionally, Pascual-Leone et al. (2013) claimed that high self-criticism manifested with excessive and inappropriate intensity. These findings are consistent with other research showing that anger, one of the emotions with high sympathetic arousal, is associated with higher levels of intensity (Kapoor & Sagar Verma, 2019; Schröder et al., 2001) and higher levels of pitch (Breitenstein et al., 2001; Yildirim et al., 2004).

Higher level of self-criticism is linked to negative feelings, behaviors, and beliefs about oneself and can further proceed to psychopathology. Hence, diagnosing the various levels of self-criticism might help in its early screening, prevention and then easier treatment (Werner et al., 2019). This work might contribute to diagnostics, because there is a lot of research on self-criticism but not in relation to facial expressions or voice units on samples of high and low self-critics, despite the potential clinical benefits.

The Aim of the Study

Therefore, the aim of the research was to identify and analyze action units, emotions, and voice units on a sample of high and low self-critics. Based on the previous research studies we formulated the following two hypotheses and to further elaborate on the differences between the two groups of high and low self-critics beyond the already known, we stated three research questions.

• H1: Compared to low-self-critical participants, high self-critical participants will more often exhibit emotions of disgust and contempt (Whelton & Greenberg, 2005).

• H2: Compared to low-self-critical participants, high self-critical participants will more often exhibit the emotions of anger (Pascual-Leone et al., 2013; Pascual-Leone & Greenberg, 2007).

• Q1: How do emotions exhibited by high self-critical participants differ from those in low self-critical participants?

• Q2: How do facial action units exhibited by high self-critical participants differ from those in low self-critical participants?

• Q3: How do vocal expressions of intensity and pitch exhibited by high and low self-critical participants differ from each other?

In our paper, we will first introduce the methods (procedure, sample, measurement instrument and data analysis consisting of analysis of emotions, action units, voice units and statistical analysis). The results provide the statistical analysis from the iMotions software – emotions, then action units, finishing with statistical analysis of voice units from the software Praat. We discuss these results in discussion, while also providing limitations and future research suggestions, ending with brief conclusion.

Methods

Procedure

The research procedure for the data collection was based on the two-chair technique from Emotion Focused Therapy (L. S. Greenberg, 2002) and research by Whelton and Greenberg (2005) and Kramer and Pascual-Leone (2016). According to L. S. Greenberg (1979), two chairs represent the two sides of the self, engaging in dialog (L. Greenberg et al., 1993). During the dialog, the client expresses their feelings, thoughts, attitudes, needs, etc. to resolve intrapsychic conflict that he/she experiences through the integration of opposing aspects of the self (L. S. Greenberg, 1983). Due to COVID-19 pandemic, there were two versions of the procedure. Face-to-face two-thirds and online one-third. Both were done with comparison in mind later on. The limiting aspects are discussed in limitations. At the beginning, participants signed an electronic informed consent form and filled in a short socio-demographic questionnaire and the FSCRS (Gilbert et al., 2004). First, the female research assistant evoked a self-critical situation through the imagination (for 2.5 min, the participant remembered a situation when they were dissatisfied with themself or when they failed – what happened, who was there, how they felt etc.). In the online version, one of the research assistants was male. Then, for 3 min, they became their critical voice and had to criticize themselves in the way they usually do. After that, again for 3 min, they had to respond to their 3 min long self-criticism (how they usually defend or protect themselves against their self-critical voice). There was only one research assistant at a time and stayed the whole time, because if the participant got stuck or couldn’t continue, the research assistant helped by asking questions for example, “What specific words do you use when you criticize yourself?,”“What else do you usually say to yourself when something goes wrong?.” The criticizing and the response to the criticizing were recorded, but only self-criticizing was studied in the present study. There was also no baseline moment recorded.

Measurement Instruments

The Forms of Self-Criticizing/Attacking & Self-Reassuring Scale

In the research study we used a 22-item scale – the Forms of Self-criticizing/attacking & Self-reassuring Scale (FSCRS, Gilbert et al., 2004) to measure level of self-criticism. The scale assesses two kinds of self-criticism: Hated Self and Inadequate Self. In addition, it assesses Reassuring Self. Participants answer questions on a 5-point scale from 0 (“not at all like me”) to 4 (“completely like me”). The reliability of the original scale was determined using Cronbach’s alpha, which was between .86 and .90. In terms of validity, the FSCRS was correlated with the Functions of Self-criticizing/attacking Scale (FSCS, Gilbert et al., 2004), and ’Hated self’ from FSCRS correlated positively with Self-correction and Self-persecution. Regarding the Slovak version of the scale, Halamová et al. (2017) reported good psychometric properties including reliability, validity, and factor structure. Like the original FSCRS, the Slovak version showed good Cronbach’s alpha, ranging from .75 to .85. According to the Mokken analysis, all the FSCRS subscales proved to be scalable (Halamová et al., 2017). The FSCRS is widely used for defining the level of self-criticism by researchers all over the world and it is the best measuring instrument for the level of self-criticism in Slovakia with huge support from cross-cultural studies as well (Halamová et al., 2018, 2019; Kanovský et al., 2021).

Research Sample

The sample was selected by snowball sampling and on the basis of availability. Similar research (Whelton & Greenberg, 2005) mentioned above had 15 participants per group in their study but used upper and lower 35th percentile to determine high self-critical participants. We selected our participants based on a lower percentile to ensure that their self-criticism is extremely low/high. Based on Gaussian distribution, we decided to select people with higher and lower score than one standard deviation σ away from the mean, where 68% of the values lie. In addition, we used score from “Hated self” from FSCRS which is considered to be the indicator of pathology and not perfectionism as it is in case of ’Inadequate self’ from FSCRS (Gilbert et al., 2004). To be sure about the exact number of needed participants, we also conducted repeated-measures ANOVA power analysis. With the nonsphericity correction (0.8) and medium effect size (0.5), the predicted total sample was 42 (21 in each group). Our sample therefore consisted of high and low self-critical individuals who ranked in the upper and lower 15th percentiles (85%–100% for highly self-critical and 0%–15% for low self-critical) on ’Hated self’ from FSCRS according to the Slovak norms for the FSCRS (Halamová et al., 2017). Participants were selected from a total sample of 262. Only 52 participants, 14 men and 38 women aged 18 to 58 years (M = 27.15, SD = 11.35) met the condition of high and low self-criticism mentioned above. The sample of low self-critics consisted of a total of 26 participants, 21 women and 5men aged 18 to 58 years (M = 30.92, SD = 13.34). The sample of high self-critics consisted of a total of 26 participants, 17 women and 9 men aged 19 to 58 (M = 23.38, SD = 7.44).

The study’s protocol was approved by the Ethical committee of a related university.

Data Analysis

Analysis of Emotions and Action Units

We used iMotions 8.2.4.0 software (The iMotions Platform, n.d.) to analyze the emotions and action units. In the iMotions analysis, the stimuli are paired with a face coding system – the Facial Action Coding System (FACS, Ekman et al., 2002; Ekman & Friesen, 1978), which allows the software to determine a given facial expression or emotion (Hjortsjö, n.d.). We analyzed all 20 action units recognizable by iMotions (AU1: Inner Brow Raise, AU2: Brow Raise, AU4: Brow Furrow, AU5: Eye Widen, AU6: Cheek Raise, AU7: Lid Tighten, AU9: Nose Wrinkle, AU10: Upper Lip Raise, AU12: Smirk, AU12: Smile, AU14: Dimpler, AU15: Lip Corner Depressor, AU17: Chin Raise, AU18: Lip Pucker, AU20: Lip Stretch, AU24: Lip Press, AU25: Mouth Open, AU26: Jaw Drop, AU28: Lip Suck, AU43: Eye Closure) and all seven basic emotions (Anger – AU4 + AU5 + AU7 + AU23, Joy – AU6 + AU12, Disgust – AU9 + AU15 + AU16, Fear – AU1 + AU2 + AU4 + AU5 + AU7 + AU20 + AU26, Sadness – AU1 + AU4 + AU15, Contempt – AU12 + AU14, and Surprise – AU1 + AU2 + AU5 + AU26; The iMotions Platform, n.d.).

Analysis of Voice Units

The Praat analysis of the voice units (Boersma & Weenink, n.d.) helped to clarify which voice units participants use and to identify recurring patterns in low and high self-critics. The software is free and works on computers with Windows, Macintosh, Linux, Solaris and FreeBSD. Boersma and Weenink (n.d.) developed Praat to analyze voice units – intensity, pitch, jitter, shimmer, and voice breaks. For the purposes of this study, we will analyze the two main commonly used voice units, intensity and pitch, because they can be directly measured using Praat (Boersma, 2013) while others cannot be analyzed by Praat. These voice units can effectively characterize emotions and are most commonly used (Boersma, 2013 Due to software limitations, we had to convert the recordings from .mp4 to .wav (Styler, 2013 We edited the sound of the recordings in Audacity, version 2.2.1, which is free to use (Team, 2017), to remove background noise, other noises, disruptive components on the soundtrack and silent intervals. Praat is used in linguistics research (Boersma & Van Heuven, 2001; Styler, 2013) but the software is also popular among researchers who study emotions (Kumbhakarn & Sathe-Pathak, 2015; Magdin et al., 2019).

We used the function “pitch listing” and “intensity listing” on the entire sample because before inserting into Praat we had cut the parts where the researcher spoke and other distracting elements – we only had the participant’s speech left. The software extracted the pitch every 0.04 s and intensity every 0.01 s.

Statistical Analysis

Statistical analyses were conducted in program R (version 4.0.2), with the use of package “lme4” (Bates et al., 2015), because in both cases the data (Praat, iMotions) contained repeated measurements of the individuals. We used multilevel random slope and random intercept analyses. We tested likelihood-ratio tests with Bayesian Information Criterion (Schwarz, 1978) and Akaike’s Information Criterion (Akaike, 1998) with information about significance (>0.05) for comparison of models (to be sure about model fit). We run three comparisons in the case of emotions, three comparisons in the case of action units, two comparisons in case of pitch and two comparisons in the case of intensity. Only the best fits are mentioned. For iMotions, the first model contained EM (variability of emotions), ID (variability of participants), and Group (high or low self-critics), whereas the second model contained AU (variability of action units), ID (variability of participants), and Group (high or low self-critics). EM, AU, and ID are the random effects, Group is the fixed effect. The first model concerned the absence or presence of emotions and the second model the absence or presence of action units. For this purpose, we used a logistic multilevel regression model and set the absolute threshold to 50 (less than 50 = 0, more than 50 = 1), which was recommended in the software manual (IMotions, 2020). For the Praat voice unit analyses, we divided intensity and pitch. Both models contained Code (variability of participants) and Group (high or low self-critics). Group and Code are the random effects.

Results

Results of Analysis of Emotions

Firstly, we present the results of the iMotions emotion analysis. The total number of observations for emotions was 1,881,159. Variability of emotions was slightly higher (7.97) than variability of participants (2.25). That means that the differences between the individual emotions are greater than the differences between the individual participants (See Appendix Tables A1–A3). Conditional R² was .72, which indicates a high effect size. Hypothesis 1 was not confirmed while hypothesis 2 was confirmed. High self-critical participants showed significantly less fear and significantly more anger than low self-critical participants. A comparison of the frequencies of emotion used by each group of high versus low self-critical participants is presented in Figure 1.

Figure 1.

Comparison of emotions in low versus high self-critical participants.

Results of Analysis of Action Units

Secondly, we present the iMotions results for the action unit analysis. The total number of observations for the action units was 5,374,740. Action unit variability had a higher value (1.86) than participant variability (0.60). We can therefore conclude that the differences between the action units are greater than the differences between participants. Conditional R² was .43, which indicates a medium effect size. Compared to low self-critical participants, High self-critical participants used the following action units significantly less often AU5 – Eye Widen, AU10 – Upper Lip Raise, AU12 – Smirk. On the other hand, high self-critics used significantly more often compared to low self-critics: AU6 – Cheek Raise, AU1 – Inner Brow Raise, AU4 – Brow Furrow. See Figure 2.

Figure 2.

Comparison of action units in low versus high self-critical participants.

Results of Analysis of Voice Units

Thirdly, we present the results of the statistical analysis of voice units. The total number of observations of voice pitch was 458,286. The variability of the participants was 1.404e + 03, whereas the variability of groups was 1.689e-06. Conditional R² had medium effect size (.39). In our sample, there were no differences in pitch between high and low self-critical participants (see Figure 3).

Figure 3.

Comparison of pitch between low and high self-critical participants.

In the case of voice intensity, the total number of observations was 690,099. Variability of participants reached 17.74, while in this case too, the value of variability of the groups reached 0. Conditional R² had small effect size (.14). Similarly to pitch, we do not observe significant differences between high and low-self-critical participants (see Figure 4).

Figure 4.

Comparison of intensity between low and high self-critical participants.

Discussion

The aim of the research was to examine differences between participants with high and low levels of self-criticism, which could further help to improve the diagnostics of self-critical people and their level of self-criticism. We focused on under-researched indicators – facial emotions, action units, and voice units. In the first hypothesis, we tested Whelton and Greenberg’s (2005) findings. However, our results were not in line with the hypothesis because highly self-critical participants did not express more disgust and contempt. Additionally, High self-critical participants showed significantly less fear and significantly more anger than low self-critical participants. The differences could be down to different methods of data collection, measurement of self-criticism, and data analysis. In the original research (Whelton & Greenberg, 2005), the emotions were identified from multiple sources (context, verbal content, facial expressions, etc.) using the Specific Affect Coding System (SPAFF, Gottman et al., 1996). Nevertheless, even based on research with very similar data collection methods and the same data analysis method – iMotions and multilevel analyses in program R (Halamová et al., 2020) – it was not possible to predict which emotions the highly self-critical participants would express during the self-criticizing compared to low self-critical participants. In the previous study (Halamová et al., 2020), the results showed that regardless of the level of self-criticism, the participants were more contemptuous, frightened, and ashamed. However, Halamová et al. (2020) did not test the differences between high and low self-critical participants and tested the occurrence of action units and not the emotions themselves. Therefore, the differing results could be down to how iMotions detects emotions on the faces of participants or they could reflect real differences between low and high self-critical participants as we used ’Hated self’ from FSCRS as the measurement of self-criticism which is more pathological form of self-criticism (Gilbert et al., 2004).

The second hypothesis tested the incidence of anger. This hypothesis was supported as high self-critical participants expressed significantly more anger compared to low self-critical ones. Our results are in line with the findings of Kramer and Pascual-Leone (2016) showing that more anger is expressed when self-criticizing because highly self-critical people have angry and hostile expressions toward themselves. However, Kramer and Pascual-Leone (2016) did not test differences between low and high self-critics so we might interpret the results in a way that low self-critics did not express so much anger because they are able to accept their inner experiences including failures more and are more kind toward themselves (Gilbert & Irons, 2008). The lower occurrence of fear in our study might be explained by habituation to self-criticizing: high self-critical participants criticize themselves regularly and are therefore used to it and not fearful as it is a very familiar albeit unpleasant situation for them. In addition, our results are supported by other research study showing that high self-critical people are rather fearful of self-compassion but not of self-criticism (Gilbert et al., 2012).

Apart from emotional differences tested in research question one, research question two was related to differences in the action units between the two groups. Compared to low self-critical participants, high self-critical participants used the following action units significantly less often: AU5 (Eye Widen), AU12 (Smirk), and AU10 (Upper Lip Raise). On the other hand, high self-critics used the following significantly more often than low self-critical participants: AU6 (Cheek Raise), AU1 (Inner Brow Raise), AU4 (Brow Furrow). In the study by Halamová et al. (2020) the following action units were found to be related to self-criticism generally for all participants: AU25 (Mouth Open), AU12 (Smile), AU26 (Jaw Drop), AU43 (Eye Closure), AU5 (Eye Widen), AU6 (Cheek Raise), AU14 (Dimpler), AU24 (Lip Press), and AU2 (Brow Raise). In fact, our study confirmed the most frequent AU in both (low and high self-critics) which were AU25 (Mouth Open), AU12 (Smile), AU6 (Cheek Raise), AU26 (Jaw Drop), AU43 (Eye Closure) and AU2 (Brow Raise). On the top of the previous research studies, our results showed that, AU5 (Eye Widen) was significantly more frequent for low self-critics and AU6 (Cheek Raise) was more frequent for high self-critics. These two action units might be possibly the key discerning features between low and high self-critics. However, that must be further explored because the spoken language was assessed by software so the opening of the mouth (AU25 Mouth Open) could be and in reality is the most frequent action unit of all the other action units in both studies. In addition, action unit AU12 (Smile) could indicate social desirability (Gladstone & Parker, 2002) and social embarrassment (Keltner & Anderson, 2000). This may also be indicative of the ability of participants in front of the other person to open up or express themselves and reflect on their thinking during the two-chair dialog. The high self-critics were probably more eloquent because they were better able to verbalize their criticism as they are more used to self-criticizing (Gilbert et al., 2004) and it may well be rewarding in a way as they themselves are in control of it and kind of good at it. In another similar study (Bailey et al., 2023), self-critical monolog was characterized by more frequent AU2 (Brow Raise), AU1 (Inner Brow Raise), AU12 (Smirk), AU7 (Lid Tighten), and less frequent AU5 (Eye Widen), AU43 (Eye Closure), AU20 (Lip Stretch). Compared to our results, only AU1 (Inner Brow Raise) as more frequent action unit with less frequent AU5 (Eye Widen) were characterized for high self-critics, but AU12 (Smirk) and AU7 (Lid Tighten) were less frequent action units compared to almost all other action units (in both cases). Moreover, AU43 (Eye Closure) was one of the most frequent action units in both – low and high self-critics. Similarly, participants in Bailey et al. (2023) showed significantly more AU43 (Eye Closure) during self-criticizing no matter how self-critical they were. According to Vredeveldt and Penrod (2013), eye closure was effective in facilitating memory of an event without an increase in errors by reducing cognitive load and enhancing visualization (Vredeveldt et al., 2011). As the task required to remember self-criticizing, it is understandable that participants have their eyes closed to remember better and probably also to more concentrate on inner experiencing generally not only on memory but thoughts, feelings, sensing etc.

We were unable to confirm emotion of contempt in high self-critics even from action unites perspective. Contempt is built from AU12 (Smirk) and AU43 (Mouth Open; Keltner, 1996) Action unit “Smirk” as unilateral smile or unilateral lip corner raise, was the least frequent among all other action units. And contrary to our expectations and Whelton and Greenberg (2005) findings, it was significantly less used by high self-critical participants compared to low self-critical participants. Unfortunately, the current research methodology and software unable us to find out parallel use of several action units at the same time. Therefore, it is difficult to know whether AU12 (Smirk) and AU43 were used simultaneously to create the expression of contempt as the software calculates probability from AU12 (Smirk) and AU14 (Dimpler).

In addition, according to Wegrzyn et al. (2017), emotions in the mouth and eye area are more recognizable. We can see that in the high self-critics the important facial action units were noticed in outer parts of the eyes and the mouth (AU6 – Cheek Raise, AU1 – Inner Brow Raise, and AU4 – Brow Furrow). However, in the high self-critical participants it was just mouth open, smile and jaw drop that were more frequent, not any of the action units relating to the eyes which might yield in less readable faces of high self-critical people and which might further negatively influence their social life and serve as a vicious circle of self-criticism.

Regarding the voice units during emotion focused therapy video recordings, the self-critical dialog (Bailey et al., 2024) was found to have a high intensity but low pitch. However, our findings do not suggest significant differences between the high and low self-critical groups. In both cases, we found relatively high state variability compared to participant variability, which may suggest that pitch and intensity play no role as a diagnostic parameter in the self-critical monolog between high and low self-critics. Similarly, comparing high and low self-protective participants during self-protective communication did not show any differences in intensity or pitch (Vráblová & Halamová, 2025).

Future Research

Future research can further establish the relationship between the emotion expressions and voice expressions and other nonverbal behaviors with a larger database of individuals with self-criticisms evaluated on a continuous spectrum and machine-learning approaches to be able to comprehensively identify the effective cues and so to easily distinguish between these groups.

As the study did not find the difference in pitch and intensity between individuals of high and low self-criticism using Praat software, the research studies in future could use the broadened set of features and include more voice characteristics to the analysis. Hence, the changes of actual pitch and intensity could only be salient in specific pragmatic scenarios (such as the expression of anger and shame expression in a specific emotional state) future research should include the discourse-pragmatic information of these specific emotions and observe their differences, rather than just compare mean differences in the acoustic cues. In addition to pitch and intensity, the timbre and duration of sound related to the subject’s acoustic physical attributes, the time of beginning and ending, and the evaluation of self-voice attractiveness related to the subject’s social attributes could also vary with the degree of self-criticism. It is highly suggested in future to investigate a larger-range of acoustic information to enrich the acoustic results.

Limitations

One of the limitations was the two variants of the research data gathering procedure: online and personal. Despite our attempts to eliminate all disruptive moments and silent intervals, it may be that the recording sound varied from camera to camera. In addition, it was not possible to alter the web cameras, microphones, and the stability of the participants’ internet connection. As Wesley et al. (2012) noted, the lighting conditions could present problems for the face recognition software. These limitations, however, may not fully apply in our case, because the iMotions software is updated frequently to fix possible problems and we also edited video recordings (e.g., cutting crashed or stuck parts on one image) to reduce possibility of such errors. An important aspect is social desirability, that is, the possibility of distortions in the expression of emotions – especially on the participant faces and in their voices, because they could be conscious about being recorded and about the research assistant being present. Another significant limitation of the research is the sample. Speaker age is also reflected in the voice (Schötz, 2007). According to the author of the article, it can be measured acoustically using automatic age estimation. He emphasizes that the voice units that may change with age include speech rate, fundamental frequency, and sound pressure level but that the relationships between individual voice units are relatively complex and are influenced by several factors. There may be differences between men and women, sick and healthy, or, for example, between people whose perceived age differs from their actual age. In our case, we had a significantly larger number of women than men and did not vary greatly in participant age. Also, there were differences in age between low and high self-critics in our sample. That meant we could examine the impact of age on the results and not the level of self-criticism. In addition, we would highly recommend that further research include the baseline measurement, as that could serve as the basis for comparisons. This is because high and low self-critical participants are likely to exhibit facial differences at the very beginning of the research study because of differences in facial appearance (Hess et al., 2009), consisting of slow (aging-related changes) and static (e.g., facial width-to-height ratio) signals from the face in comparison with fast (movements of facial muscles) signals (Ekman, 2003).

Conclusion

So far, no one has examined facial expressions and vocal units together during self-criticism comparing a sample of high and low self-critical participants. In our study of testing differences between these two groups, we found out that high self-critical participants showed less fear and more anger than low self-critical participants. High self-critical participants also used the following action units significantly less often than low self-critical participants: AU5 – Eye Widen, AU10 – Upper Lip Raise, AU12 – Smirk. Furthermore, high self-critics used AU6 – Cheek Raise, AU1 – Inner Brow Raise, AU4 – Brow Furrow significantly more often than low self-critical participants. These results could serve as the basis for further research on the specification of a diagnostic indicator for highly self-critical individuals. For practice, these results might help in training helping professionals to distinguish between high and low self-critical clients and patients in screening so to do the early prevention and the easier treatment.