Sage Journals: Discover world-class research

Abstract

Emotional facial expressions convey crucial information in nonverbal communication and serve as a mediator in face-to-face relationships. Their recognition would rely on specific facial traits depending on the perceived emotion. During the COVID-19 pandemic, wearing a facemask has thus disrupted the human ability to read emotions from faces. Yet, these effects are usually assessed across studies from faces expressing stereotypical and exaggerated emotions, which is far removed from real-life conditions. The objective of the present study was to evaluate the impact of facemasks through an emotion categorization task using morphs ranging from a neutral face and an expressive face (anger, disgust, fear, happiness, and sadness) (from 0% neutral to 100% expressive in 20% steps). Our results revealed a strong impact of facemasks on the recognition of expressions of disgust, happiness, and sadness, resulting in a decrease in performance and an increase in misinterpretations, both for low and high levels of intensity. In contrast, the recognition of anger and fear, as well as neutral expression, was found to be less impacted by mask-wearing. Future studies should address this issue from a more ecological point of view with the aim of taking concrete adaptive measures in the context of daily interactions.

Keywords

emotion recognition facial expressions facemasks emotional intensity

Since the beginning of the COVID-19 pandemic in 2020, facemasks have been widely used as a protective measure against the virus transmission. By altering the amount of available information on faces, masks would impair emotion recognition (Grundmann et al., 2021; Lau, 2021) and could thus strongly impact daily social interactions (Bani et al., 2021). Indeed, a correct perception of facial emotion is crucial in interpersonal relationships, since facial expressions convey information about emotions, states of mind, intentions, or opinions of a person (Guo et al., 2022; Shehu et al., 2022).

Studies on emotion recognition revealed that happiness is the most easily recognized emotion (Kesler-West et al., 2001; Leppänen & Hietanen, 2004; Palermo & Coltheart, 2004) while fear is less well recognized (Palermo & Coltheart, 2004; Russell, 1994). The expression of fear is indeed often confused with surprise (Palermo & Coltheart, 2004), and this confusion has been mainly attributed to the physical proximity between these two emotions (Gosselin & Simard, 1999).

These differences in the processing of expressions would be based on the specific involvement of certain facial features, located either in the lower, upper, or central part of the face, as well as to their respective salience (Bassili, 1979; Grahlow et al., 2022; Wegrzyn et al., 2017). Several techniques have been used to determine the relative importance of facial features in emotion recognition, such as isolating the upper or lower part of a face (Blais et al., 2012) or masking facial features individually with a rectangular shape, for example (Bassili, 1979; Carbon, 2020; Grahlow et al., 2022). Objects, such as sunglasses (Roberson et al., 2012), burqas, caps, shawls, niqabs, or scarves (Kret & De Gelder, 2012), have also been used as masking techniques. These studies revealed that individuals focus more frequently on the eye region in order to recognize emotions such as fear or surprise (Guo, 2012) and that facial emotion recognition is strongly altered when this region is occluded (Kim et al., 2022). Furthermore, Blais et al. (2012), by means of the Bubbles technique, revealed that occluding the lower part of the face reduced the accuracy of overall performances during an emotion categorization task. In particular, the lower part of the face would play an important role in the recognition of happiness and disgust (Adolphs et al., 2005; Bombari et al., 2013; Smith et al., 2005; Wegrzyn et al., 2017). Further studies also argued in favor of the importance of the region around the mouth in the recognition of other emotions, such as sadness or anger (Carbon, 2020).

Recent studies following the COVID pandemic found, mainly through emotion categorization tasks, significant effects of facemasks reflected by a decrease in accuracy (Carbon, 2020; Grahlow et al., 2022; Lau, 2021; Shehu et al., 2022) and an increase in reaction time (Maiorana et al., 2022) of participants in mask condition compared to no-mask condition. More particularly, when a facemask was worn, the recognition of some emotions would be particularly impacted, including happiness. In contrast, the expression of fear (Bani et al., 2021) would still be accurately recognized despite facemasks. This impairment in the recognition of expressions with facemasks could be due to a reduction in holistic perception of faces, which would lead individuals to rely more on available components within faces to recognize them (Freud et al., 2020). For instance, using the mouth as a face cue seems to be an effective strategy when recognizing happiness, but this strategy would be strongly impaired by facemasks (Lau, 2021). Masking the lower part of the face might also lead to a misinterpretation of emotions (Shehu et al., 2022). Indeed, participants would often interpret emotions as neutral (Carbon, 2020; Shehu et al., 2022) or even confuse disgust with anger or sadness (Carbon, 2020; Kim et al., 2022). The use of face cues located in the upper part of the face, such as the eyes and eyebrows, for example, might improve facial emotion recognition (Barrick et al., 2021), but this strategy would not always be enough to accurately recognize emotions.

Despite the growing number of studies published on this topic since the start of the COVID-19 pandemic, most have however been conducted online, inducing response biases (lack of control in the time and context in which the survey has been answered). This could explain the heterogeneity of results of the literature concerning the type of expressions whose recognition is impacted by wearing a mask. Besides, many of these studies have used expressive faces from databases whose emotions are often stereotyped and exaggerated, while in our everyday life, emotions are expressed subtly along a continuum ranging from 0 to 100% of their intensity. Taking these issues into consideration, the objective of the present study was to examine the impact of wearing a mask on the recognition of expressions presented at different levels of their intensity by creating morphs between a neutral face and an expressive face (anger, disgust, fear, happiness, and sadness) (ranging from 0% neutral to 100% expressive in 20% steps). The use of these stimuli appeared more ecological (even if this is still far from real-life conditions) and seemed to reflect more accurately the plurality of situations in which individuals perceive the expressions conveyed by the faces of others. Based on data from the literature, we hypothesized that wearing a facemask would alter emotion recognition, in particular, disgust, sadness, and happiness, both for high and low degrees of their intensity.

Methods

Participants

Thirty-six undergraduate students (20 females, 18–32 years old, $M_{a g e} = 22 years$ , SD = 3.04 years) at the Université de Lorraine (Nancy, France) provided signed and informed consent for their participation in the experiment. All participants reported normal or corrected-to-normal vision. None of the participants reported any history of psychiatric or neurological disorders. Prior to the study, the sample size was determined from a GPower analysis (Faul et al., 2007) which was conducted targeting a repeated-measures analysis of variance (ANOVA) with Mask* (mask vs. no mask), Emotion (five emotions and a neutral expression), and Intensity (0% to 100% of emotion) conditions. Through this analysis, we required a sample size of 36 participants to achieve a statistical power of 0.8 with an effect size of $f^{2} = 0.25$ , at a significant level of 5%.

Stimuli

The face stimuli were created from colored photographs of four different individuals (two females), all Caucasian, taken from the Karolinska Directed Emotional Faces (Lundqvist et al., 1998; set of stimuli used by Baudouin et al., 2023). For each face identity were selected five basic emotions (anger, disgust, happiness, fear, and sadness) as defined by Ekman (1992), as well as a neutral expression. All face stimuli were cropped to an oval form, to remove backgrounds, from the top of the skull to the middle of the neck. To elaborate the different emotional intensities, the neutral expression was morphed with each of the five emotions for each individual face (using Morpheus Photo Morpher 1.85, USA) to produce linear continua of morphs. Five images were extracted along each morphed face continuum (in 20% steps from 20% neutral to 100% expressive), corresponding to 20%, 40%, 60%, 80%, and 100% of maximum emotional intensity of each expression. Finally, masks were applied to each facial stimulus using Procreate software (see Figure 1A). At the end of these different steps, 208 stimuli were created with 52 images for each face identity (5 expressions × 5 intensities + a neutral face, all faces presented without and with mask).

Figure 1.
(A) Example of stimuli with different intensities of anger (from 0% neutral to 100% expressive in 20% steps) without and with a mask. (B) Timeline of stimulus presentation.

Procedure

Participants were seated in front of a computer screen (at about 50 cm) on which a custom emotion categorization task was programed. Stimuli were displayed at the center of the screen on a light gray background. For each trial, after the presentation of a fixation cross (200 ms) followed by a gray screen (600 ms), a face stimulus was displayed as well as the six response buttons indicating the different emotions (5 expressions + neutral), for a duration of 1,000 ms. After that, the face stimulus disappeared, and the response buttons were maintained until the participant's response (see Figure 1B). They were asked to decide which expression the presented face conveyed from the six listed choices by clicking on one of the buttons using the computer mouse as accurately and as fast as possible. The order of the buttons was counterbalanced across participants. They performed five blocks of trials (four blocks of 42 images and one block of 40 images) presented in a randomized order, with self-timed breaks taken in-between.

Data Analysis

Two repeated-measures analyses of variance (ANOVA) on the percentages of correct responses were performed using the Jamovi software (The Jamovi Project‌, 2020; version 2.3.). The first ANOVA was conducted on the accuracy obtained toward emotions expressed at 100% of their intensity with Emotion (disgust, fear, happiness, anger, sadness, and neutral) and Mask (with and without facemasks) as within-subject factors. The second ANOVA, including the intensity of emotions, was conducted with Emotion (disgust, fear, happiness, anger, and sadness), Mask (with and without facemasks), and Intensity (20%, 40%, 60%, 80%, and 100%) as within-subject factors. Greenhouse–Geisser corrections were applied with adjusted degrees of freedom, corresponding to the epsilon (ε) values. Effect size was estimated with partial eta-squared (η²p). The alpha significance value used was .05, and for significant effects or interactions, post hoc comparisons were performed using Tukey (p) adjustment. In addition, confusion matrices were carried out to better characterize the distribution of participants’ responses according to the expressed emotions and their intensities, without and with masks.

Results

Accuracy at 100% of Emotion Intensity of Faces Without and With Mask

For the accuracy at 100% of the emotional intensity of expressions, the ANOVA revealed significant effects of Emotion [F(5, 175) = 25.6, p < .001, η²p = .422] and Mask [F(1, 35) = 91.6, p < .001, η²p = .724) (Figure 2A) with a drop of performances from 87% of correct responses (SD = .19) without mask to 69% (SD = .34) with mask. Most interestingly, an interaction between these two factors [F(5, 175) = 28.7, p < .001, η²p = .451] revealed that the presence of a facemask significantly reduced categorization performances only for disgust (p < .001), happiness (p = .007), and sadness (p < .001) (Figure 2). Among them, disgust was the most impacted by facemasks, with a strong decrease in accuracy, from 85% without facemasks (SD = .201) to 29% (SD = .335) with facemasks.

Figure 2.
Accuracy in percent of correct responses (±standard error) for each expression expressed at 100% of their intensity. Effect of wearing a facemask was observed for disgust, happiness, and sadness but not for anger, fear, and neutral expression (p < .001).

Accuracy According to the Intensity of Emotions in Faces Without and With Mask

Concerning the accuracy obtained at different intensities of expressions (20%, 40%, 60%, 80%, and 100%), significant effects of Emotion* [F(4, 140) = 25.12, p < .001, η²p = .418], Mask [F(1, 35) = 474.30, p < .001, η²p = .931], and Intensity [F(4, 140) = 788.65, p < .001, η²p = .958] were found. A decrease in performances was observed across all emotions at different intensities, from 63% of correct responses (SD = .36) without mask to 40% (SD = .37) with mask. Moreover, independently of expression and mask-wearing, lower intensities of emotions (20%–40%) were recognized with less accuracy than higher intensities of emotions (60%–100%). Most importantly, a three-way interaction between Mask, Emotion, and Intensity factors [F(16, 560) = 4.74, ɛ = .615, p < .001, η²p = .119] was revealed (see Figure 3). In order to explain this interaction, the impact of wearing facemasks was examined according to the emotional intensity of each expression. Concerning the recognition of fearful faces, only one significant mask effect was found at 20% of their intensity (p = .012), explained by slightly better performances with a mask (8.3%) than without (0.69%). Likewise, for angry faces, an effect of facemasks was only found at 60% of their emotional intensity (p < .001), with a decrease of performances with a mask (65%) compared to its no-mask counterpart (84%). No significant difference (p > .05) was found between mask and no-mask conditions for other intensities of anger. Therefore, for fearful and angry faces, wearing a mask would have a very moderated impact on their recognition regardless of their degree of intensity.

Figure 3.
Impact of wearing a facemask according to the emotional intensity (20%, 40%, 60%, 80%, and 100%) of each expression (disgust, happiness, sadness, anger, and fear) (percent of correct responses ± standard error). Facemasks especially impair the categorization of disgust, happiness, and sadness, both for high and low degrees of their intensity (p < .05).

In contrast, concerning the recognition of sadness, the difference between mask and no-mask conditions was nonsignificant at 20% (p > .05), but this difference became significant from 40% to 100% of intensity with better performances without than with mask (all p < .001). The recognition of happiness was also strongly impacted when a facemask was worn, with a significant mask effect from 20% to 100% (all p < .2) of intensity. Furthermore, similarly to happiness, all intensities of disgust were significantly impacted by facemasks (all p < .001) (see Figure 3). Consequently, contrary to fearful and angry faces, masks would strongly impact the recognition of sadness, happiness, and disgust, both at high and low levels of their intensity.

Confusion Matrices

In order to better understand the drop in performances observed when masks were worn and the misinterpretations of certain emotions with others, we performed confusion matrices according to the type of emotions and their degree of intensity, without and with masks (see Figure 4).

Figure 4.
Confusion matrices of emotions expressed at their different degrees of intensity and the emotions perceived by the participants, without (top matrix) and with (bottom matrix) masks. The percentage of responses, up to 100% for each expressed emotion at a given intensity, is indicated and represented by a color code: the higher the responses, the darker the cell.

The correct detection of the presented emotions, regardless of their intensity, was much higher without than with masks but also, in both conditions, for the highest emotional intensities (as reflected by the darker cells in the confusion matrices). Thus, the misinterpretations increased when facial emotions were ambiguous and not obvious (low intensity) and when a mask was worn.

When faces were presented without mask, all emotions at 20% of their intensity were most frequently recognized as a neutral expression. However, from 40%, the expressed emotions were generally and progressively recognized as such, except for fear and sadness being mostly recognized from 60%. Beyond the neutral expression, sadness was also often confused with disgust from 60% and disgust with anger at all levels of intensity.

However, when the mask was worn, the misinterpretations between expressed emotions and other emotions increased. In particular, all emotions were mostly recognized as neutral up to 40% of their intensity. Beyond that, with the exception of disgust, they were gradually correctly recognized as their emotional intensity increased, although the accuracy remained lower compared to the unmasked condition. The most detrimental effect associated with wearing a facemask concerned disgust, which was, in addition to the neutral expression, essentially confused with anger at both low and high intensities. Thus, disgust was rarely recognized correctly, even at 100% of its intensity. In addition to the neutral expression, sadness was also confused with fear, disgust, and happiness, at high intensities (from 60 to 100%), leading to very poor performance in the correct detection of the expressed emotion. Unlike the other expressions confused with the neutral expression up to 60%, happiness was confused with the latter at all levels of emotional intensity. Thus, when facemasks were worn, confusions of emotions expressed with other emotions increased, especially regarding disgust, sadness, and happiness.

Discussion

The present study investigated how facemasks impair the recognition of specific emotions depending on their degree of intensity. Regardless of the type of expression, we first observed that facemasks reduce the accuracy of approximately 20% of responses (18% considering intensity at 100% and 23% from 20% to 100% of intensity), suggesting that wearing a mask would prevent optimal recognition of expressions. Our results are congruent with studies showing a decline of performances with facemasks during an emotion categorization task using forced and unnatural facial expressions (Grahlow et al., 2022; Grundmann et al., 2021; Kastendieck et al., 2022). Beyond these results, mask-induced impairments in facial emotion recognition might actually have a greater impact on facial emotion recognition in our everyday lives than in experimental studies, since we are daily confronted with a larger range of emotions of different intensities. Thus, our study aimed to better understand to which extent facemasks can impair expression recognition when emotions are conveyed along a continuum ranging from 0% to 100% of their intensity (by step of 20%). In this way, our findings revealed that the impact of wearing a mask differed depending on the type of expression and their intensity.

First, we found that the recognition of neutral expression and also of fear and anger, regardless of their level of emotional intensity, were the least impacted by facemasks. These observations suggest the use of facial cues located primarily in the upper part of the face, such as eyes and eyebrows, for example, to recognize these emotions. For the neutral expression, the lack of mask effect (see also for similar results, Bani et al., 2021; Carbon, 2020; Grenville & Dwyer, 2022) could be partially explained, as revealed by the confusion matrices, by the misinterpretation of all the emotions, in particular for those ambiguous at low intensity with the neutral expression, especially when the mask was worn (mainly at 20% without a mask and up to 40% with a mask). Moreover, the use of the “neutral” button by the participants could also underlie an “I don’t know” answer (78% of accuracy without mask and 86% of accuracy with mask).

Regarding anger, our study indicated that wearing a mask had little effect on its recognition, for both low and high intensities. However, the results are mixed in the literature. Using expressions conveyed at 100% of intensity, some studies found no effect of facemasks (Grenville & Dwyer, 2022) as well as no reduction of perceived intensity with the mask (Tsantani et al., 2022). However, others showed that the recognition of anger was impaired when a facemask was worn (Kim et al., 2022; Maiorana et al., 2022) and that this expression was sometimes confused with disgust (Carbon, 2020; Rinck et al., 2022), neutral (Kim et al., 2022), or even surprise (Kim et al., 2022). Corroborating and reconciling these heterogeneous findings, the present study showed that at a low emotional intensity, anger was confused with neutral, and that when the intensity increased, it was not confused with any other emotion, whether a mask was worn or not. These controversial results could be partially explained by the psychosocial importance of the expression of anger in recognizing and communicating a threat or a danger, which could lead individuals to use more recognition strategies such as using facial cues located in both the upper and lower parts of the face (Tsantani et al., 2022).

Concerning fear, our results, highlighting a weak impact of wearing a mask on the recognition of this expression, are consistent with previous studies (Bani et al., 2021; Carbon, 2020; Carbon et al., 2022; Grenville & Dwyer, 2022), some of which further suggest that lower facial occlusion may improve the recognition of fear and also of anger (Grenville & Dwyer, 2022). In this sense, Kim et al. (2022) found that fear was less recognized when the eyes were covered than when the mouth was covered, arguing anew in favor of the hypothesis that the eye region is more important than the mouth region in the recognition of this emotion. In contrast, one study reported that fear was one of the least recognized emotions when a facemask was worn, but this discrepancy might be due to the face stimuli used, which were photographs of actors expressing different emotions with masks worn in real life (Proverbio & Cerri, 2022). Our results also revealed that, with or without a mask, fear was mainly confused with neutral and sometimes with sadness or disgust, but only at low emotional intensities. This was in line with most research pointing out that fear was one of the most easily recognized expressions with a mask and therefore less prone to confusion (Carbon, 2020; Carbon et al., 2022). Although less frequently reported, this expression was sometimes confused with anger (Kim et al., 2022), neutral (Kim et al., 2022), or surprise (Kim et al., 2022; Rinck et al., 2022; Verroca et al., 2022), especially when a mask was worn (see also in the absence of masks for surprise: Palermo & Coltheart, 2004; Wegrzyn et al., 2017).

Unlike neutral, anger, and fear expressions, those most impacted by facemask in our study were sadness, happiness, and disgust.

First, the recognition of sadness was impacted by facemasks, both for low and high degrees of intensity (from 40% to 100%). These observations indicate that the lower part of the face is important in the recognition of this emotion, which was previously found by Smith et al. (2005) using the Bubbles* technique. Several studies have also observed a significant effect of facemasks on sadness expressed at 100% of intensity (Grenville & Dwyer, 2022; Maiorana et al., 2022). This drop in performance with the mask in our results can be explained by many confusions of sadness, at all intensities (low as high), with other emotions, such as disgust (see, for similar results, Carbon, 2020; Carbon et al., 2022; Kim et al., 2022; Rinck et al., 2022), fear (see also, Carbon et al., 2022; Rinck et al., 2022), and also neutral expression (see also Carbon, 2020; Carbon et al., 2022; Kim et al., 2022; Rinck et al., 2022).

In the same vein, happiness, according to the present study, was one of the most impacted emotions when a facemask was worn, at all levels of intensity (see also Carbon, 2020; Carbon et al., 2022; Kim et al., 2022; Shehu et al., 2022). This impact was the source of many confusions of this expression exclusively with neutral. In other studies, happiness was often confused with surprise (Verroca et al., 2022) and also with neutral (Carbon, 2020; Carbon et al., 2022; Rinck et al., 2022). These difficulties in recognizing happiness with the mask could be due to the occlusion of the mouth region, which plays an important role in the recognition of this expression (Wegrzyn et al., 2017). These observations could be complementary to other findings showing an effect of facemasks on happiness even when the presented faces express two different types of smiles (social and Duchenne smiles, Sheldon et al., 2021). Furthermore, covering the upper part of the face with sunglasses would improve the recognition of happiness, while covering the mouth region with a facemask would strongly impair the recognition of this emotion (Kim et al., 2022). However, these observations are contradictory with other studies that found no impairment in the recognition of happiness at 100% of intensity with facemasks (Lau, 2021; Maiorana et al., 2022; Shehu et al., 2022). Indeed, the importance of the orientation of eyebrows in the recognition of happiness, or the fact that it was the only positive emotions of the set, thereby making a contrast in emotional valence in comparison to other emotions, could help the recognition of happiness with a facemask (Maiorana et al., 2022) and cause a potential ceiling effect (Blazhenkova et al., 2022; Carbon, 2020). The heterogeneity of results in the literature might be explained by the use, in most studies, of exaggerated and stereotyped emotions at only 100% of their intensity and also of a few positive emotions, such as surprise (Kim et al., 2022; Rinck et al., 2022; Verroca et al., 2022). However, in the present study in which emotions were conveyed along a continuum ranging from 0 to 100% of their intensity, an effect of wearing a mask was observed for this expression at all intensities.

Even more than happiness, disgust was, across our results, the most impacted emotion when a facemask was worn, both for low and high levels of emotional intensity (see also Carbon, 2020; Carbon et al., 2022; Grahlow et al., 2022; Kim et al., 2022; Proverbio & Cerri, 2022; Rinck et al., 2022; Verroca et al., 2022). As a result, when a mask was worn, disgust was often confused with neutral but even more frequently with anger, at all levels of intensity, causing extremely low correct recognition rates even at 100% of intensity (29%). Other studies also showed that with a mask, disgust was regularly misinterpreted with anger (Carbon, 2020; Carbon et al., 2022; Kim et al., 2022; Rinck et al., 2022; Verroca et al., 2022) and to a lesser extent with happiness (Carbon et al., 2022) and sadness (Kim et al., 2022). All these observations thus underline the crucial role of the lower part of face in the recognition of this expression. Indeed, the curling of the nose and the lifting of the upper lip, which are important features in the recognition of disgust, were not available with facemasks (Verroca et al., 2022). With the absence of important diagnosis features for the recognition of disgust, the remaining facial units (eyes and eyebrows) would more likely evoke an expression of anger (Verroca et al., 2022). However, face cues located at the mouth region and also at the eye region might both be important in the recognition of disgust since Kim et al. (2022) observed a drop in performances on the recognition of this emotion when a facemask or sunglasses were worn.

Consequently, the present study, using several levels of emotional intensity of expressions, extended previous findings by highlighting that wearing facemasks disrupts the recognition of certain emotions, specifically disgust, sadness, and happiness at both low and high intensities, which would rely heavily on cues from the lower facial region. The mask, reducing the available information in the lower part of the face, might lead the participants to focus more on the upper part of the face (Barrick et al., 2021; Freud et al., 2020), which would be ineffective in recognizing these emotions. These interpretations are congruent with eye movements and gaze tracking studies showing increased focalization duration and number of fixations on the upper part of masked faces compared to unmasked faces, during tasks assessing judgments of trustworthiness and approachability (Bylianto & Chan, 2022) or during facial identity recognition tasks (Chen et al., 2023; DeBolt & Oakes, 2023; Hsiao et al., 2022). However, to our knowledge, no study using this methodology has been conducted in order to explore more objectively the changes of strategies involved in recognizing the emotions of faces wearing a mask.

Finally, the results of the literature concerning the impact of the mask on the recognition of facial emotions are often heterogeneous. Several reasons can be mentioned, particularly in terms of methodology (number, type and intensity of expressions used, and variability of tasks) and also concerning the mode of data collection (online or face-to-face). Moreover, studies could approach this issue from a more ecological point of view by addressing, for instance, more implicit processes in emotion recognition. They could also use more ecological stimuli, such as faces presented in natural environments of persons of different ethnicities, ages, and genders. In this vein, some studies used dynamic expression changes (Henke et al., 2022; Kleiser et al., 2022; Langbehn et al., 2022; Leitner et al., 2022), videos of people in two different environmental contexts (Kastendieck et al., 2022), or expressive faces combined with other types of stimuli, such as voice recordings (Aguillon-Hernandez et al., 2022; Cohn et al., 2021) or pictures of the whole body (Ross & George, 2022). Most of these studies observed an impact of facemasks on facial expression recognition (Aguillon-Hernandez et al., 2022; Cohn et al., 2021; Henke et al., 2022; Kastendieck et al., 2022; Kleiser et al., 2022; Leitner et al., 2022). Nevertheless, this impact would be less consequent when the whole body is visible (Ross & George, 2022), thus suggesting that body language might be a useful strategy to compensate the effect of facemasks. Thus, in order to better understand the impact of facemasks on the recognition of facial expressions in everyday life, future research should be based on more ecological experimental approaches.

Conclusion

The present study aimed to investigate the impact of wearing a mask on the recognition of facial expressions (anger, disgust, happiness, fear, sadness, and neutral) presented at different degrees of their emotional intensity. We found that facemasks strongly impair emotion recognition by lowering the amount of visual information in the lower part of the face. This impact mainly and significantly concerns the recognition of disgust, happiness, and sadness, both at low and high levels of their intensity, thus causing much confusion of these emotions with others for all emotional intensities. In contrast, anger, fear, and neutral expressions do not appear, or very slightly, impacted by wearing a mask. Thus, facemasks, by hiding the lower part of the face, would affect the way we perceive certain emotional expressions, which could have altered our social interactions in our daily lives. Facemasks have been used in the general population as protective measures against the COVID-19 virus and are still worn in healthcare-related places, so there is a need to better understand the impact that wearing a mask may have in our daily lives in order to consider taking adaptive measures and strategies to minimize the effect of facemasks on emotion recognition and on interpersonal relationships.

Footnotes

Acknowledgements

The authors thank Jean-Yves Baudouin for the set of morphed stimuli from the KDEF database as well as Ernesto Palmero Soler for his help during the conceptualization of the task. We also thank the editor and the two anonymous reviewers for their careful reading and constructive comments on a previous version of this paper.

Author contribution(s)

Pauline Thomas: Conceptualization; Data curation; Formal analysis; Investigation; Methodology; Visualization; Writing – original draft; Writing – review & editing.

Stéphanie Caharel: Conceptualization; Formal analysis; Investigation; Methodology; Supervision; Validation; Visualization; Writing – original draft; Writing – review & editing.

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Stéphanie Caharel

References

Adolphs

Gosselin

Buchanan

T. W.

Tranel

Schyns

Damasio

A. R.

(2005). A mechanism for impaired fear recognition after amygdala damage. Nature, 433, 68–72. https://doi.org/10.1038/nature03086

Aguillon-Hernandez

Jusiak

Latinus

Wardak

(2022). COVID-19 masks: A barrier to facial and vocal information. Frontiers in Neuroscience. https://doi.org/10.3389/fnins.2022.982899

Bani

Russo

Ardenghi

Rampoldi

Wickline

Nowicki

Strepparava

M. G.

(2021). Behind the mask: Emotion recognition in healthcare students. Medical Science Educator. https://doi.org/10.1007/s40670-021-01317-8

Barrick

E. M.

Thornton

M. A.

Tamir

D. I.

(2021). Mask exposure during COVID-19 changes emotional face processing. Plos One, 16, e0258470. https://doi.org/10.1371/journal.pone.0258470

Bassili

J. N.

(1979). Emotion recognition: The role of facial movement and the relative importance of upper and lower areas of the face. Journal of Personality and Social Psychology, 37, 2049–2058. https://doi.org/10.1037//0022-3514.37.11.2049

Baudouin

J.-Y.

Poncet

Polinori

Rekow

Damon

Leleu

Faivre

Baltenneck

(2023). Modulation of facial expression processing: An FVPS-EEG study. Emotion. https://doi.org/10.1037/emo0001223

Blais

Roy

Fiset

Arguin

Gosselin

(2012). The eyes are not the window to basic emotions. Neuropsychologia, 50, 2830–2838. https://doi.org/10.1016/j.neuropsychologia.2012.08.010

Blazhenkova

Dogerlioglu-Demir

Booth

R. W.

(2022). Masked emotions: Do face mask patterns and colors affect the recognition of emotions? Cognitive Research: Principles and Implications, 7, 33. https://doi.org/10.1186/s41235-022-00380-y

Bombari

Schmid

P. C.

Schmid Mast

Birri

Mast

F. W.

Lobmaier

J. S.

(2013). Emotion recognition: The role of featural and configural face information. Quarterly Journal of Experimental Psychology, 66, 2426–2442. https://doi.org/10.1080/17470218.2013.789065

10.

Bylianto

L. O.

Chan

K. Q.

(2022). Face masks inhibit facial cues for approachability and trustworthiness: An eyetracking study. Current Psychology (New Brunswick, N.J.), 1–12. Advance online publication. https://doi.org/10.1007/s12144-022-03705-8

11.

Carbon

C.-C.

(2020). Wearing face masks strongly confuses counterparts in reading emotions. Frontiers in Psychology, 11. https://doi.org/10.3389/fpsyg.2020.566886

12.

Carbon

C.-C.

Held

M. J.

Schütz

(2022). Reading emotions in faces with and without masks is relatively independent of extended exposure and individual difference variables. Frontiers in Psychology, 13, 856971. https://doi.org/10.3389/fpsyg.2022.856971

13.

Chen

Y.-L.

C.-Y.

S.-C.

T.-M.

S.-P.

(2023). Effect of mask coverage on face identification in Taiwanese men and women. Frontiers in Psychology, 14. https://doi.org/10.3389/fpsyg.2023.1082376

14.

Cohn

Pycha

Zellou

(2021). Intelligibility of face-masked speech depends on speaking style: Comparing casual, clear, and emotional speech. Cognition, 210, 104570. https://doi.org/10.1016/j.cognition.2020.104570

15.

DeBolt

M. C.

Oakes

L. M.

(2023). The impact of face masks on infants’ learning of faces: An eye tracking study. Infancy, 28, 71–91. https://doi.org/10.1111/infa.12516

16.

Ekman

(1992). An argument for basic emotions. Cognition and Emotion, 6, 169–200. https://doi.org/10.1080/02699939208411068

17.

Faul

Erdfelder

Lang

A. G.

Buchner

(2007). G*Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behavioral Research and Methods, 39, 175–191. https://doi.org/10.3758/BF03193146

18.

Freud

Stajduhar

Rosenbaum

R. S.

Avidan

Ganel

(2020). The COVID-19 pandemic masks the way people perceive faces. Scientific Reports, 10, 22344. https://doi.org/10.1038/s41598-020-78986-9

19.

Gosselin

Simard

(1999). Children’s knowledge of facial expressions of emotions: Distinguishing fear and surprise. Journal of Genetic Psychology, 160, 181–193. https://doi.org/10.1080/00221329909595391

20.

Grahlow

Rupp

C. I.

Derntl

(2022). The impact of face masks on emotion recognition performance and perception of threat. Plos One, 17, e0262840. https://doi.org/10.1371/journal.pone.0262840

21.

Grenville

Dwyer

D. M.

(2022). Face masks have emotion-dependent dissociable effects on accuracy and confidence in identifying facial expressions of emotion. Cognitive Research: Principles and Implications, 7, 15. https://doi.org/10.1186/s41235-022-00366-w

22.

Grundmann

Epstude

Scheibe

(2021). Face masks reduce emotion-recognition accuracy and perceived closeness. Plos One, 16, e0249792. https://doi.org/10.1371/journal.pone.0249792

23.

Guo

(2012). Holistic gaze strategy to categorize facial expression of varying intensities. Plos One, 7, e42585. https://doi.org/10.1371/journal.pone.0042585

24.

Guo

Hare

Liu

C. H.

(2022). Impact of face masks and viewers’ anxiety on ratings of first impressions from faces. Perception, 51, 37–50. https://doi.org/10.1177/03010066211065230

25.

Henke

Guseva

Wagemans

Pischedda

Haynes

J.-D.

Jahn

Anders

(2022). Surgical face masks do not impair the decoding of facial expressions of negative affect more severely in older than in younger adults. Cognitive Research: Principles and Implications, 7, 63. https://doi.org/10.1186/s41235-022-00403-8

26.

Hsiao

J. H.

Liao

Tso

R. V. Y.

(2022). Impact of mask use on face recognition: An eye-tracking study. Cognitive Research: Principles and Implications, 7, 32. https://doi.org/10.1186/s41235-022-00382-w

27.

Kastendieck

Zillmer

Hess

(2022). (Un)mask yourself! Effects of face masks on facial mimicry and emotion perception during the COVID-19 pandemic. Cognition and Emotion, 36, 59–69. https://doi.org/10.1080/02699931.2021.1950639

28.

Kesler-West

M. L.

Andersen

A. H.

Smith

C. D.

Avison

M. J.

Davis

C. E.

Kryscio

R. J.

Blonder

L. X.

(2001). Neural substrates of facial emotion processing using fMRI. Cognitive Brain Research, 11, 213–226. https://doi.org/10.1016/s0926-6410(00)00073-2

29.

Kim

Seong

S. H.

Hong

S.-S.

Choi

(2022). Impact of face masks and sunglasses on emotion recognition in South Koreans. Plos One, 17, e0263466. https://doi.org/10.1371/journal.pone.0263466

30.

Kleiser

Raffelsberger

Trenkler

Meckel

Seitz

R. J.

(2022). What influence do face masks have on reading emotions in faces? Neuroimage, Reports, 2, 100141. https://doi.org/10.1016/j.ynirp.2022.100141

31.

Kret

De Gelder

(2012). Islamic headdress influences how emotion is recognized from the eyes. Frontiers in Psychology, 3. https://doi.org/10.3389/fpsyg.2012.00110

32.

Langbehn

A. T.

Yermol

D. A.

Zhao

Thorstenson

C. A.

Niedenthal

P. M.

(2022). Wearing N95, surgical, and cloth face masks compromises the perception of emotion. Affective Science, 3, 105–117. https://doi.org/10.1007/s42761-021-00097-z

33.

Lau

W. K.

(2021). Face masks bolsters the characteristics from looking at a face even when facial expressions are impaired. Frontiers in Psychology, 12. https://doi.org/10.3389/fpsyg.2021.704916

34.

Leitner

M. C.

Meurer

Hutzler

Schuster

Hawelka

(2022). The effect of masks on the recognition of facial expressions: A true-to-life study on the perception of basic emotions. Frontiers in Psychology, 13, 933438. https://doi.org/10.3389/fpsyg.2022.933438

35.

Leppänen

Hietanen

(2004). Positive facial expressions are recognized faster than negative facial expressions, but why? Psychological Research. https://doi.org/10.1007/S00426-003-0157-2

36.

Lundqvist

Flykt

Öhman

(1998). The Karolinska directed emotional faces (KDEF). Department of Neurosciences Karolinska Hospital.

37.

Maiorana

Dini

Poletti

Tagini

Rita Reitano

Pravettoni

Priori

Ferrucci

(2022). The effect of surgical masks on the featural and configural processing of emotions. International Journal of Environmental Research and Public Health, 19, 2420. https://doi.org/10.3390/ijerph19042420

38.

Palermo

Coltheart

(2004). Photographs of facial expression: Accuracy, response times, and ratings of intensity. Behavior Research Methods, Instruments, & Computers, 36, 634–638. https://doi.org/10.3758/BF03206544

39.

Proverbio

A. M.

Cerri

(2022). The recognition of facial expressions under surgical masks: The primacy of anger. Frontiers in Neuroscience, 16, 864490. https://doi.org/10.3389/fnins.2022.864490

40.

Rinck

Primbs

M. A.

Verpaalen

I. A. M.

Bijlstra

(2022). Face masks impair facial emotion recognition and induce specific emotion confusions. Cognitive Research: Principles and Implications, 7, 83. https://doi.org/10.1186/s41235-022-00430-5

41.

Roberson

Kikutani

Döge

Whitaker

Majid

(2012). Shades of emotion: What the addition of sunglasses or masks to faces reveals about the development of facial expression processing. Cognition, 125, 195–206. https://doi.org/10.1016/j.cognition.2012.06.018

42.

Ross

George

(2022). Are face masks a problem for emotion recognition? Not when the whole body is visible. Frontiers in Neuroscience, 16. https://doi.org/10.3389/fnins.2022.915927

43.

Russell

J. A.

(1994). Is there universal recognition of emotion from facial expression? A review of the cross-cultural studies. Psychological Bulletin, 115, 102–141. https://doi.org/10.1037/0033-2909.115.1.102

44.

Shehu

H. A.

Browne

W. N.

Eisenbarth

(2022). A comparison of humans and machine learning classifiers categorizing emotion from faces with different coverings. Applied Soft Computing, 130, 109701. https://doi.org/10.1016/j.asoc.2022.109701

45.

Sheldon

K. M.

Goffredi

Corcoran

(2021). The glow still shows: Effects of facial masking on perceptions of Duchenne versus social smiles. Perception, 50, 720–727. https://doi.org/10.1177/03010066211027052

46.

Smith

M. L.

Cottrell

Gosselin

Schyns

(2005). Transmitting and decoding facial expressions. Psychological Science. https://doi.org/10.1111/j.0956-7976.2005.00801.x

47.

The Jamovi Project. (2020). Jamovi (Version 2.3) [Computer Software]. Retrieved from https://www.jamovi.org.

48.

Tsantani

Podgajecka

Gray

K. L. H.

Cook

(2022). How does the presence of a surgical face mask impair the perceived intensity of facial emotions? Plos One, 17. https://doi.org/10.1371/journal.pone.0262344

49.

Verroca

de Rienzo

C. M.

Gambarota

Sessa

(2022). Mapping the perception-space of facial expressions in the era of face masks. Frontiers in Psychology, 13, 956832. https://doi.org/10.3389/fpsyg.2022.956832

50.

Wegrzyn

Vogt

Kireclioglu

Schneider

Kissler

(2017). Mapping the emotional face. How individual face parts contribute to successful emotion recognition. Plos One, 12, e0177239. https://doi.org/10.1371/journal.pone.0177239

Do masks cover more than just a face? A study on how facemasks affect the perception of emotional expressions according to their degree of intensity

Abstract

Keywords

Methods

Participants

Stimuli

Procedure

Data Analysis

Results

Accuracy at 100% of Emotion Intensity of Faces Without and With Mask

Accuracy According to the Intensity of Emotions in Faces Without and With Mask

Confusion Matrices

Discussion

Conclusion

Footnotes

Acknowledgements

Author contribution(s)

Declaration of Conflicting Interests

Funding

ORCID iD

References