Contextualising Smiles: Is Perception of Smile Genuineness Influenced by Situation and Culture?

Abstract

Considerable evidence for contextual effects in emotion perception has been reported, but little is known about how contexts influence the perception of smiles, a rich source of social information. We investigated whether the perceived genuineness of a smile depends on the valence of the situation accompanying the smile, and whether such contextual effects depend on culture. Seventy-two North Americans and 83 mainland Chinese rated the genuineness of smiles displayed by Caucasians and East Asians in three situational contexts (positive, negative, and in isolation). Smiles in a negative situation were considered less genuine than the same smiles rated in isolation; this effect was observed for both groups of observers but stronger for North Americans, a finding at odds with the notion that East Asians are more likely to engage in holistic perceptual processes. Our study demonstrates contextual effects in assessment of smile genuineness, contributing new insights into the perception of affective information.

People frequently form impressions of others based on facial features (e.g., Jaeger, Wagemans, Evans, & van Beest, 2018; Thorstenson, Pazda, Elliot, & Perrett, 2017). One such feature is the smile, a salient characteristic of happy expressions for which people have perceptual expertise (Maher, Ekstrom, & Chen, 2014). The perception of smile genuineness plays a crucial role in interpersonal evaluations and decision-making. For example, if our request to a person was met with a verbal promise of compliance together with a relatively nongenuine smile, we might have reasons to believe that our request would not be entertained with as much care and promptness as what the verbal promise itself might suggest. Such a perception might urge us to look for alternatives to ensure that our request would be handled in a thoughtful and timely manner, for nongenuine smiles have been associated with unfavourable perceptions of competence, cooperativeness, and trustworthiness (e.g., Gunnery & Ruben, 2016; Johnston, Miles, & Macrae, 2010; Krumhuber, Manstead, Cosker, Marshall, & Rosin, 2009; Krumhuber et al., 2007). If we knew the person well, our judgement of his or her smile being nongenuine might encourage us to offer him or her extra social support, for nongenuine smiles have been regarded as smiles that mask negative affect (e.g., Ekman, Davidson, & Friesen, 1990; Rychlowska et al., 2014).

In everyday life, perceptions of smile genuineness mostly take place in contextualised situations; previous empirical work has also shown that such perceptions are context-dependent: Observers make use of information available to them to process smiles so that smiles displayed in a positive situation are perceived as more genuine than the same smiles displayed in an ambiguous situation (Maringer, Krumhuber, Fischer, & Niedenthal, 2011 ¹ ). However, these findings were based on smiles displayed by virtual agents, leaving open the question of whether such effects generalise to human smiles. Also, the integration of contextual information for perceiving affective stimuli such as emotional faces has been shown to differ across cultures, notably between North Americans and East Asians (e.g., Ishii, 2013; Masuda et al., 2008). Yet, it is unclear whether the contextual effects in perception of smile genuineness reported by Maringer et al. (2011) are subjected to cultural variation too. In this study, we investigate the role of situational valance and culture in the perception of human smiles.

Situation as Context

Various forms of contexts have been shown to affect the perception of facial emotions (see Barrett, Mesquita, & Gendron, 2011; Keltner, Sauter, Tracy, & Cowan, 2019). For example, when the emotion displayed in a face is congruent with the emotional content of the background visual scene, observers identify the target emotion faster and more accurately (Righart & de Gelder, 2008). Stories describing happy surprise and fearful surprise also facilitate categorical perception of surprised facial expressions (Cheal & Rutherford, 2013). Moreover, when an emotional face (e.g., a sad face) is paired with a description of a social situation suggesting a different emotion (e.g., disgust), the emotion identified by observers is consistent with the situation, not the face (Carroll & Russell, 1996). This type of context is referred to as situation or situational hereafter, following Keltner et al. (2019).

The aforementioned studies demonstrated that when processing an emotion expression, observers also attend to the social situation in which the expression is displayed. Given these studies and the close relationship between smiles and emotions (e.g., a smile is a salient feature of a happy expression; see Maher et al., 2014), similar effects of situation on perceptions of smiles can be expected. The work by Maringer et al. (2011) is the only study investigating the effects of situational contexts (i.e., descriptions of social situations; see Carroll & Russell, 1996; Keltner et al., 2019) in judgements of smile genuineness we are aware of. In their study, each smile was paired with a written description of the situation in which it was displayed, which was either positive (successfully selling a pair of shoes) or ambiguous (trying to sell a pair of shoes); smiles paired with the positive situation were rated as more genuine than the same smiles paired with the ambiguous situation. Nevertheless, the synthetic stimuli they used lacked the naturalness of actual human faces, and the ambiguous situation possibly obscured the extent of the effect, as the situation was open to interpretation due to a lack of clearly defined valence.

Culture as Context

Together with situation, culture is a type of context that shapes emotion recognition. When North Americans and Japanese were asked to rate the intensity of emotions displayed by cartoon figures, Americans only attended to the central target figure, but Japanese perceivers also paid attention to the emotion displayed by the background figures (Masuda et al., 2008). Interestingly, in another study in which North American and Japanese observers were asked to judge whether positive emotions or negative emotions were expressed in facial expressions displayed by actual humans, no cultural difference was observed (Ito, Masuda, & Hioki, 2012). This discrepancy suggests that the effect of culture might vary by stimulus type (artificial vs. naturalistic) but is surprising nonetheless, considering established cultural differences in cognition and perception: East Asians (e.g., Japanese and Chinese) are often regarded as members of high-context cultures in which much of the communication is embedded in the context, with a tendency to perceive objects and contexts as interrelated due to a holistic perceptual tendency. By contrast, North Americans (e.g., European Americans and Canadians) are regarded as members of low-context cultures in which relatively little is coded contextually, with a tendency to engage in analytic processing and perceive objects as isolated and detached from context (e.g., Hall, 1976; Ishii, 2013; Nisbett, Peng, Choi, & Norenzayan, 2001). We are not aware of any study examining the joint effects of situation and culture in perceptions of smile genuineness, but based on the literature on cultural differences in cognition and perception, we hypothesise that East Asians are more susceptible to situational influence when perceiving smiles as well.

Present Study

We set out to investigate the effects of situation and culture on the perception of human smiles. Instead of synthetic faces, we used spontaneous smiles displayed by Caucasians and East Asians as stimuli, which were presented to North American observers and Chinese observers. We also avoided the use of an ambiguous situation; observers in our study were shown smiles paired with a description of a positive situation, a description of a negative situation, or no description.

If social situation and cultural context would not influence smile perception, all smiles should be rated similarly by all observers. But based on previous works, we hypothesised that observers’ perception of smile genuineness would be influenced by the valence of the given situation. Specifically, smiles paired with a positive (negative) situation would be perceived as the most (least) genuine. We also hypothesised that such an effect would be more salient among Chinese observers than American observers.

Method

Participants

We determined our sample size based on the study by Maringer et al. (2011) and a power analysis using G*Power 3.1 (Faul, Erdfelder, Lang, & Buchner, 2007; for an effect size of f  =  .25, at α = .05, with power of .80). We stopped collecting data after 173 participants had taken part. Of those, 155 completed the whole test and passed questions probing for diligence; this final sample consisted of 83 mainland Chinese who self-identified as East Asian (38 females; M_age = 25.11 years, standard deviation [SD] = 3.56), and 72 North Americans who self-identified as Caucasian (35 females; M_age = 31.40 years, SD = 7.22). Participants provided written informed consent; this study was granted a declaration of ethical nonobjection by the Research Ethics and Data Management Committee, Tilburg School of Humanities and Digital Sciences, Tilburg University (REDC #2019/90 b).

Materials

Smiles

Recordings depicting spontaneous smiles displayed by 20 Caucasians (16 females; M_age = 23.50, SD = 2.54) were randomly selected from the UvA-NEMO smiles database (Dibeklioğlu, Salah, & Gevers, 2012). As this database contains few smiles displayed by non-Caucasians, we developed a set of comparable East Asian smile stimuli. Following the procedure adopted by Dibeklioğlu et al. (2012), we obtained spontaneous smiles by presenting short funny video clips to 20 East Asians (14 female; M_age = 23.80, SD = 2.17). Further information concerning this procedure is provided as Supplemental Material (S1).

The recordings we selected from the UvA-NEMO database vary considerably in duration (ranging from 1.91 to 11.28 seconds; M = 5.31, SD = 2.75), and while some recordings noticeably begin with a neutral expression, others do not. To standardise the stimuli, we isolated the moment at which the smile appeared to be the most intense, which resulted in a total of 40 images of spontaneous smiles (20 Caucasians and 20 East Asians). These smiles were pretested and used as stimuli in the perception test (see S2). Smiles displayed by East Asians are shown in Figure 1. ²

Figure 1.

Images of East Asians used as stimuli in the perception test. Subject numbers of Caucasian expressers whose images were used as stimuli are listed in Note 2.Note: Please refer to the online version of the article to view the figures in colour.

Situations

For determining the situations to be paired with smiles in the perception test, we developed a number of vignettes and conducted a pretest with Caucasians and East Asians (see S3). The final vignette described an individual (i.e., the expresser) being greeted by someone he or she either likes very much (positive situation) or not at all (negative situation). In the condition where smiles were presented in isolation (i.e., no situation), no vignette was provided.

Procedure

Participants learnt that they were going to see images of individuals (i.e., expressers) one at a time. Following Maringer et al. (2011), we instructed participants to rate the genuineness of smiles displayed by every expresser on a scale from 1 (not at all genuine) to 5 (very genuine). Participants were randomly assigned to one of the three conditions: positive situation (N = 49; 23 Caucasians and 26 East Asians), negative situation (N = 58; 25 Caucasians and 33 East Asians), or no situation (N = 48; 24 Caucasians and 24 East Asians). Also following Maringer et al., we ensured that the smile situation remained accessible to participants throughout the test by showing a description below every image. Images of Caucasians and East Asians were mixed in presentation in a randomised order. Finally, participants were probed for suspicion, answered questions probing for diligence, and provided demographic information.

Results

A 3 (Situation [positive, negative, none]) × 2 (Expressers [East Asian, Caucasian]) × 2 (Observers [mainland Chinese, North American]) mixed analysis of variance was conducted, with genuineness ratings as the dependent variable. Bonferroni corrections were used for pairwise comparisons. As hypothesised, a main effect of situation was found, F(2, 149) = 29.30, p < .001, η² = .28. Planned contrasts showed that smiles presented in a negative situation (M = 2.69, standard error [SE] = .066) were rated as significantly less genuine than smiles presented in isolation (M = 3.24, SE = .072), M_difference = −0.55, SE = .097, p < .001. Smiles presented in a positive situation (M = 3.39, SE = .071) were rated as significantly more genuine than smiles in a negative situation, M_difference = 0.70, SE = .097, p < .001, but were rated similarly to smiles presented in isolation, M_difference = 0.15, SE = .10, p = .44.

In addition, situation interacted with the culture of observers, F(2, 149) = 4.27, p = .016, η² = .054. Overall, American observers (M = 3.26, SE = .055) gave lower genuineness ratings than Chinese (M = 2.95, SE = .059), F(1, 149) = 14.95, p < .001, η² = .091. But as illustrated in Figure 2, this effect of culture was limited to the negative situation (M_difference = 0.61, SE = .13, p < .001); it did not emerge in the positive situation (M_difference = 0.055, SE = .14, p = .70) or when smiles were presented in isolation (M_difference = 0.27, SE = .14, p = .067). In short, a negative situation made a smile appear less genuine for American (M_difference = −0.72, SE = .14, p < .001) and Chinese (M_difference = −0.38, SE = .13, p = .017) observers but more so for the former.

Figure 2.

Interaction between situation and culture of observers on genuineness ratings. Error bars represent 95% confidence intervals. **p < . 001.

No three-way interaction was observed, F(2, 149) = 0.37, p = .69, η² = .005. Other results not pertinent to our hypotheses or focus of our study are found in S4.

Discussion

The perception of smile genuineness plays an important role in social perception and is especially interesting from a cross-cultural perspective given known cultural variance in cognition and perception. But while many studies have reported contextual effects in emotion perception (e.g., Carroll & Russell, 1996; Cheal & Rutherford, 2013; Masuda et al., 2008; Righart & de Gelder, 2008), only one has hitherto investigated the influence of situation on smile perception (Maringer et al., 2011). Even so, the study in question was a single-culture study; one of the situations investigated did not have an obvious valence either, and the stimuli lacked the naturalness of actual human smiles. In view of these limitations, we conducted a cross-cultural study to investigate the role of situation and culture in shaping perception of smiles. Specifically, we obtained Caucasian stimuli from the UvA-NEMO smiles database (Dibeklioğlu et al., 2012) and developed a set of comparable East Asian stimuli. We then presented each smile with a description of a positive situation, a negative situation, or no description to American observers and Chinese observers, who rated the genuineness of the smiles.

As expected, based on the work by Maringer et al. (2011) and the literature on contextual effects in emotion perception, we found that smiles presented in a negative situation were perceived as less genuine than smiles presented in a positive situation and in isolation. This is the first evidence we know of that shows an effect of situation in the perception of genuineness of human smiles. Contrary to our expectation, however, no statistically significant difference was observed between smiles presented in a positive situation and smiles presented in isolation, even though the difference in ratings was in the expected direction. It could be the case that when rating spontaneous smiles without additional information, observers had no reason to believe that the smiles were not genuine, much in the same way as tears are generally considered a reliable signal and not crocodile tears (Gračanin, Bylsma, & Vingerhoets, 2018). Accordingly, observers who were informed of a positive situation might have regarded such information as a confirmation of what they would have assumed anyway. Also, although Maringer et al. reported a difference in perceived smile genuineness between a positive situation and an ambiguous situation, the latter (i.e., trying to sell a pair of shoes) might not have been so ambiguous after all, for observers might have inferred such negative affect as nervousness or frustration from the description. Thus, the supposedly ambiguous situation could have been interpreted as a negative one, suggesting that the findings of their study and ours are not necessarily at odds. Gradations within each intended valence (e.g., situations that range from slightly positive or negative to extremely positive or negative) would be conducive to a more comprehensive investigation of the effect of situation.

We also expected the effect of situation to be more salient among Chinese observers than American observers. Interestingly, we found that a negative situation decreased perceived genuineness for both Americans and Chinese observers but more so for Americans. On the one hand, this suggests that the effect of situation was so pronounced that even American observers, who supposedly had a foreground focus, were susceptible. On the other hand, it goes against established cultural differences between North Americans and East Asians in cognition and perception (e.g., Hall, 1976; Ishii, 2013; Nisbett et al., 2001). If we were to regard the pairing of a smile with a negative situation as contradicting in nature (as smiles are often considered positive in valence), this finding could be attributed to the notion of naive dialecticism (Peng & Nisbett, 1999; see also Spencer-Rodgers, Williams, & Peng, 2010), which posits that East Asians display greater tolerance of contradictory information than their Western counterparts. This would suggest that compared with East Asian observers, American observers might have been more propelled to resolve the discrepancy by attenuating one source of information, which was the smile—a premise that would assume that American observers considered the situation more diagnostic than the smile itself, which we have no way of verifying with the data we collected, unfortunately. Another plausible explanation concerns the cultural orientation of observers. Cultural variances in cognition and perception are inevitably tied to differences in norms and practices prescribed by society. Although observers self-identified as Caucasian Americans and East Asian Chinese, we did not assess the extent to which they actually identified with their cultural backgrounds (which could be measured by the General Ethnicity Questionnaire; Tsai, Ying, & Lee, 2000). Even though self-reported ethnicity or geographic location is often used as a proxy for culture, measuring the actual degree of cultural orientation allows for more accurate and fine-grained cross-cultural comparisons.

Our cross-cultural interest partly arose from literature on perception of emotional faces, as we are not aware of studies examining the joint effects of situation and culture on perception of smile genuineness. Also, both cultural similarity (Ito et al., 2012) and variation (Masuda et al., 2008) in the role of context in emotion perception have been reported. These suggest that the mechanism of evaluating smiles perhaps differs from that of evaluating emotional faces, and that even within the realm of emotion perception, the joint role of situation and culture does not appear to be robust. Further investigation is necessary to determine whether, and to what degree, the influence exerted by situation varies by culture.

Much of the existing knowledge surrounding contextual effects in the perception of affect pertains to emotions or synthetic smiles. Our results show that a negative situation reduces the perceived genuineness of a human smile. We further provide evidence suggesting this effect is moderated by the culture of the observer. Altogether, our study is the first to show that people also integrate situational information when assessing the genuineness of human smiles and that the tendency to do so appears to vary across cultures.

Supplemental Material

PEC904510 Supplemental Material - Supplemental material for Contextualising Smiles: Is Perception of Smile Genuineness Influenced by Situation and Culture?

Supplemental material, PEC904510 Supplemental Material for Contextualising Smiles: Is Perception of Smile Genuineness Influenced by Situation and Culture? by Phoebe H. C. Mui, Yangfan Gan, Martijn B. Goudbeek and Marc G. J. Swerts in Perception