Author Reply: Aligning Social Relations With Faces,Words,and Emotions

Abstract

How do facial movements and verbal statements relate to emotional processes? A familiar answer is that the primary phenomenon is an internally located emotion that may then get expressed on the face and represented in words. In this view, emotion’s social functions and effects are indirect consequences of prior intrapsychic states or events. By contrast, my target article argued that facial and verbal activity are constituents rather than consequences of the dynamic production of fundamentally relational emotions. This article clarifies this alternative position and evaluates potential counterarguments.

How do our emotions affect other people? This question seems to imply that the process begins with an intact experiential object residing in a place that is separate from the social world; something that needs to find a way out before its impact can be felt by anyone else. But what if many emotions are already oriented to or directed at other people? What if the starting point for their operation is not somewhere in a private head or heart but rather in the configuration of changing relations between people engaging in practical and communicative actions in a shared sociomaterial environment? In that case, social effects and functions are built into the constitution of emotions rather than secondary consequences of any prior single-minded essence. In this article, I defend this alternative account.

Facial “Expression”

Facial “expression” is already a loaded term (e.g., Fridlund, 1994; Russell & Fernández-Dols, 1997). It reflects a commonly held view that internal emotional states exert outward pressure on facial muscles and leak out if not held in check (e.g., Ekman, 1972). But we all also know that faces move for other reasons. People sneeze, yawn, bite, stare, gurn, and stick their tongues out. Sometimes these movements seem involuntary and sometimes more contrived and deliberate, but the underlying process is not necessarily emotional in either case. Why then should prototypical “emotion expressions” follow different principles? What’s special about them?

Even Darwin (1872/1998) did not believe that facial movements evolved specifically to express emotions (Fridlund, 1992; Parkinson, 2017). Instead, he emphasized their practical and communicative functions. Faces that serve practical functions often contain cues about action readiness, and conveying those cues to certain conspecifics in certain circumstances clearly carries adaptive benefits (e.g., Fridlund, 1994; Frijda & Tcherkassof, 1997), leading to selective pressures for ritualization (e.g., Andrew, 1963). A glaring scowl displays focused visual attention at a person whom you are visibly prepared to push aside. If this facial movement thereby encourages the potential antagonist to back off, then both of you avoid a potentially costly altercation (Fridlund, 1994). Relatedly, van Kleef (2021) argues that,

[T]he wrinkling of the nose in disgust, which may initially have served primarily or exclusively to reduce the inflow of potentially contaminated air into the lungs, over time took on a signaling function that allowed conspecifics to avoid the apparent source [of the disgust]. (p. 92)

Despite the clear parallels between these accounts of nose wrinkles and glaring scowls, they differ in three important respects. First, van Kleef (2021) focuses on a specific stimulus-dependent behavioral response rather than a form of concern-relevant readiness for action that could be achieved by a wider variety of specific movements. Second, this nose-wrinkling response is triggered by a nonsocial object to bolster van Kleef’s claim that primarily individual states can get expressed on the face. Third, van Kleef presupposes that the emotion of “disgust” is what drove the initial stimulus rejection achieved by nose wrinkling among ancestors who had not yet attached any signal value to this facial movement. Thus, van Kleef’s account of nose wrinkling is presented as an example of the “coevolution of emotional experience and emotional expression [which] implies that emotional expressions as we know them today have traces in intrapersonal functions of emotions” (p. 92). But it is nose wrinkling and not disgust that served any original stimulus-rejecting function and thereby left its trace on the ways that faces now move in social situations. And even if that nose wrinkling were originally associated with some form of “emotional experience” (rather than simply being a reflexive response, for example), it is not clear that this would add anything extra to the evolutionary explanation.

Darwin fully recognized that his own account of the adaptive functions of facial movements provided no direct reasons for the supposed link between expressions and subjectively experienced emotions. He knew that nothing in the configural properties of glaring scowls or wrinkled noses directly maps onto the supposed phenomenal qualities of the internal emotional states that they were believed to express. So where did the connection come from? Darwin’s (1872/1998) answer was that links between emotions and faces that were once practically or communicatively functional developed over the course of natural history so that “whenever the same state of mind is induced, however feebly, there is a tendency through the force of habit and association for the same movements to be performed, though they may not then be of the least use” (p. 34). This Lamarckian account of natural selection no longer holds credence. It now seems more plausible that facial movements continue to serve practical and communicative functions and that those functions sometimes (but not always) overlap with, or contribute to, the functions of emotions themselves.

Let’s return to the example of anger. A readiness to push someone or something aside clearly does characterize many, if not all, instances of this emotion (Frijda, 1986; Parkinson, 2005). However, the specific form of attentional and physical preparation for angry actions differs depending on the nature and remoteness of the obstacle impeding physical or intellectual progress. Getting angry when reading about structural inequality or resistance to change requires different bodily adjustments from squaring up to someone who has just insulted you face-to-face. A ritualized ready-to-attack display would provide only a crude means of indicating the trajectory of action in the former kind of anger. And there seems to be little advantage in signalling threat when no appropriate addressee is available in any case (Fridlund, 1994). For all these reasons, it seems likely that glaring scowls will be seen in some but not all cases of anger, as confirmed by meta-analytic evidence from observational and experimental studies (Durán et al., 2017).

But why then is there such a strong conventional link between glaring scowls and the anger concept in Anglo-American societies? It is unlikely that a fortuitous association just happened to catch hold and spread through the population. Instead, the kind of action tendency conveyed by readiness to strike encapsulates one of the hypercognized (Levy, 1973) anger episodes in many of the societies that use anger-related concepts. This means that Anglo-Americans can indicate their emotional orientation to what is happening simply by showing the ready-to-strike face in an appropriate context. It will then be interpreted by other Anglo-Americans in accordance with a shared prototypical script (Russell & Fehr, 1994) for English-language “anger” that includes the threatened action (and any associated appraisals) as a key feature. Glaring scowls thus provide a socialized shorthand with useful pragmatic effects on other people. Many Westerners (but apparently not Trobrianders, see Crivelli et al., 2016) learn to use these facial movements as explicit and implicit means of communicating anger-related emotional meanings and thereby influencing the social situation. However, glaring scowls do not exactly “express” any experienced emotion even under these specific circumstances. Their consensual meaning often depends on where the eyes are directed and on whether they focus on a plausible object of anger. Glaring scowls are more likely to carry the implication of “I am angry with you” or “I am angry about this” than to reveal any undirected decontextualized subjective feeling.

How exactly does this account differ from the view developed in van Kleef’s (2021) response to my target article? It should be clear that I am not denying that facial movements can convey emotional information or arguing that “emotional expressions are entirely independent from emotional experience” (van Kleef, 2021, p. 92). As van Kleef rightly states, my main argument is simply “that the primary [emphasis added] function of emotional faces may not be to express emotions” (p. 91). But he is also right that my target article went further by questioning whether facial activity ever really involves the “expression” of “emotional experience” in the first place (see also Fridlund, 1994; Frijda & Tcherkassof, 1997; Gendron et al., 2018). Unlike van Kleef (2021), I do not believe that faces can provide “observers with direct access to . . . internal feelings” (p. 92). For me, the process of facial communication is not about revealing or concealing private states but instead involves orienting attention, directing action, and more generally aligning relations between people and objects located in the shared environment. I am not persuaded that the allusion to private emotional states adds anything useful to this story.

Functional Equivalence

No one would want to argue that the words associated with emotion concepts have a direct or intrinsic connection with an underlying emotion. Although verbal morphology may partly influence the likelihood of a word’s association with a concept (e.g., Köhler, 1929), nothing in its sound or visual appearance directly maps onto characteristics of the phenomenon that it represents (except perhaps in cases of onomatopoeia). Words, unlike faces, do not contain cues that indicate direction or intensity of attention, or muscular preparation for action or adjustment. They therefore cannot embody emotions in the same way that facial configurations are said to do. Despite this obvious difference, van Kleef (2021) argues that words and faces have equivalent effects on other people because both convey the same emotional meanings.

There seems to be a tension here. On one hand, van Kleef (2021) apparently believes that some authentic facial expressions have evolved specifically to signal internal feelings. On the other hand, he argues that words representing emotions operate in the same way as those faces, despite the obvious fact that their connections with emotional concepts must be largely conventional rather than natural and determinate. If the social effects and functions are the same in the two cases, why do we need a fundamentally different origin narrative for facial “expressions”? Why couldn’t faces, like words, have acquired their vernacular emotional meanings as a result of cultural rather than purely phylogenetic processes? In the former view, a facial movement that is perceived as an inauthentic expression might be equivalent to an ambivalent or hedged statement about the speaker’s emotional orientation (e.g., “it should go without saying that I love you”). Of course, people can often tell the difference between faces and sentences that unambiguously convey a direct implication and those that are equivocal or mixed, but that doesn’t imply that this depends on the fact that something natural and authentic is released in the first case but not in the second (Fridlund, 1994).

However, I do believe that there are important differences between the ways in which verbal statements and facial movements operate (as van Kleef [2021] rightly notes). First, facial activity can be dynamically co-ordinated with the unfolding course of a social episode, thus permitting reciprocal interpersonal adjustment in real time. Second, facial movements can perform social functions that require lower levels of information processing than are involved the perception of verbal meaning or the registration of verbal pragmatic force. For example, smiles can serve as rewarding stimuli (e.g., Martin et al., 2017), and gaze modulation can serve to direct other people’s visual attention in real time without any need for categorization or inference processes (e.g., Driver et al., 1999).

Despite these distinctions, I agree with van Kleef (2021) that facial and verbal communication may be treated as functionally equivalent on those particular occasions when the social influence process is mediated by the other person’s registration of emotional meaning (i.e., in the special case where a facial configuration or word is perceived as a sign of emotion by someone else who then responds in accordance with their categorization of the first person’s emotion). I only disagree that the emotional meaning that is communicated under these circumstances concerns a prior private state rather than an orientation to what is happening.

Emotions are intrinsically intentional processes and are always about something (e.g., Gordon, 1974). Knowing how things subjectively feel for someone else (if that were possible) would not provide directly useful information in the first place. Registering their stance towards something is a different matter. In this regard, it is relatively uncommon for a person to represent their emotion to someone else simply by saying “I am angry” or “I am in love.” Instead, they specify what they are angry about and with whom they are angry or in love (unless the specific object of their orientation is self-evident from the shared context). The intrinsic relationality of emotions already features in the ways that faces and voices present them to other people who respond in accordance with their relational qualities too.

“Contagion” Versus Orientational Calibration

Single-minded experiential accounts make it sound difficult for anyone to access, understand, or empathize with other people’s emotions (see Reddy, 2008). Rather than operating in a shared space where object-directed orientations can be calibrated, each individual needs to pick up cues and attach meanings to them before they can experience any corresponding affective state. Hatfield et al.’s (1994) primitive emotion contagion (PEC) account was formulated precisely to bridge this manufactured gap. PEC aligns with the emotion expression view in its assumption that facial (and bodily) configurations and movements are directly connected to emotional experiences. The twist is that this connection works in the perceiver as well as in the person whose face is perceived (cf. Lipps, 1907). We catch someone else’s emotion by first catching their expression of that emotion and then registering the internal experience that the caught expression induces (as the result of an automatic interoceptive process; e.g., Laird & Bresler, 1992).

It should be no surprise that I am skeptical about this process. If faces don’t express private emotions in the first place, they can hardly provide internal signals that then produce those same emotions in an individual who mimics them. I also don’t believe that direct motor-matching would be consistent or strong enough to support genuine interpersonal emotion transfer in any case (Parkinson, 2011). But that conclusion doesn’t rule out the possibility of continuous mutual adjustment of two or more people’s respective facial orientations to what is happening. Dynamic object-directed facial activity can still entrain and be entrained by someone else’s dynamic object-directed facial activity.

Von Scheve (2021) detects an apparent inconsistency between my rejection of the PEC account and my acknowledgement of the role of low-level adjustment processes in intragroup emotion convergence. However, the proposed implicit process of “orientational calibration” (see Parkinson, 2020) is distinct from PEC in several ways. In the former case, people are not automatically mimicking someone else’s emotion “expressions” but instead adjusting to their dynamic orientations to what is going on around them. Second, the orientational calibration process operates reciprocally between interactants who are commonly engaged in a shared activity. Third, there is no internal registration of experiential qualities implied by the relational stances that emerge during this process. It is primarily the relational orientations of group members that become mutually aligned and not their separate subjective experiences of emotion. As I suggest in the article, any feedback process operates between rather than within the involved parties and reflects the fact that every ingroup member implicitly recognizes that other ingroup members’ observed orientations match their own, producing a group-level sense of shared engagement with events that carry collective significance (e.g., Neville & Reicher, 2011).

My emphasis on this bottom-up emergent variety of intragroup emotional convergence (Parkinson, 2020) was not intended as a denial of the importance of social appraisal or the more articulated and stratified social processes to which von Scheve (2021) rightly draws attention. Indeed, throughout my article, I tried to make the point that relation-alignment processes operate in different ways and at different levels. Most real-world instances of group-level emotions probably involve combinations, concatenations, and interactions of these conceptually distinct but empirically intertwined mechanisms.

Socially Constituted Emotions

Von Scheve (2021) correctly detects a degree of equivocation on my part concerning the social ontology of emotion. This reflects my concern that debates about what emotions really are (e.g., Griffiths, 1997) often end up as unproductive definitional disputes. Like Russell (e.g., 1991), I am inclined to believe that the vernacular English-language concept of “emotion” is not amenable to precise demarcation in the first place. Refining the concept for scientific purposes might seem to make sense (e.g., Scarantino & Griffiths, 2011), but then any resulting formulation would partly depend on exactly how those purposes are specified. A tidied up concept of emotion designed to map directly onto facial configurations selected to be consistently associated with Anglo-American concepts of so-called basic emotions would look different to one that derived from extracting consistent patterns of autonomic activity (e.g., Kreibig, 2010), for example. And neither of these would be capable of accommodating the fundamentally relational aspects of many of the phenomena that are flexibly referenced by nontechnical emotion discourse.

I am happier, then, to draw conclusions about what emotions really are not. They cannot be entirely intrapsychic objects that operate in a separate subjective realm of personal experience, or else languages could find no purchase on them, leaving them fundamentally unshareable (Wittgenstein, 1953). They cannot be specified by any momentary state or signal, because their intrinsic intentionality requires attunement to over-time changes in the things that they are about. They cannot be cleaved from the objects at which they are directed, or abstracted from the relations that they align, configure, and reconfigure. In short, there is no such thing as a private, momentary, or decontextualized emotion.

But reconnecting emotions to their relational dynamic context does not stop questions about their nature or demarcation. If emotions are distributed processes (e.g., Griffiths & Scarantino, 2009), how widely and across what domains are they distributed? In my view, our understanding is enriched by pushing the shifting boundaries of the phenomenon ever outwards into material, interpersonal, intergroup, institutional, economic, and cultural worlds. At some point, a clear line may need to be drawn, but we have not got there yet.