How Can a Multimodal Approach to Primate Communication Help Us Understand the Evolution of Communication?

Abstract

Scientists studying the communication of non-human animals are often aiming to better understand the evolution of human communication, including human language. Some scientists take a phylogenetic perspective, where the goal is to trace the evolutionary history of communicative traits, while others take a functional perspective, where the goal is to understand the selection pressures underpinning specific traits. Both perspectives are necessary to fully understand the evolution of communication, but it is important to understand how the two perspectives differ and what they can and cannot tell us. Here, we suggest that integrating phylogenetic and functional questions can be fruitful in better understanding the evolution of communication. We also suggest that adopting a multimodal approach to communication might help to integrate phylogenetic and functional questions, and provide an interesting avenue for research into language evolution.

Introduction

Scientists who study non-human primate (hereon “primate”) communication, with the goal of understanding human communication, pursue two different, yet related, questions. Some scientists ask phylogenetic evolutionary questions (what was the historical pathway of a specific communicative ability?), and others ask functional evolutionary questions (what were the selection pressures that lead to evolutionary changes in this domain?). These two types of question are both necessary to fully understand the evolution of communication, and one is not necessarily more important or useful than the other. Interestingly, integration between these two foci is rare, despite the potential benefits of integration. Integration between phylogenetic and functional questions could be highly informative when considering the evolution of communication, as understanding the reasons for change could help elucidate the specific process of change, and vice versa. Here, we argue that one way to bridge the gap between phylogenetic and functional questions could be to adopt a more multimodal (MM) approach to the study of primate communication, which is usually neglected in favor of a unimodal approach.

First, we outline the general advantages of adopting a MM approach, regardless of whether the research questions are phylogenetic or functional. Second, we discuss the difference between phylogenetic and functional questions. Finally, we propose that integration between phylogenetic and functional questions would be helpful to move the field forward, and that a MM approach could be particularly useful in this endeavor.

Why Adopt a Multimodal Approach?

Primate communication is usually broken down to its constituent parts and studied as distinct, unimodal systems, often as facial expression (e.g., Parr and Waller, 2006), gesture (e.g., Hobaiter and Byrne, 2011; Liebal, Pika, and Tomasello, 2006; Pika, Liebal, and Tomasello, 2005), vocalization (e.g., Fitch and Hauser, 1995; Slocombe and Zuberbühler, 2005a) or olfaction (e.g., Heymann, 2006). Perhaps as a direct result, the study of these isolated modalities seem to have taken different research trajectories, where the study of each modality attracts different methods and theoretical assumptions (Slocombe, Waller, and Liebal, 2011). We use the term modality to refer to the different types of behavioral communication commonly isolated in the literature, but others have used modality to refer specifically to the sensory mode of the stimulus (Partan and Marler, 1999). We use a broader definition of modality for two reasons. First, some types of communicative acts can generate more than one form of information: i.e., gesture can be visual, tactile and/or auditory (e.g., Liebal, Pika, and Tomasello, 2004), and facial expressions can have auditory components generated from facial movement rather than vocalization (e.g., macaque lipsmacking: Micheletta, Engelhardt, Matthews, Agil, and Waller, 2013). Second, different mechanisms may underlie the production of different types of communication regardless of the specific sense used to detect them. Primate gestures, for example, are often proposed to involve more complex cognitive processes (in both production and comprehension) than facial expressions, despite the fact that both use the visual domain (Tomasello, 2008).

A multimodal approach may offer several advantages over the dominant, unimodal approach, regardless of whether the specific research question is phylogenetic or functional. To fully understand how and why communication systems have changed during primate evolution (and to identify what human communication has built on), we need to be accurate in our assessment of primate signals. A multimodal approach could help scientists achieve this goal in three ways:

1. Facial expression, gestures, vocalization and olfactory signals are studied in different ways in primates, using radically different theoretical approaches and different methods (Slocombe et al., 2011). Scientists think they know which modalities exhibit certain characteristics, and thus were more likely to offer stepping stones to consequent developments, but as they are studied in such different ways, it is possible that these conclusions are erroneous. One modality could exhibit characteristics that were co-opted into another at a later stage, of course, (e.g., gesture may have been a pre-cursor to spoken language), but proposing such steps are largely irrelevant if we can't make accurate comparisons between modalities. A multimodal approach is necessary to make better comparisons between modalities, and will give us a more complete picture of how these modalities operate differently (if at all).

2. Scientists often isolate a phenomenon from the holistic setting in order to understand the core properties. However, in the communication context, scientists could be making false conclusions by removing signals from the context in which they occur. Specifically, if a single signal is part of a composite signal, the meaning and characteristics of that signal could be entirely altered if it is removed from that composite signal and studied in isolation. For example, apes use a slap gesture in playful and aggressive contexts, and it seems that the facial expression accompanying the gesture allows the receiver to respond appropriately (Rijksen, 1978). When the slap is paired with a playface, it leads to play. Isolating the slap from the playface, will not, therefore, help us understand the signal better. In fact, it could lead one to incorrect characterization of this gesture. By examining the gesture alone, researchers may conclude that this signal is used flexibly across contexts. Yet if the composite signal is examined, researchers might conclude that the composite is context specific. Thus, studying signals as part of an integrated, multimodal and holistic system is essential to understand the characteristics of primate communication.

3. Combining and integrating signals has the potential to increase a signal repertoire exponentially (Partan and Marler, 1999). Therefore, communicative complexity might be less about how each single modality is used, and more about signal integration. In which case, the historical focus on single modalities might be missing an inherent feature of primate communication systems. Mapping socioecological variables onto facial, vocal, gestural, and olfactory repertoires independently may overlook important patterns. Instead, investigating how multimodality relates to variables such as group size, brain size, and social structure could be an important next step.

What is the Difference between Phylogenetic and Functional Questions?

The manner in which scientists can answer questions at different levels was originally outlined by Niko Tinbergen. In his seminal paper, Tinbergen (1963) discussed four different ways in which a question about behavior can be answered, often referred to as Tinbergen's “4 questions” or “4 whys”: causation, survival value, ontogeny, and evolution. Explanations of behavior at the level of causation and ontogeny concern the underlying physiological (or cognitive) and developmental causes, and are often termed proximate explanations, where proximate refers to the immediate, short-term mechanisms at work during the animal's lifetime (how it works). Explanations of behavior in terms of survival value concern how the behavior promotes an individual's ability to survive and reproduce (the function), and explanations in terms of evolution concern the phylogenetic history of the behavior in terms of evolutionary change. The latter explanations are often termed ultimate, as they refer to past events in contrast to immediate ones (why it works that way). Tinbergen's framework is very helpful to understand why different explanations for a behavior need not be in competition; indeed all are necessary to fully understand the manifestation of a trait in an individual. Confusion between the two types of explanations is common, however, which can result in questions being posed at one level, but being answered at another (see Scott-Phillips, Dickins, and West, 2011 for a recent review of this issue).

The distinction between the two ultimate levels of explanation is less often discussed, but understanding the distinction is nevertheless still relevant to avoid confusion. The difference between phylogenetic and functional questions bears some similarity to the theoretical distinction between comparative and evolutionary psychologists, where comparative psychologists are often interested in identifying similarities and differences between related species, and evolutionary psychologists are often interested in identifying the adaptive reasons for these differences. Scientists asking phylogenetic questions about communication study animal communication in an attempt to trace the evolutionary development of a specific communication system, often human, and often human language specifically (e.g., Arbib, Liebal, and Pika, 2008; Ouattara, Lemasson, and Zuberbühler, 2009; Tomasello and Camaioni, 1997). Scientists asking functional questions focus on what animal communication can reveal about the evolution of complex social systems in general, in an attempt to understand the fundamental principles of behavioral change, and the selection pressures underlying this change (e.g., Dobson, 2012; McComb and Semple, 2005). Such an approach can also generate hypotheses about the evolution of human communication (Dunbar, 2003), but this is not necessarily the main focus.

The focus of phylogenetic questions is usually anthropocentric, where an attempt is made to identify which aspects of human communication are species-specific and which have been inherited from or built upon established primitive primate ground plans. The aim is to search for the building blocks that specific communication systems are built on, in order to better understand the course of events over evolutionary time: “Nonhuman primates (primates) are our closest living relatives, and their behavior can be used to estimate the capacities of our extinct ancestors” (Fedurek and Slocombe, 2011, p. 153). Study species tend to be very closely related to humans, and are most often closely related primates (e.g., chimpanzees: Pan troglodytes).

A phylogenetic approach helps us understand what traits and skills were available for evolution to work with when new systems were developed. For example, it is clear that olfactory communication is less intensively used in humans compared to some other primates, reflecting the relative reduction in size of the olfactory apparatus from prosimians to humans (Martin, 1990). In contrast, numerous studies have shown that chimpanzee and human facial expressions are produced using similar muscles and neural substrates (Burrows, Waller, Parr, and Bonar, 2006; Sherwood et al., 2005; Sherwood, Holloway, Erwin, and Hof, 2004), and also processed in a similar manner to human facial expressions (Parr, Hopkins, and de Waal, 1998). Human facial expression systems are thus very unlikely to be species-specific. Instead, human facial expression must have built on an existing system of facial communication present in the shared ancestor. Indeed, similarity in facial musculature between humans and distantly related primates (Burrows and Smith, 2003) suggests that human facial expression is built on relatively archaic systems. Davila Ross and colleagues conducted a phylogenetic analysis of ape laughter vocalizations, and found that similarities mapped closely to genetic relationships between species (Ross, Owren, and Zimmermann, 2009). Thus, laughter is also unlikely to be a trait unique to humans, and instead probably developed from homologous behaviors in the shared ancestor.

Some aspects of the human language system have also been identified in extant primate communication. For example, a form of referentiality, a key feature of human language, has been demonstrated in the vocalizations of monkeys (Seyfarth and Cheney, 1990) and apes (Slocombe and Zuberbühler, 2005b). Studies of apes have also suggested that intentionality, another key feature of human language, is a characteristic of ape gestural communication (Tomasello, 2008). Whether this is equivalent to the referentiality and intentionality that abounds in human language and develops early and automatically in human ontogeny is a matter of debate. Importantly, of course, primates do not develop language spontaneously in their natural environment, and attempts to teach primates language in captivity have had far more success in comprehension than production (Savage-Rumbaugh and Lewin, 1996). Thus, any language-like skills identified in primates are only potential precursors to the human language system, and not evidence that the species has been selected to produce language. Nevertheless, such findings suggest that human language could have built on existing systems rather than evolving from scratch.

Although phylogenetic questions are focused on an ultimate, evolutionary explanation (what was the path of evolutionary change), scientists are often interested in the proximate mechanisms of primate communication in order to find out which core processes were available for other communicative systems to develop from. When the focus is explicitly cognitive (as it often is in language evolution research), scientists face the question inherent in all animal cognition research: Can the cognitive processes of other minds ever be fully exposed through scientific investigation? It is exceedingly difficult, particularly in studies of spontaneous communication, to exclude low-level interpretations of behavior and design studies that are informative regarding mental processes. In many studies even though the aim may be to increase our understanding of cognitive processes, it could be argued that we rarely succeed in addressing anything beyond surface behavior (but see Zuberbühler, Cheney, and Seyfarth, 1999 where the goal was to distinguish between mental representation and stimulus response). One approach to tackling this issue is to elucidate whether primates have the fundamental cognitive capacities to deal with different aspects of language, by trying to teach apes an artificial language system (as discussed above), and seeing which aspects can be learned and which cannot. As a whole, these studies have left us with the main message that primates do not have the capacity to grasp all aspects of human language, but that some of the key cognitive building blocks may be present (Savage-Rumbaugh and Lewin, 1996).

Scientists interested in understanding why human communication systems have evolved as they have focus on the evolutionary function of communication. Such an approach is, of course, still related to the pathway of phylogenetic change, but includes an additional question relating to the selection pressures that lead to these changes. Here, there is focus on a much wider range of species in order to elucidate general selective pressures on communication (e.g., the function of alarm calls in meerkats: Manser, Seyfarth, and Cheney, 2002). Examples of convergent evolution are also helpful in order to identify selection pressures. Domestic dogs, for example, have emerged as an interesting model for the evolution of human-like communicative skills (e.g., Kaminski, Call, and Fischer, 2004). Identifying the function of a behavioral trait, however, is not straightforward. Even if current fitness consequences are found (i.e., a behavior leads to greater survival or reproduction), it does not always follow that this was why the behavior evolved in the first place (Gould and Lewontin, 1979). Phylogenetic inertia may cause traits and behaviors to be present in a species when it serves no current function (fitness neutral), or serves a different function (exaptation), but it has nevertheless been inherited from an ancestral species where the behavior was originally selected due to bestowing a fitness advantage. Comparative modeling approaches often strive to factor out the effects of phylogenetic inertia in order to identify the real selective pressures (Shultz, Opie, and Atkinson, 2011).

Analyzing the social context and determining the social function of communicative signals is often crucial to identify any potential fitness consequences of performing the specific signal (e.g., what is the advantage of using this signal?). In some cases, the advantage to sender and receiver is clearer than others. Predator alarm vocalizations, for example, presumably help the listener avoid predators, and could benefit the sender in terms of kin selection or cooperative defense. The specific advantage of many social signals, however, can be less easy to predict and/or measure. One approach to solving this problem is to quantify immediate or short term social effects of a signal (e.g., an increase in affiliative social contact) and extrapolate from this to infer social function (see Waller and Dunbar, 2005). It can also be helpful to identify the socioecological variables associated with high levels of a communicative behavior, and thus factors which could have acted as potential drivers for the evolution of the system. Comparative and modeling approaches are often employed to identify the relationships between different social and ecological factors, across many species. Dobson (2009) found a positive relationship between social group size and facial mobility in primates, suggesting that social group size has driven the evolution of facial expression. Similarly, McComb and Semple (2005) found that increases in primate vocal repertoire size were associated with increases in group size and time spent grooming. Both studies suggest that at least some communicative signals have coevolved with social bonding, and have functioned to aid social cohesion. Such an interpretation fits well with the theory that language evolved as an efficient alternative to grooming when in a large social group (Dunbar, 1996). In sum, evidence points toward a relationship between social complexity and communicative complexity, with communication evolving as a result of increased social complexity (Freeberg, Dunbar, and Ord, 2012).

How Will a Multimodal Approach Help Us Understand the Evolution of Communication?

Integration between phylogenetic and functional questions is not at all common. Consideration of findings at the two levels could, however, be useful in better understanding the course of events that have led to the evolution of specific aspects of communication. The evolution of the human smile, for example, proposed as homologous to the primate bared teeth display (Bolwig, 1964; Hooff, 1972), is difficult to understand unless phylogenetic and functional questions are considered simultaneously. Based on FACS analysis (Ekman, Friesen, and Hager, 2002; Vick, Waller, Parr, Pasqualini, and Bard, 2007), the two displays are similar morphologically, but not identical. The human smile is formed of Action Units 6+12+25, whereas a typical primate bared-teeth display is formed of Action Units 10+12+16+25 (Parr, Waller, Vick, and Bard, 2007; Vick, et al., 2007). The phylogenetic question is whether these can be truly homologous displays if they have different muscular components. Consideration of functional questions, however, can be extremely helpful in better understanding the answer to this phylogenetic question.

Preuschoft and van Hooff (1995) proposed the Power Asymmetry Hypothesis of Motivational Emancipation to explain the pattern of facial expression across primate species. Species with strict, linear dominance hierarchies use facial expressions (such as the bared teeth display) in narrow contexts, and asymmetrically (from sub-ordinates to dominants). In contrast, species with more relaxed dominance styles use the same facial expressions flexibly in broader contexts. Regardless of the immediate social context, the social outcome of the bared teeth display seems to be to reduce aggression and/or increase affinitive contact (Bout and Thierry, 2005; Flack and de Waal, 2007; Preuschoft, 1992; Waller and Dunbar, 2005). Likewise, another facial expression (the play face, proposed as a homologue of human laughter face: van Hooff, 1972) is similarly emancipated from narrow usage in play when a species is less constrained by dominance hierarchies. Such species sometimes exhibit a facial expression which seems a blended, or converged, display between the bared teeth and playface (Thierry, Demaria, Preuschoft, and Desportes, 1989). As humans are characterized by relatively relaxed dominance, the human smile may similarly represent convergence between the primate bared teeth display and the playface. In which case, we might not expect the human smile to be physically identical to the bared teeth display in other primates, but to also bear some similarity to the primate playface (Action Unit 12+25+26: Parr et al., 2007). Note that the primate playface does sometimes include upper teeth exposure, which could also result from convergence of two displays (Waller and Cherry, 2012).

Another reason to believe that the human smile represents convergence between the two displays is that the adaptive functions of the bared teeth display and playface (in primates), as well as smiling and laughter (in humans), all seem to be broadly similar. Proximate mechanisms (cognitive, physiological, developmental bases) may differ, but all have been argued to have some sort of social bonding function (Dunbar, 2012; Mehu, Grammer, and Dunbar, 2007; Waller and Dunbar, 2005). It is possible that occupying the same functional niche caused convergence to occur. In sum, it is only through consideration of function in humans and other species that we can truly understand how the human smile has become manifest in human social interaction, and how it is rooted in ancestral display. If the primate gestures and vocalizations that are often proposed as precursors to human language (e.g., Arbib et al., 2008; Cartmill and Byrne, 2007; Slocombe and Zuberbühler, 2007) were similarly considered in terms of function, it could become clearer which (if any) was a more likely precursor to human language.

Such integration of phylogenetic and functional questions when considering the evolution of communication is rare, however. A multimodal approach may help bridge this gap. First, an understanding of communication in its true, holistic form may reveal adaptive function when it is not clear from the component parts. Second, and perhaps more importantly, multimodality may itself have been an important precursor to more complex forms of communication, such as language, as it may support the simultaneous transmission of both emotional and cognitive information through different channels. Jablonka, Ginsburg, and Dor (2012) argue that human language co-evolved with human social emotions as part of a complex gene-culture co-evolutionary framework. Crucial to this argument is that language built on the socio-communicative skills used in cooperative contexts (Tomasello, 2008), as collaborative social practice was necessary for the development of instructive communication, such as during tool making and alloparenting. The behaviors that primates use to facilitate cooperation and social bonding are not usually those that appear particularly cognitively based, such as referential vocal signals or intentional gestures, but instead are those considered to be more emotionally driven, such as laughter and facial expression (Dunbar, 2012). In which case, any consideration of what might have been the precursors to language in this scenario would benefit from consideration of primate communication as a multimodal system. Multimodal communication may require incorporation of different emotional and cognitive systems (e.g., combining “emotional” facial expressions with “cognitive” gestures). In sum, such integration may have been a potentially important precursor to human language (which answers a phylogenetic question) precisely because integration had some advantage (which answers a functional question). Such speculations could fuel interesting avenues for future research.