Motor-based theories of facial expression recognition propose that the visual perception of facial expression is aided by sensorimotor processes that are also used for the production of the same expression. Accordingly, sensorimotor and visual processes should provide congruent emotional information about a facial expression. Here, we report evidence that challenges this view. Specifically, the repeated execution of facial expressions has the opposite effect on the recognition of a subsequent facial expression than the repeated viewing of facial expressions. Moreover, the findings of the motor condition, but not of the visual condition, were correlated with a nonsensory condition in which participants imagined an emotional situation. These results can be well accounted for by the idea that facial expression recognition is not always mediated by motor processes but can also be recognized on visual information alone.
BarchiesiG.WacheS.CattaneoL. (2012). The frames of reference of the motor-visual aftereffect. PLOS ONE, 7(7), Article e40892. doi:10.1371/journal.pone.0040892
2.
BarracloughN. E.KeithR. H.XiaoD.OramM. W.PerrettD. I. (2009). Visual adaptation to goal-directed hand actions. Journal of Cognitive Neuroscience, 21, 1805–1819.
3.
BentonC. P. (2009). Effect of photographic negation on face expression aftereffects. Perception, 38, 1267–1274. doi:10.1068/p6468
4.
BlaesiS.WilsonM. (2010). The mirror reflects both ways: Action influences perception of others. Brain and Cognition, 72, 306–309. doi:10.1016/j.bandc.2009.10.001
5.
BrainardD. H. (1997). The psychophysics toolbox. Spatial Vision, 10, 433–436.
6.
BruceV.YoungA. (1986). Understanding face recognition. British Journal of Psychology, 77, 305–327. doi:10.1111/j.2044-8295.1986.tb02199.x
7.
CaggianoV.PomperJ. K.FleischerF.FogassiL.GieseM. A.ThierP. (2013). Mirror neurons in monkey area F5 do not adapt to the observation of repeated actions. Nature Communications, 4, Article 1433. doi:10.1038/ncomms2419
8.
ChangC.-C.LinC.-J. (2011). LIBSVM: A library for support vector machines. ACM Transactions on Intelligent Systems and Technology, 2(3), Article 27. doi:10.1145/1961189.1961199
9.
ChongT. T.CunningtonR.WilliamsM. A.KanwisherN. (2008). fMRI adaptation reveals mirror neurons in human inferior parietal cortex. Current Biology, 18, 1576–1580. doi:10.1016/j.cub.2008.08.068
10.
CurioC.KleinerM.BreidtM.BülthoffH. H. (2010, January). The virtual face mirror project: Revealing dynamic self-perception in humans. Poster presented at the 4th International Conference on Cognitive Systems, Zürich, Switzerland. Retrieved from http://cogsys2010.ethz.ch/doc/cogsys2010_proceedings/cogsys2010_0137.pdf
11.
DarwinC. (1872). The expression of the emotions in man and animals. London, England: J. Murray.
12.
de la RosaS.FerstlY.BülthoffH. H. (2016). Visual adaptation dominates bimodal visual-motor action adaptation. Scientific Reports, 6, Article 23829. doi:10.1038/srep23829
13.
de la RosaS.GieseM. A.BülthoffH. H.CurioC. (2013). The contribution of different cues of facial movement to the emotional facial expression adaptation aftereffect. Journal of Vision, 13(1), Article 23. doi:10.1167/13.1.23
14.
de la RosaS.GordonM.SchneiderB. A. (2009). Knowledge alters visual contrast sensitivity. Attention, Perception, & Psychophysics, 71, 451–462. doi:10.3758/APP.71.3.451
15.
de la RosaS.SchillingerF. L.BülthoffH. H.SchultzJ.UludagK. (2016). fMRI adaptation between action observation and action execution reveals cortical areas with mirror neuron properties in human BA 44/45. Frontiers in Human Neuroscience, 10, Article 78. doi:10.3389/fnhum.2016.00078
16.
DinsteinI.HassonU.RubinN.HeegerD. J. (2007). Brain areas selective for both observed and executed movements. Journal of Neurophysiology, 98, 1415–1427. doi:10.1152/jn.00238.2007
17.
EkmanP.FriesenW. (1978). Facial Action Coding System: A technique for the measurement of facial movement. Palo Alto, CA: Consulting Psychologists Press.
18.
FoxC. J.BartonJ. J. S. (2007). What is adapted in face adaptation? The neural representations of expression in the human visual system. Brain Research, 1127, 80–89. doi:10.1016/j.brainres.2006.09.104
19.
FrisbyJ. P. (1979). Seeing: Illusion, brain, and mind. Oxford, England: Oxford University Press.
20.
FristonK. (2010). The free-energy principle: A unified brain theory?Nature Reviews Neuroscience, 11, 127–138. doi:10.1038/nrn2787
21.
GieseM. A. (2016). Face recognition: Canonical mechanisms at multiple timescales. Current Biology, 26, R534–R537. doi:10.1016/j.cub.2016.05.045
22.
GoldmanA. I.SripadaC. S. (2005). Simulationist models of face-based emotion recognition. Cognition, 94, 193–213. doi:10.1016/j.cognition.2004.01.005
23.
HaxbyJ. V. (2001). Distributed and overlapping representations of faces and objects in ventral temporal cortex. Science, 293, 2425–2430. doi:10.1126/science.1063736
24.
HaxbyJ. V.HoffmanE. A.GobbiniM. I. (2000). The distributed human neural system for face perception. Trends in Cognitive Sciences, 4, 223–233. doi:10.1016/S1364-6613(00)01482-0
25.
HyniewskaS.SatoW. (2015). Facial feedback affects valence judgments of dynamic and static emotional expressions. Frontiers in Psychology, 6, Article 291. doi:10.3389/fpsyg.2015.00291
26.
IpserA.CookR. (2016). Inducing a concurrent motor load reduces categorization precision for facial expressions. Journal of Experimental Psychology: Human Perception and Performance, 42, 706–718. doi:10.1037/xhp0000177
27.
KilnerJ. M.FristonK. J.FrithC. D. (2007). Predictive coding: An account of the mirror neuron system. Cognitive Processing, 8, 159–166. doi:10.1007/s10339-007-0170-2
28.
KilnerJ. M.NealA.WeiskopfN.FristonK. J.FrithC. D. (2009). Evidence of mirror neurons in human inferior frontal gyrus. The Journal of Neuroscience, 29, 10153–10159. doi:10.1523/JNEUROSCI.2668-09.2009
29.
KleinerM.BrainardD.PelliD. G. (2007). What’s new in Psychtoolbox-3? [Abstract]. Perception, 36(Suppl.), 14.
KorbS.GrandjeanD.SchererK. R. (2010). Timing and voluntary suppression of facial mimicry to smiling faces in a go/nogo task—an EMG study. Biological Psychology, 85, 347–349. doi:10.1016/j.biopsycho.2010.07.012
32.
KrumhuberE. G.LikowskiK. U.WeyersP. (2014). Facial mimicry of spontaneous and deliberate Duchenne and non-Duchenne smiles. Journal of Nonverbal Behavior, 38, 1–11. doi:10.1007/s10919-013-0167-8
33.
LairdJ. D. (1974). Self-attribution of emotion: The effects of expressive behavior on the quality of emotional experience. Journal of Personality and Social Psychology, 29, 475–486.
LingnauA.GesierichB.CaramazzaA. (2009). Asymmetric fMRI adaptation reveals no evidence for mirror neurons in humans. Proceedings of the National Academy of Sciences, USA, 106, 9925–9930. doi:10.1073/pnas.0902262106
36.
LuoC.WangQ.SchynsP. G.KingdomF. A. A.XuH. (2015). Facial expression aftereffect revealed by adaption to emotion-invisible dynamic bubbled faces. PLOS ONE, 10(12), Article e0145877. doi:10.1371/journal.pone.0145877
37.
MaringerM.KrumhuberE. G.FischerA. H.NiedenthalP. M. (2011). Beyond smile dynamics: Mimicry and beliefs in judgments of smiles. Emotion, 11, 181–187. doi:10.1037/a0022596
38.
MukamelR.EkstromA. D.KaplanJ.IacoboniM.FriedI. (2010). Single-neuron responses in humans during execution and observation of actions. Current Biology, 20, 750–756. doi:10.1016/j.cub.2010.02.045
39.
ObermanL. M.WinkielmanP.RamachandranV. S. (2007). Face to face: Blocking facial mimicry can selectively impair recognition of emotional expressions. Social Neuroscience, 2, 167–178. doi:10.1080/17470910701391943
PelliD. G. (1997). The VideoToolbox software for visual psychophysics: Transforming numbers into movies. Spatial Vision, 10, 437–442.
43.
PonariM.ConsonM.D’AmicoN. P.GrossiD.TrojanoL. (2012). Mapping correspondence between facial mimicry and emotion recognition in healthy subjects. Emotion, 12, 1398–1403. doi:10.1037/a0028588
44.
RizzolattiG.FogassiL.GalleseV. (2001). Neurophysiological mechanisms underlying the understanding and imitation of action. Nature Reviews Neuroscience, 2, 661–670.
45.
RizzolattiG.SinigagliaC. (2010). The functional role of the parieto-frontal mirror circuit: Interpretations and misinterpretations. Nature Reviews Neuroscience, 11, 264–274. doi:10.1038/nrn2805
46.
RutherfordM. D.ChatthaH. M.KryskoK. M. (2008). The use of aftereffects in the study of relationships among emotion categories. Journal of Experimental Psychology: Human Perception and Performance, 34, 27–40. doi:10.1037/0096-1523.34.1.27
47.
RychlowskaM.CañadasE.WoodA.KrumhuberE. G.FischerA.NiedenthalP. M. (2014). Blocking mimicry makes true and false smiles look the same. PLOS ONE, 9(3), Article e90876. doi:10.1371/journal.pone.0090876
48.
SkinnerA. L.BentonC. P. (2010). Anti-expression aftereffects reveal prototype-referenced coding of facial expressions. Psychological Science, 21, 1248–1253. doi:10.1177/0956797610380702
49.
StrackF.MartinL. L.StepperS. (1988). Inhibiting and facilitating conditions of the human smile: A nonobtrusive test of the facial feedback hypothesis. Journal of Personality and Social Psychology, 54, 768–777.
50.
WebsterM. A. (2011). Adaptation and visual coding. Journal of Vision, 11(5), Article 3. doi:10.1167/11.5.3
51.
WebsterM. A. (2015). Visual adaptation. Annual Review of Vision Science, 1, 547–569. doi:10.1007/bf00535812
WebsterM. A.MacLeodD. I. A. (2011). Visual adaptation and face perception. Philosophical Transactions of the Royal Society of London, Series B: Biological Sciences, 366, 1702–1725. doi:10.1098/rstb.2010.0360
54.
WichmannF. A.HillN. J. (2001). The psychometric function: I. Fitting, sampling, and goodness of fit. Perception & Psychophysics, 63, 1293–1313.
WoodA.RychlowskaM.KorbS.NiedenthalP. (2016). Fashioning the face: Sensorimotor simulation contributes to facial expression recognition. Trends in Cognitive Sciences, 20, 227–240. doi:10.1016/j.tics.2015.12.010
57.
XuH.DayanP.LipkinR. M.QianN. (2008). Adaptation across the cortical hierarchy: Low-level curve adaptation affects high-level facial-expression judgments. The Journal of Neuroscience, 28, 3374–3383. doi:10.1523/JNEUROSCI.0182-08.2008
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
