Inferring human vision in a human-like way: Key factors influencing the cognitive processing of level-1 visual perspective-taking

Abstract

The advancement of artificial intelligence (AI) has expanded the potential for human-machine communication and collaboration in complex contexts, necessitating AI to exhibit human-like behavior in order to align with its human counterpart. Consequently, understanding human behavioral traits becomes advantageous for developing AI agents that resemble humans. This study investigated how individuals process visual information from others to inform the future design of intelligent vision systems. Through four experiments, participants were tasked with assessing whether a given number corresponds to the number of balls while manipulating the gaze direction of an avatar by averting its eyes or altering its head orientation. The results indicate that participant response times were influenced regardless of the avatar’s gaze direction. Specifically, when the avatar was positioned with its back facing the balls, any disparity in participant performance across different conditions is eliminated. These findings suggest that implicit level-1 visual perspective-taking may not primarily rely on gaze direction but rather on perceiving affordances within the environment. Such insights contribute to a deeper understanding of cognitive mechanisms underlying level-1 visual perspective-taking and can serve as a theoretical foundation for advancing AI vision algorithms in human-machine communication and collaboration.

Keywords

visual perspective-taking human-machine communication and collaboration mentalizing gaze processing

Get full access to this article

View all access options for this article.

References

Apperly

I. A.

Butterfill

S. A.

(2009). Do humans have two systems to track beliefs and belief-like states? Psychological Review, 116, 953–970. https://doi.org/10.1037/a0016923

Araujo

(2018). Living up to the chatbot hype: The influence of anthropomorphic design cues and communicative agency framing on conversational agent and company perceptions. Computers in Human Behavior, 85, 183–189. https://doi.org/10.1016/j.chb.2018.03.051

Asaoka

Takahashi

Chen

Fujiwara

Watanabe

Noro

(2019). Difficulties in spontaneously performing level 2 perspective-taking skills in children with autism spectrum disorder. Advances in Autism, 5, 243–254.

Baker

L. J.

Levin

D. T.

Saylor

M. M.

(2016). The extent of default visual perspective taking in complex layouts. Journal of Experimental Psychology: Human Perception and Performance, 42(4), 508–516. http://dx.doi.org/10.1037/xhp0000164

Bach

(2021). Laboratory website. Retrieved from https://www.actionprediction.org/perspective-taking/. Accessed August 2022.

Buck

Losow

J. I.

Murphy

M. M.

Costanzo

(1992). Social facilitation and inhibition of emotional expression and communication. Journal of Personality and Social Psychology, 63(6), 962–968. https://doi.org/10.1037/0022-3514.63.6.962

Chandra

Shirish

Srivastava

S. C.

(2022). To Be or Not to Be . . .Human? Theorizing the Role of Human-Like Competencies in Conversational Artificial Intelligence Agents. Journal of Management Information Systems, 39(4), 969–1005. https://doi.org/10.1080/07421222.2022.2127441

Chuang

M.-C.

Bala

Bernal

E. A.

Paul

Burry

(2014). Estimating Gaze Direction of Vehicle Drivers using a Smartphone Camera. In Proceedings of the IEEE/CVF international conference on computer vision (pp. 165–170).

Ciechanowski

Przegalinska

Magnuski

Gloor

(2019). In the shades of the uncanny valley: An experimental study of human–chatbot interaction. Future Generation Computer Systems, 92, 539–548. https://doi.org/10.1016/j.future.2018.01.055

10.

Coello

Quesque

Gigliotti

M.-F.

Ott

Bruyelle

J.-L.

(2018). Idiosyncratic representation of peripersonal space depends on the success of one’s own motor actions, but also the successful actions of others! PloS One, 13(5), e0196874. https://doi.org/10.1371/journal.pone.0196874

11.

Cole

G. G.

Atkinson

A. T. D.

Smith

D. T.

(2016). Do humans spontaneously take the perspective of others? Acta Psychologica, 164, 165–168. https://doi.org/10.1016/j.actpsy.2016.01.007

12.

Cole

G. G.

Millett

A. C.

Samuel

Eacott

M. J.

(2020). Perspective-Taking: In Search of a Theory. Vision (Basel, Switzerland), 4(2), 30. https://doi.org/10.3390/vision4020030

13.

Cole

G. G.

Samuel

Eacott

M. J.

(2022). A return of mental imagery: The pictorial theory of visual perspective-taking. Consciousness and Cognition, 102, 103352. https://doi.org/10.1016/j.concog.2022.103352

14.

Conway

J. R.

Lee

Ojaghi

Catmur

Bird

(2017). Submentalizing or mentalizing in a Level 1 perspective-taking task: A cloak and goggles test. Journal of Experimental Psychology: Human Perception and Performance, 43(3), 454–465. https://doi.org/10.1037/xhp0000319

15.

Chopik

W. J.

O’Brien

Konrath

S. H.

(2017). Differences in Empathic Concern and Perspective Taking Across 63 Countries. Journal of Cross-Cultural Psychology, 48(1), 23–38. https://doi.org/10.1177/0022022116673910

16.

Creem-Regehr

Gagnon

Geuss

Stefanucci

(2013). Relating spatial perspective taking to the perception of other’s affordances: Providing a foundation for predicting the future behavior of others. Frontiers in Human Neuroscience, 7. https://www.frontiersin.org/article/10.3389/fnhum.2013.00596

17.

Doi

Kanai

Tsumura

Shinohara

Kato

(2020). Lack of implicit visual perspective taking in adult males with autism spectrum disorders. Research in Developmental Disabilities, 99(11), 103593. https://doi.org/10.1016/j.ridd.2020.103593.

18.

Dolk

Hommel

Colzato

L. S.

Schütz-Bosbach

Prinz

Liepelt

(2014). The joint Simon effect: A review and theoretical integration. Frontiers in psychology, 5, 974.

19.

Driver

Davis

Ricciardelli

Kidd

Maxwell

Baron-Cohen

(1999). Gaze Perception Triggers Reflexive Visuospatial Orienting. Visual Cognition, 6(5), 509–540. https://doi.org/10.1080/135062899394920

20.

Eack

S. M.

Wojtalik

J. A.

Newhill

C. E.

Keshavan

M. S.

Phillips

M. L.

(2013). Prefrontal cortical dysfunction during visual perspective-taking in schizophrenia. Schizophrenia Research, 150(2–3), 491–497. https://doi.org/10.1016/j.schres.2013.08.022

21.

Elekes

Varga

Kiraly

(2016). Evidence for spontaneous level-2 perspective taking in adults. Consciousness and Cognition, 41, 93–103.

22.

Fan

Susilo

Low

(2021). Consistency effect in Level-1 visual perspective-taking and cue-validity effect in attentional orienting: Distinguishing the mentalising account from the submentalising account. Visual Cognition, 29(1), 22–37. https://doi.org/10.1080/13506285.2020.1857488

23.

Faul

Erdfelder

Buchner

Lang

A.-G.

(2009). Statistical power analyses using G* Power 3.1: Tests for correlation and regression analyses. Behavior Research Methods, 41(4), 1149–1160.

24.

Flavell

J. H.

Everett

B. A.

Croft

Flavell

E. R.

(1981). Young children’s knowledge about visual perception: Further evidence for the Level 1–Level 2 distinction. Developmental Psychology, 17(1), 99–103. https://doi.org/10.1037/0012-1649.17.1.99

25.

Frischen

Bayliss

A. P.

Tipper

S. P.

(2007). Gaze cueing of attention: Visual attention, social cognition, and individual differences. Psychological Bulletin, 133(4), 694–724. https://doi.org/10.1037/0033-2909.133.4.694

26.

Furlanetto

Becchio

Samson

Apperly

(2016). Altercentric interference in level 1 visual perspective taking reflects the ascription of mental states, not submentalizing. Journal of Experimental Psychology: Human Perception and Performance, 42(2), 158–163. https://doi.org/10.1037/xhp0000138

27.

Gardner

M. R.

Bileviciute

A. P.

Edmonds

C. J.

(2018). Implicit Mentalising during Level-1 Visual Perspective-Taking Indicated by Dissociation with Attention Orienting. Vision, 2(1), 3. https://doi.org/10.3390/vision2010003

28.

Gibson

J. J.

(2014). The Ecological Approach to Visual Perception: Classic Edition. Psychology Press. https://doi.org/10.4324/9781315740218

29.

Guzman

A. L.

Lewis

S. C.

(2020). Artificial intelligence and communication: A Human–Machine Communication research agenda. New Media & Society, 22(1), 70–86. https://doi.org/10.1177/1461444819858691

30.

Hamilton

A. F. de C.

Brindley

Frith

(2009). Visual perspective taking impairment in children with autistic spectrum disorder. Cognition, 113(1), 37–44. https://doi.org/10.1016/j.cognition.2009.07.007

31.

Hassanin

Khan

Tahtali

(2021). Visual Affordance and Function Understanding: A Survey. ACM Computing Surveys, 54(3), 47:1-47:35. https://doi.org/10.1145/3446370

32.

Heyes

(2014). Submentalizing: I Am Not Really Reading Your Mind. Perspectives on Psychological Science, 9(2), 131–143. https://doi.org/10.1177/1745691613518076

33.

Holland

Shin

S. M.

Phillips

(2021). Do you see what I see? A meta-analysis of the Dot Perspective Task. Proceedings of the Annual Meeting of the Cognitive Science Society, 43(43). https://escholarship.org/uc/item/7cs5r2xq

34.

Kellnhofer

Recasens

Stent

Matusik

Torralba

(2019). Gaze360: Physically unconstrained gaze estimation in the wild. In Proceedings of the IEEE/CVF international conference on computer vision (pp. 6912-6921).

35.

Kim

H. B.

Choi

Kim

Pop-Eleches

(2018). The role of education interventions in improving economic rationality. Science, 362(6410), 83–86. https://doi.org/10.1126/science.aar6987

36.

Langton

S. R. H.

(2018). I Don’t See It Your Way: The Dot Perspective Task Does Not Gauge Spontaneous Perspective Taking. Vision, 2(1), 6. https://doi.org/10.3390/vision2010006

37.

Marshall

Gollwitzer

Santos

L. R.

(2018). Does altercentric interference rely on mentalizing?: Results from two level-1 perspective-taking tasks. PLOS ONE, 13(3), e0194101. https://doi.org/10.1371/journal.pone.0194101

38.

Martin

A. K.

Perceval

Davies

Huang

Meinzer

(2019). Visual perspective taking in young and older adults. Journal of Experimental Psychology: General, 148(11), 2006–2026. https://doi.org/10.1037/xge0000584

39.

Mattan

B. D.

Rotshtein

Quinn

K. A.

(2016). Empathy and visual perspective-taking performance. Cognitive Neuroscience, 7(1–4), 170–181. https://doi.org/10.1080/17588928.2015.1085372

40.

Mazzarella

Hamilton

Trojano

Mastromauro

Conson

(2012). Observation of another’s action but not eye gaze triggers allocentric visual perspective. Quarterly Journal of Experimental Psychology, 65(12), 2447–2460. https://doi.org/10.1080/17470218.2012.697905

41.

Michelon

Zacks

J. M.

(2006). Two kinds of visual perspective taking. Perception & Psychophysics, 68(2), 327–337. https://doi.org/10.3758/BF03193680

42.

Morgan

E. J.

Freeth

Smith

D. T.

(2018). Mental State Attributions Mediate the Gaze Cueing Effect. Vision, 2(1), 11. https://doi.org/10.3390/vision2010011

43.

Nass

Moon

Fogg

B. J.

Reeves

Dryer

D. C.

(1995). Can computer personalities be human personalities? International Journal of Human-Computer Studies, 43(2), 223–239. https://doi.org/10.1006/ijhc.1995.1042

44.

Nawroth

Brett

J. M.

McElligott

A. G.

(2016). Goats display audience-dependent human-directed gazing behaviour in a problem-solving task. Biology Letters, 12(7), 20160283. https://doi.org/10.1098/rsbl.2016.0283

45.

O’Grady

Scott-Phillips

Lavelle

Smith

(2020). Perspective-taking is spontaneous but not automatic. Quarterly Journal of Experimental Psychology, 73(10), 1605–1628. https://doi.org/10.1177/1747021820942479

46.

Pavlidou

Gallagher

Lopez

Ferrè

E. R.

(2019). Let’s share our perspectives, but only if our body postures match. Cortex, 119, 575–579. https://doi.org/10.1016/j.cortex.2019.02.019

47.

Pearson

Ropar

Hamilton

(2013). A review of visual perspective taking in autism spectrum disorder. Frontiers in Human Neuroscience, 7, 652. https://doi.org/10.3389/fnhum.2013.00652

48.

Pesimena

Soranzo

(2022). Both the domain-general and the mentalising processes affect visual perspective taking. Quarterly Journal of Experimental Psychology, 17470218221094310. https://doi.org/10.1177/17470218221094310

49.

Posner

M. I.

(1980). Orienting of Attention. Quarterly Journal of Experimental Psychology, 32(1), 3–25. https://doi.org/10.1080/00335558008248231

50.

Samson

Apperly

I. A.

Kathirgamanathan

Humphreys

G. W.

(2005). Seeing it my way: A case of a selective deficit in inhibiting self-perspective. Brain, 128(5), 1102–1111. https://doi.org/10.1093/brain/awh464

51.

Samson

Apperly

Braithwaite

Andrews

Scott

(2010). Seeing It Their Way: Evidence for Rapid and Involuntary Computation of What Other People See. Journal of Experimental Psychology: Human Perception and Performance, 36, 1255–1266. https://doi.org/10.1037/a0018729

52.

Samuel

Eacott

M. J.

Cole

G. G.

(2022). Visual perspective taking without visual perspective taking. Journal of Experimental Psychology: Learning, Memory, and Cognition, 48(7), 959-965. https://doi.org/10.1037/xlm0001121

53.

Santiesteban

Banissy

M. J.

Catmur

Bird

(2015). Functional lateralization of temporoparietal junction – imitation inhibition, visual perspective-taking and theory of mind. European Journal of Neuroscience, 42(8), 2527–2533. https://doi.org/10.1111/ejn.13036

54.

Sato

Kochiyama

Uono

Yoshikawa

(2009). Commonalities in the neural mechanisms underlying automatic attentional shifts by gaze, gestures, and symbols. NeuroImage, 45(3), 984–992. https://doi.org/10.1016/j.neuroimage.2008.12.052

55.

Schurz

Aichhorn

Martin

Perner

(2013). Common brain areas engaged in false belief reasoning and visual perspective taking: A meta-analysis of functional brain imaging studies. Frontiers in Human Neuroscience, 7. https://www.frontiersin.org/article/10.3389/fnhum.2013.00712

56.

Spence

P. R.

(2019). Searching for questions, original thoughts, or advancing theory: Human-machine communication. Computers in Human Behavior, 90, 285–287. https://doi.org/10.1016/j.chb.2018.09.014

57.

Song

Dong

Feng

Liu

(2024). Influence of interpersonal distance on collaborative performance in the joint Simon task—An fNIRS-based hyperscanning study. NeuroImage, 285, 120473.

58.

Stoet

(2010). PsyToolkit: A software package for programming psychological experiments using Linux. Behavior Research Methods, 42(4), 1096–1104.

59.

Surtees

A. D. R.

Apperly

I. A.

(2012). Egocentrism and Automatic Perspective Taking in Children and Adults. Child Development, 83(2), 452–460. https://doi.org/10.1111/j.1467-8624.2011.01730.x

60.

Surtees

Samson

Apperly

(2016). Unintentional perspective-taking calculates whether something is seen, but not how it is seen. Cognition, 148, 97–105.

61.

Tucker

Ellis

(1998). On the relations between seen objects and components of potential actions. Journal of Experimental Psychology: Human Perception and Performance, 24(3), 830–846. https://doi.org/10.1037/0096-1523.24.3.830

62.

van Rooijen

Junge

C. M. M.

Kemner

. (2018). The Interplay between Gaze Following, Emotion Recognition, and Empathy across Adolescence; a Pubertal Dip in Performance? Frontiers in Psychology, 9. https://www.frontiersin.org/articles/10.3389/fpsyg.2018.00127

63.

von Salm-Hoogstraeten

Bolzius

Müsseler

. (2019). Seeing the World through the Eyes of an Avatar? Comparing Perspective Taking and Referential Coding. Journal of Experimental Psychology. Human Perception and Performance, 46. https://doi.org/10.1037/xhp0000711

64.

Ward

Ganis

Bach

(2019). Spontaneous Vicarious Perception of the Content of Another’s Visual Perspective. Current Biology, 29(5), 874-880.e4. https://doi.org/10.1016/j.cub.2019.01.046

65.

Ward

Ganis

McDonough

K. L.

Bach

(2020). Perspective taking as virtual navigation? Perceptual simulation of what others see reflects their location in space but not their gaze. Cognition, 199, 104241. https://doi.org/10.1016/j.cognition.2020.104241

66.

Wilson

Soranzo

Bertamini

(2017). Attentional interference is modulated by salience not sentience. Acta Psychologica, 178, 56–65. https://doi.org/10.1016/j.actpsy.2017.05.010

67.

Keysar

(2007). The Effect of Culture on Perspective Taking. Psychological Science, 18(7), 600–606. https://doi.org/10.1111/j.1467-9280.2007.01946.x

68.

Yaniv

Shatz

(1990). Heuristics of reasoning and analogy in children’s visual perspective taking. Child Development, 61(5), 1491–1501.

69.

Zhai

Xie

Chen

Huang

(2021). The presence of other-race people disrupts spontaneous level-2 visual perspective taking. Scandinavian Journal of Psychology, 62(5), 655–664. https://doi.org/10.1111/sjop.12751

70.

Zhang

Liu

Yuan

S.-M.

Lin

S.-F.

(2017). Eye Tracking Based Control System for Natural Human-Computer Interaction. Computational Intelligence and Neuroscience, 2017, e5739301. https://doi.org/10.1155/2017/5739301

71.

Zhou

Yang

Wang

Zhou

(2023). Spontaneous visual perspective-taking with constant attention cue: A modified dot-perspective task paradigm. Attention, Perception, & Psychophysics. https://doi.org/10.3758/s13414-023-02772-8

72.

Złotowski

Proudfoot

Yogeeswaran

Bartneck

(2015). Anthropomorphism: Opportunities and Challenges in Human–Robot Interaction. International Journal of Social Robotics, 7(3), 347–360. https://doi.org/10.1007/s12369-014-0267-6