Comparing ChatGPT and human ratings of affective images

Abstract

As ChatGPT continues to impress with its ability to generate human-like text, its capabilities in emotion recognition remain an open question. Unlike previous research comparing ChatGPT and humans on tasks with objective answers, we explored an affective domain where no correct answer exists: emotional ratings of images, a task requiring visual-perceptual analysis of complex input to recover an affective judgment. Using the MATTER database, rated on valence and arousal dimensions, I prompted ChatGPT-4 to do the same. The results revealed that ChatGPT rated images as less positive and less arousing than humans on average, particularly for images categorized as ‘mirthful,’ ‘fearful,’ and ‘disgusting.’ These findings suggest that while ChatGPT is able to process affective information, its responses reflect an analytical rather than experiential framework, differing from human interpretations.

Keywords

ChatGPT emotion valence arousal MATTER database

Get full access to this article

View all access options for this article.

References

Bubeck

Chandrasekaran

Eldan

Gehrke

Horvitz

Kamar

Lee

Y. T.

Lundberg

Nori

Palangi

Ribeiro

M. T.

Zhang

(2023). Sparks of Artificial General Intelligence: Early experiments with GPT-4 (arXiv:2303.12712). arXiv. https://doi.org/10.48550/arXiv.2303.12712

Chapman

H. A.

Anderson

A. K.

(2012). Understanding disgust. Annals of the New York Academy of Sciences, 1251, 62–76. https://doi.org/10.1111/j.1749-6632.2011.06369.x

Courtney

C. G.

Dawson

M. E.

Schell

A. M.

Iyer

Parsons

T. D.

(2010). Better than the real thing: Eliciting fear with moving and static computer-generated stimuli. International Journal of Psychophysiology, 78, 107–114. https://doi.org/10.1016/j.ijpsycho.2010.06.028

Elyoseph

Hadar-Shoval

Asraf

Lvovsky

(2023). ChatGPT outperforms humans in emotional awareness evaluations. Frontiers in Psychology, 14, 1199058. https://www.frontiersin.org/articles/10.3389/fpsyg.2023.1199058

Greene

M. R.

Oliva

(2009). Recognition of natural scenes from global properties: Seeing the forest without representing the trees. Cognitive Psychology, 58, 137–176. https://doi.org/10.1016/j.cogpsych.2008.06.001

Kramer

R. S.

(2025). Face to face: Comparing ChatGPT with human performance on face matching. Perception, 54, 65–68. https://doi.org/10.1177/03010066241295992

(2025). Evaluating mathematical problem-solving abilities of generative AI models: Performance analysis of o1-preview and gpt-4o using the Korean college scholastic ability test. IEEE Access, 13, 1227–1235. https://doi.org/10.1109/ACCESS.2024.3523703

Oliva

Torralba

(2006). Building the gist of a scene: The role of global image features in recognition. Progress in Brain Research, 155, 23–36. https://doi.org/10.1016/S0079-6123(06)55002-2

Orrù

Piarulli

Conversano

Gemignani

(2023). Human-like problem-solving abilities in large language models using ChatGPT. Frontiers in Artificial Intelligence, 6, 1199350. https://doi.org/10.3389/frai.2023.1199350

10.

Park

Kim

(2024). Exploring affective representations in emotional narratives: An exploratory study comparing ChatGPT and human responses. Cyberpsychology, Behavior, and Social Networking, 27, 736–741. https://doi.org/10.1089/cyber.2024.0100

11.

Ruiz-Padial

Pastor

M. C.

Mercado

Mata-Martín

J. L.

García-León

(2021). MATTER in emotion research: Spanish standardization of an affective image set. Behavior Research Methods, 53, 1973–1985. https://doi.org/10.3758/s13428-021-01567-9

12.

Yao

Zhang

Song

Ouyang

Wang

(2023). GPT4Vis: What can GPT-4 do for zero-shot visual recognition? arXiv preprint arXiv:2311.15732.