Image emotion distribution learning based on enhanced fuzzy KNN algorithm with sparse learning

Abstract

With the trend of people expressing opinions and emotions via images online, increasing attention has been paid to affective analysis of visual content. Traditional image affective analysis mainly focuses on single-label classification, but an image usually evokes multiple emotions. To this end, emotion distribution learning is proposed to describe emotions more explicitly. However, most current studies ignore the ambiguity included in emotions and the elusive correlations with complex visual features. Considering that emotions evoked by images are delivered through various visual features, and each feature in the image may have multiple emotion attributes, this paper develops a novel model that extracts multiple features and proposes an enhanced fuzzy k-nearest neighbor (EFKNN) to calculate the fuzzy emotional memberships. Specifically, the multiple visual features are converted into fuzzy emotional memberships of each feature belonging to emotion classes, which can be regarded as an intermediate representation to bridge the affective gap. Then, the fuzzy emotional memberships are fed into a fully connected neural network to learn the relationships between the fuzzy memberships and image emotion distributions. To obtain the fuzzy memberships of test images, a novel sparse learning method is introduced by learning the combination coefficients of test images and training images. Extensive experimental results on several datasets verify the superiority of our proposed approach for emotion distribution learning of images.