Why a user prefers an artwork: A deep attention model for artwork recommendation

Abstract

The combination of art market and emerging e-commerce has brought new trade opportunities and has achieved continuous growth in recent years. More and more people, especially young people, are keen to browse art information and buy artwork on the Internet. Therefore, designing an effective method of recommending artworks may not only effectively enhance the user experience in the art market but also benefit the greater transaction growth. The artwork recommendation task in e-commerce has not received much attention. Previous research works often regard the artworks as ordinary pictures and do not take into account the particularity of the artwork. To solve this problem, we modelled the aesthetic features into artwork recommendation and used the attention mechanism to learn user preferences for various features. We proposed a DAAR (Deep Attention Artwork Recommendation) model and used the attention mechanism to model the user’s preference weights for different features (including content features, aesthetic features and authors). To verify the validity of the proposed model, we collected data and conducted experiments on a real artwork community website. The experimental results show that the proposed DAAR model was better than the current state-of-the-art recommendation methods.

Keywords

Artwork recommendation attention model deep learning image aesthetic

Get full access to this article

View all access options for this article.

References

Smith

Linden

. Two decades of recommender systems at Amazon. com. IEEE Int Comput 2017; 21: 12–18.

Koren

Bell

Volinsky

Matrix factorization techniques for recommender systems. Computer 2009; 42: 30–37.

Bogdanov

Haro

Fuhrmann

et al. Semantic audio content-based music recommendation and visualization based on user preference examples. Inf Proc Manage 2013; 49: 13–33.

Guo

Zhang

Yorke-Smith

A novel recommendation model regularized with user trust and item ratings. IEEE Trans Knowled Data Eng 2016; 28: 1607–1620.

Zhang

Yuan

Han

et al. Collaborative multi-level embedding learning from reviews for rating prediction. In: IJCAI 2016, New York, 9–15 July, pp. 2986–2992. New York: AAAI Press.

Colombo-Mendoza

Valencia-García

Rodríguez-González

et al. Towards a knowledge-based probabilistic and context-aware social recommender system. J Inf Sci 2018; 44: 464–490.

LeCun

Bengio

Hinton

Deep learning. Nature 2015; 521: 436–444.

Lin

Jin

et al. Rapid: rating pictorial aesthetics using deep learning. In: Proceedings of the 22nd ACM international conference on multimedia, Orlando, FL, 3–7 November 2014, pp. 457–466. New York: ACM.

Kao

Huang

Maybank

Hierarchical aesthetic quality assessment using deep convolutional neural networks. Sig Process Image Commun 2016; 47: 500–510.

10.

Talebi

Milanfar

Nima: neural image assessment. IEEE T Image Process 2018; 27: 3998–4011.

11.

De Francisci Morales

Gionis

Lucchese

. From chatter to headlines: harnessing the real-time web for personalized news recommendation. In: Proceedings of the fifth ACM international conference on Web search and data mining, Seattle, WA, 8–12 February 2012, pp. 153–162. New York: ACM.

12.

Liu

Zhao

Social temporal collaborative ranking for context aware movie recommendation. ACM T Intel Syst Tec 2013; 4: 15.

13.

Sun

Jiang

Cheng

et al. A hybrid approach for article recommendation in research social networks. J Inf Sci 2018; 44: 696–711.

14.

Shao

Wang

et al. Music recommendation based on acoustic features and user access patterns. IEEE T Audio Speech Lang Proc 2009; 17: 1602–1611.

15.

Fang

Wang

et al. Vista: a visually, socially, and temporally-aware model for artistic recommendation. In: Proceedings of the 10th ACM conference on recommender systems, Boston, MA, 15–19 September 2016, pp. 309–316. New York: ACM.

16.

Messina

Dominguez

Parra

et al. Content-based artwork recommendation: integrating painting metadata with neural and manually-engineered visual features. User Model User-Adap 2019; 29: 251–290.

17.

Dominguez

Donoso-Guzmán

Messina

et al. Algorithmic and HCI aspects for explaining recommendations of artistic images. ACM T Interac Intell Syst 2020; 10: 1–31.

18.

Messina

Cartagena

Cerda-Mardini

et al. CuratorNet: visually-aware recommendation of art images, 2020, https://arxiv.org/abs/2009.04426.

19.

Creusen

Schoormans

JP.

The different roles of product appearance in consumer choice. J Prod Innov Manage 2005; 22: 63–81.

20.

Moshagen

Thielsch

MT.

Facets of visual aesthetics. Int J Human-Computer Stud 2010; 68: 689–709.

21.

Gelli

Uricchio

et al. Beyond the product: discovering image posts for brands in social media. In: ACM multimedia, Seoul, 22–26 October 2018, pp. 465–473. New York: ACM.

22.

Krizhevsky

Sutskever

Hinton

GE.

Imagenet classification with deep convolutional neural networks. In: Advances in neural information processing systems, 2012, pp. 1097–1105, https://papers.nips.cc/paper/2012/hash/c399862d3b9d6b76c8436e924a68c45b-Abstract.html

23.

McAuley

VBPR: visual Bayesian personalized ranking from implicit feedback. In: Thirtieth AAAI conference on artificial intelligence, Phoenix, AZ, 12–17 February 2016.

24.

Wang

Tang

et al. What your images reveal: exploiting visual contents for point-of-interest recommendation. In: Proceedings of the 26th international conference on world wide web, Perth, Australia, 3–7 April 2017, pp. 391–400. Geneva: International World Wide Web Conferences Steering Committee.

25.

Niu

Caverlee

. Neural personalized ranking for image recommendation. In: Proceedings of the eleventh ACM international conference on web search and data mining, Marina Del Rey, CA, USA, 5–9 February 2018, pp. 423–431. New York: ACM.

26.

Chen

Hong

et al. A hierarchical attention model for social contextual image recommendation. IEEE Trans Knowl Data Eng 2020; 32: 1854–1867.

27.

Xiao

et al. Attentional factorization machines: learning the weight of feature interactions via attention networks. In: Proceedings of the 26th international joint conference on artificial intelligence, Melbourne, VIC, Australia, 19–25 August 2017, pp. 3119–3125. New York: AAAI Press.

28.

Tao

Wang

et al. HoAFM: a high-order attentive factorization machine for CTR prediction. Inf Proc Manage 2020; 57: 102076.

29.

Chen

Zhang

et al. Attentive collaborative filtering: multimedia recommendation with item-and component-level attention. In: Proceedings of the 40th international ACM SIGIR conference on research and development in information retrieval, Tokyo, Japan, 7–11 August 2017, pp. 335–344. New York: ACM.

30.

Song

et al. NAIS: neural attentive item similarity model for recommendation. IEEE T Knowl Data Eng 2018; 30: 2354–2366.

31.

Cao

Miao

et al. Attentive group recommendation. In: The 41st international ACM SIGIR conference on research & development in information retrieval, Ann Arbor, MI, 8–12 July 2018, pp. 645–654. New York: ACM.

32.

Benouaret

Lenne

. Personalizing the museum experience through context-aware recommendations. In: 2015 IEEE international conference on systems, man, and cybernetics, Hong Kong, China, 9–12 October 2015, pp. 743–748. New York: IEEE.

33.

Batjargal

Maeda

et al. Artwork information embedding framework for multi-source Ukiyo-e record retrieval. In: International conference on Asian digital libraries, Kyoto, Japan, 30 November–1 December 2020, pp. 255–261. Berlin: Springer.

34.

Pires

Schlinger

Garrette

How multilingual is multilingual BERT? In: Proceedings of the 57th annual meeting of the association for computational linguistics, Florence, Italy, 28 July–2 August 2019, pp. 4996–5001.

35.

McAuley

Targett

Shi

et al. Image-based recommendations on styles and substitutes. In: Proceedings of the 38th international ACM SIGIR conference on research and development in information retrieval, Santiago, 9–13 August 2015, pp. 43–52. New York: ACM.

36.

Simonyan

Zisserman

. Very deep convolutional networks for large-scale image recognition. Arxiv Preprint Arxiv:14091556, 2014.

37.

Deng

Dong

Socher

et al. Imagenet: a large-scale hierarchical image database. In: 2009 IEEE conference on computer vision and pattern recognition, Miami, FL, 20–25 June 2009, pp. 248–255. New York: IEEE.

38.

Wang

Zhao

Wang

et al. A multi-scene deep learning model for image aesthetic evaluation. Sig Proc Image Commun 2016; 47: 511–518.

39.

Murray

Marchesotti

Perronnin

. AVA: a large-scale database for aesthetic visual analysis. In: 2012 IEEE conference on computer vision and pattern recognition, Providence, RI, 16–21 June 2012, pp. 2408–2415. New York: EEE.

40.

Nair

Hinton

GE.

Rectified linear units improve restricted Boltzmann machines. In: Proceedings of the 27th international conference on machine learning (ICML-10), Haifa, 21–24 June 2010, pp. 807–814. New York: ACM.

41.

Chua

T-S

. Neural factorization machines for sparse predictive analytics. In: Proceedings of the 40th international ACM SIGIR conference on research and development in information retrieval, Tokyo, Japan, 7–11 August 2017, pp. 355–364. New York: ACM

42.

Xue

Wang

et al. Deep item-based collaborative filtering for top-N recommendation. ACM Trans Inf Syst 2019; 37: 33.

43.

Shan

Hoens

Jiao

et al. Deep crossing: web-scale modeling without manually crafted combinatorial features. In: Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, San Francisco, CA, 13–17 August 2016, pp. 255–262. New York: ACM.

44.

Liao

Zhang

et al. Neural collaborative filtering. In: Proceedings of the 26th international conference on world wide web, Perth, WA, Australia, 3–7 April 2017, pp. 173–182. Geneva: International World Wide Web Conferences Steering Committee.

45.

Rendle

Freudenthaler

Gantner

et al. BPR: Bayesian personalized ranking from implicit feedback. In: Proceedings of the twenty-fifth conference on uncertainty in artificial intelligence, Montreal, QC, Canada, 18–21 June 2009, pp. 452–461. Arlington, VA: AUAI Press.