Click-through rate (CTR) prediction, which aims to predict the probability of a user clicking on an ad, is a critical task in online advertising systems. The problem is very challenging since(1) an effective prediction relies on high-order combinatorial features, and(2)the relationship to auxiliary ads that may impact the CTR. In this paper, we propose Deep Context Interaction Network on Attention Mechanism(DCIN-Attention) to process feature interaction and context at the same time. The context includes other ads in the current search page, historically clicked and unclicked ads of the user. Specifically, we use the attention mechanism to learn the interactions between the target ad and each type of auxiliary ad. The residual network is used to model the feature interactions in the low-dimensional space, and with the multi-head self-attention neural network, high-order feature interactions can be modeled. Experimental results on Avito dataset show that DCIN outperform several existing methods for CTR prediction.
GroverA. and LeskovecJ., node2vec: Scalable Feature Learning for Networks, in: Knowledge Discovery and Data Mining (2016), 855–864.
2.
VaswaniA., ShazeerN., ParmarN., et al., Attention is all you need, Advances in Neural Information Processing Systems (2017).
3.
ZukovgregoricA., BachrachY., MinkovskyP., et al., Neural Named Entity Recognition Using a Self-Attention Mechanism, in: International conference on tools with artificial intelligence (2017), 652–656.
4.
BradleyA.P., The use of the area under the ROC curve in the evaluation of machine learning algorithms, Pattern Recognition1997, 1145–1159.
5.
PerozziB., AlrfouR., SkienaS., et al., DeepWalk: online learning of social representations, in: Knowledge discovery and data mining (2014), 701–710.
6.
SegerC., An investigation of categorical variable encoding techniques in machine learning: binary versus one-hot and feature hashing (2018).
7.
ZhouC., XuG. and LiuZ., Incentive contract design for internet referral services: cost per click vs cost per sale, Kybernetes49(02) (2019), 601–626.
8.
KingmaD.P. and BaJ., Adam: A Method for Stochastic Optimization, in: International conference on learning representations (2015).
9.
LvF., JinT., YuC., et al., SDM: Sequential Deep Matching Model for Online Large-scale Recommender System, in: Conference on information and knowledge management (2019), 2635–2643.
10.
ZhouG., ZhuX., SongC., et al., Deep Interest Network for Click-Through Rate Prediction, in: Knowledge discovery and data mining (2018), 1059–1068.
11.
GuoH., TangR., YeY., et al., DeepFM: a factorization-machine based neural network for CTR prediction, Information Retrieval (2017).
12.
LiuH., YunW., LinB. and DingY., A survey on feature learning and technologies of online advertising click-through rate estimation, Journal of Zhejiang University(Science Edition)46(05) (2019), 565–573.
13.
MinoH., UtiyamaM., SumitaE., et al., Key-value Attention Mechanism for Neural Machine Translation, in: International joint conference on natural language processing (2017), 290–295.
14.
YingH., ZhuangF., ZhangF., et al., Sequential Recommender System based on Hierarchical Attention Networks, in: International joint conference on artificial intelligence (2018), 3926–3932.
15.
ZhuH., JinJ., TanC., et al., Optimized Cost per Click in Taobao Display Advertising, in: Knowledge discovery and data mining (2017), 2191–2200.
16.
ChengH.T., KocL., HarmsenJ., et al., Wide&deep learning for recommender systems., in: Proceedings of the 1st workshop on deep learning for recommender systems, (2016), 7–10.
17.
GoodfellowI., BengioY. and CourvilleA., Deep learning, MIT Press (2016), 321–362.
18.
iResearch, Annual Monitoring Report of China’s Online Advertising Market, iResearch Consulting Series Research Report2019(06) (2019), 456–537.
19.
WangJ., HuangP., ZhaoH., et al., Billion-scale Commodity Embedding for E-commerce Recommendation in Alibaba, in: Knowledge discovery and data mining (2018), 839–848.
20.
FengL., WeiH., GuoQ., et al., Rethinking Generalized Matrix Factorization for Recommendation: The Importance of Multihot Encoding (2019).
21.
CovingtonP., AdamsJ., SarginE., et al., Deep Neural Networks for YouTube Recommendations, in: Conference on recommender systems (2016), 191–198.
22.
LuS. and ZhengX., The Original Climate, Operating Mechanism, and Evolution of Computational Advertising, Shandong Social Sciences2019(11), 164–169.
23.
DozatT., Incorporating nesterov momentum into adam, (2016).
24.
TijmenT. and HintonG., RMSprop gradient optimization, (2014).
25.
VovkV., The fundamental nature of the log loss function, Fields of Logic and Computation II (2015), 307–318.
26.
ZhangW., DuT., WangJ., et al., Deep Learning over Multi-field Categorical Data, in: European conference on information retrieval (2016), 45–57.
27.
HeX. and ChuaT.-S., Neural Factorization Machines for Sparse Predictive Analytics, Research and Development in Information Retrieval (2017).
28.
Le CunY., Le Net-5, Convolutional Neural Networks (2015).
29.
QuY., CaiH., RenK., et al., Product-Based Neural Networks for User Response Prediction, in: International conference on data mining (2016), 1149–1154.
30.
ShanY., HoensT.R., JiaoJ., et al., Deep Crossing: Web-Scale Modeling without Manually Crafted Combinatorial Features, in: Knowledge discovery and data mining, (2016), 255–262.
31.
TaiY., YangJ., LiuX., et al., Image Super-Resolution via Deep Recursive Residual Network, in: Computer vision and pattern recognition (2017), 2790–2798.
