skip to main content
10.1145/3477495.3531988acmconferencesArticle/Chapter ViewAbstractPublication PagesirConference Proceedingsconference-collections
research-article

HIEN: Hierarchical Intention Embedding Network for Click-Through Rate Prediction

Published:07 July 2022Publication History

ABSTRACT

Click-through rate (CTR) prediction plays an important role in online advertising and recommendation systems, which aims at estimating the probability of a user clicking on a specific item. Feature interaction modeling and user interest modeling methods are two popular domains in CTR prediction, and they have been studied extensively in recent years. However, these methods still suffer from two limitations. First, traditional methods regard item attributes as ID features, while neglecting structure information and relation dependencies among attributes. Second, when mining user interests from user-item interactions, current models ignore user intents and item intents for different attributes, which lacks interpretability. Based on this observation, in this paper, we propose a novel approach Hierarchical Intention Embedding Network (HIEN), which considers dependencies of attributes based on bottom-up tree aggregation in the constructed attribute graph. HIEN also captures user intents for different item attributes as well as item intents based on our proposed hierarchical attention mechanism. Extensive experiments on both public and production datasets show that the proposed model significantly outperforms the state-of-the-art methods. In addition, HIEN can be applied as an input module to state-of-the-art CTR prediction methods, bringing further performance lift for these existing models that might already be intensively used in real systems.

References

  1. Yin-Wen Chang, Cho-Jui Hsieh, Kai-Wei Chang, Michael Ringgaard, and Chih-Jen Lin. 2010. Training and testing low-degree polynomial data mappings via linear SVM. Journal of Machine Learning Research (JMLR), Vol. 11, 4 (2010).Google ScholarGoogle Scholar
  2. Olivier Chapelle, Eren Manavoglu, and Romer Rosales. 2014. Simple and scalable response prediction for display advertising. ACM Transactions on Intelligent Systems and Technology (TIST), Vol. 5, 4 (2014), 1--34.Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. Heng-Tze Cheng, Levent Koc, Jeremiah Harmsen, Tal Shaked, Tushar Chandra, Hrishi Aradhye, Glen Anderson, Greg Corrado, Wei Chai, Mustafa Ispir, Rohan Anil, Zakaria Haque, Lichan Hong, Vihan Jain, Xiaobing Liu, and Hemal Shah. 2016. Wide & deep learning for recommender systems. In Workshop on Deep Learning for Recommender Systems (DLRS). 7--10.Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. Paul Covington, Jay Adams, and Emre Sargin. 2016. Deep neural networks for youtube recommendations. In ACM Recommender Systems Conference (RecSys). 191--198.Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. Wei Deng, Junwei Pan, Tian Zhou, Deguang Kong, Aaron Flores, and Guang Lin. 2021. DeepLight: Deep lightweight feature interactions for accelerating CTR predictions in ad serving. In ACM International Conference on Web Search and Data Mining (WSDM). 922--930.Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. John Duchi, Elad Hazan, and Yoram Singer. 2011. Adaptive subgradient methods for online learning and stochastic optimization. Journal of Machine Learning Research (JMLR), Vol. 12, 7 (2011).Google ScholarGoogle Scholar
  7. Yufei Feng, Fuyu Lv, Weichen Shen, Menghan Wang, Fei Sun, Yu Zhu, and Keping Yang. 2019. Deep session interest network for click-through rate prediction. In International Joint Conference on Artificial Intelligence (IJCAI). 2301--2307.Google ScholarGoogle ScholarCross RefCross Ref
  8. Thore Graepel, Joaquin Quinonero Candela, Thomas Borchert, and Ralf Herbrich. 2010. Web-scale bayesian click-through rate prediction for sponsored search advertising in microsoft's bing search engine. In International Conference on Machine Learning (ICML). 13--20.Google ScholarGoogle Scholar
  9. Huifeng Guo, Ruiming Tang, Yunming Ye, Zhenguo Li, and Xiuqiang He. 2017. DeepFM: a factorization-machine based neural network for CTR prediction. In International Joint Conference on Artificial Intelligence (IJCAI). 1725--1731.Google ScholarGoogle ScholarCross RefCross Ref
  10. Wei Guo, Rong Su, Renhao Tan, Huifeng Guo, Yingxue Zhang, Zhirong Liu, Ruiming Tang, and Xiuqiang He. 2021. Dual graph enhanced embedding neural network for CTR prediction. In ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD). 496--504.Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. Xiangnan He and Tat-Seng Chua. 2017. Neural factorization machines for sparse predictive analytics. In International ACM SIGIR conference on Research and Development in Information Retrieval (SIGIR). 355--364.Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Xiangnan He, Kuan Deng, Xiang Wang, Yan Li, Yong-Dong Zhang, and Meng Wang. 2020. LightGCN: Simplifying and powering graph convolution network for recommendation. In International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR). 639--648.Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. Yuchin Juan, Yong Zhuang, Wei-Sheng Chin, and Chih-Jen Lin. 2016. Field-aware factorization machines for CTR prediction. In ACM Conference on Recommender Systems (RecSys). 43--50.Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. Thomas N. Kipf and Max Welling. 2017. Semi-supervised classification with graph convolutional networks. In International Conference on Learning Representations (ICLR).Google ScholarGoogle Scholar
  15. Feng Li, Zhenrui Chen, Pengjie Wang, Yi Ren, Di Zhang, and Xiaoyu Zhu. 2019 a. Graph intention network for click-through rate prediction in sponsored search. In International ACM SIGIR conference on Research and Development in Information Retrieval (SIGIR). 961--964.Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Zekun Li, Zeyu Cui, Shu Wu, Xiaoyu Zhang, and Liang Wang. 2019 b. Fi-GNN: Modeling feature interactions via graph neural networks for CTR prediction. In ACM International Conference on Information and Knowledge Management (CIKM). 539--548.Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. Jianxun Lian, Xiaohuan Zhou, Fuzheng Zhang, Zhongxia Chen, Xing Xie, and Guangzhong Sun. 2018. xDeepFM: Combining explicit and implicit feature interactions for recommender systems. In ACM SIGKDD International Conference on Knowledge Discovery & Data Mining (KDD). 1754--1763.Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. Feng Liu, Wei Guo, Huifeng Guo, Ruiming Tang, Yunming Ye, and Xiuqiang He. 2020. Dual-attentional factorization-machines based neural network for user response prediction. In The Web Conference (WWW). 26--27.Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Wantong Lu, Yantao Yu, Yongzhe Chang, Zhen Wang, Chenhui Li, and Bo Yuan. 2020. A dual input-aware factorization machine for CTR prediction.. In International Joint Conference on Artificial Intelligence (IJCAI). 3139--3145.Google ScholarGoogle ScholarCross RefCross Ref
  20. H Brendan McMahan, Gary Holt, David Sculley, Michael Young, Dietmar Ebner, Julian Grady, Lan Nie, Todd Phillips, Eugene Davydov, Daniel Golovin, et al. 2013. Ad click prediction: a view from the trenches. In ACM SIGKDD International conference on Knowledge Discovery & Data Mining (KDD). 1222--1230.Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. Junwei Pan, Yizhi Mao, Alfonso Lobos Ruiz, Yu Sun, and Aaron Flores. 2019. Predicting different types of conversions with multi-task learning in online advertising. In ACM SIGKDD International Conference on Knowledge Discovery & Data Mining (KDD). 2689--2697.Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. Junwei Pan, Jian Xu, Alfonso Lobos Ruiz, Wenliang Zhao, Shengjun Pan, Yu Sun, and Quan Lu. 2018. Field-weighted factorization machines for click-through rate prediction in display advertising. In World Wide Web Conference (WWW). 1349--1357.Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Qi Pi, Weijie Bian, Guorui Zhou, Xiaoqiang Zhu, and Kun Gai. 2019. Practice on long sequential user behavior modeling for click-through rate prediction. In ACM SIGKDD International Conference on Knowledge Discovery & Data Mining (KDD). 2671--2679.Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. Qi Pi, Guorui Zhou, Yujing Zhang, Zhe Wang, Lejian Ren, Ying Fan, Xiaoqiang Zhu, and Kun Gai. 2020. Search-based user interest modeling with lifelong sequential behavior data for click-through rate prediction. In ACM International Conference on Information & Knowledge Management (CIKM). 2685--2692.Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. Yanru Qu, Han Cai, Kan Ren, Weinan Zhang, Yong Yu, Ying Wen, and Jun Wang. 2016. Product-based neural networks for user response prediction. In International Conference on Data Mining (ICDM). 1149--1154.Google ScholarGoogle ScholarCross RefCross Ref
  26. Kan Ren, Jiarui Qin, Yuchen Fang, Weinan Zhang, Lei Zheng, Weijie Bian, Guorui Zhou, Jian Xu, Yong Yu, Xiaoqiang Zhu, et al. 2019. Lifelong sequential modeling with personalized memorization for user response prediction. In International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR). 565--574.Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. Steffen Rendle. 2010. Factorization machines. In IEEE International Conference on Data Mining (ICDM). 995--1000.Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. Matthew Richardson, Ewa Dominowska, and Robert Ragno. 2007. Predicting clicks: estimating the click-through rate for new ads. In World Wide Web Conference (WWW). 521--530.Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. Ying Shan, T Ryan Hoens, Jian Jiao, Haijing Wang, Dong Yu, and JC Mao. 2016. Deep crossing: Web-scale modeling without manually crafted combinatorial features. In ACM SIGKDD International Conference on Knowledge Discovery & Data Mining (KDD). 255--262.Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. Si Shen, Botao Hu, Weizhu Chen, and Qiang Yang. 2012. Personalized click model through collaborative filtering. In ACM International Conference on Web Search and Data Mining (WSDM). 323--332.Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. Weiping Song, Chence Shi, Zhiping Xiao, Zhijian Duan, Yewen Xu, Ming Zhang, and Jian Tang. 2019. AutoInt: Automatic feature interaction learning via self-attentive neural networks. In ACM International Conference on Information and Knowledge Management (CIKM). 1161--1170.Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. Yang Sun, Junwei Pan, Alex Zhang, and Aaron Flores. 2021. (FM 2): Field-matrixed factorization machines for recommender systems. The Web Conference (WWW) (2021).Google ScholarGoogle Scholar
  33. Ruoxi Wang, Bin Fu, Gang Fu, and Mingliang Wang. 2017. Deep & cross network for ad click predictions. In International Workshop on Data Mining for Online Advertising (ADKDD). 1--7.Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. Xiang Wang, Xiangnan He, Yixin Cao, Meng Liu, and Tat-Seng Chua. 2019 a. KGAT: Knowledge graph attention network for recommendation. In ACM SIGKDD International Conference on Knowledge Discovery & Data Mining (KDD). 950--958.Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. Xiang Wang, Xiangnan He, Meng Wang, Fuli Feng, and Tat-Seng Chua. 2019 b. Neural graph collaborative filtering. In International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR). 165--174.Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. Xiang Wang, Tinglin Huang, Dingxian Wang, Yancheng Yuan, Zhenguang Liu, Xiangnan He, and Tat-Seng Chua. 2021. Learning Intents behind Interactions with Knowledge Graph for Recommendation. In The Web Conference (WWW). 878--887.Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. Jun Xiao, Hao Ye, Xiangnan He, Hanwang Zhang, Fei Wu, and Tat-Seng Chua. 2017. Attentional factorization machines: Learning the weight of feature interactions via attention networks. In International Joint Conference on Artificial Intelligence (IJCAI). 3119--3125.Google ScholarGoogle ScholarCross RefCross Ref
  38. Guorui Zhou, Na Mou, Ying Fan, Qi Pi, Weijie Bian, Chang Zhou, Xiaoqiang Zhu, and Kun Gai. 2019. Deep interest evolution network for click-through rate prediction. In AAAI Conference on Artificial Intelligence (AAAI), Vol. 33. 5941--5948.Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. Guorui Zhou, Xiaoqiang Zhu, Chenru Song, Ying Fan, Han Zhu, Xiao Ma, Yanghui Yan, Junqi Jin, Han Li, and Kun Gai. 2018. Deep interest network for click-through rate prediction. In ACM SIGKDD International Conference on Knowledge Discovery & Data Mining (KDD). 1059--1068.Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. HIEN: Hierarchical Intention Embedding Network for Click-Through Rate Prediction

    Recommendations

    Comments

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Sign in
    • Published in

      cover image ACM Conferences
      SIGIR '22: Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval
      July 2022
      3569 pages
      ISBN:9781450387323
      DOI:10.1145/3477495

      Copyright © 2022 ACM

      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      • Published: 7 July 2022

      Permissions

      Request permissions about this article.

      Request Permissions

      Check for updates

      Qualifiers

      • research-article

      Acceptance Rates

      Overall Acceptance Rate792of3,983submissions,20%

    PDF Format

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader