skip to main content
research-article

Modeling Embedding Dimension Correlations via Convolutional Neural Collaborative Filtering

Authors Info & Claims
Published:13 September 2019Publication History
Skip Abstract Section

Abstract

As the core of recommender systems, collaborative filtering (CF) models the affinity between a user and an item from historical user-item interactions, such as clicks, purchases, and so on. Benefiting from the strong representation power, neural networks have recently revolutionized the recommendation research, setting up a new standard for CF. However, existing neural recommender models do not explicitly consider the correlations among embedding dimensions, making them less effective in modeling the interaction function between users and items. In this work, we emphasize on modeling the correlations among embedding dimensions in neural networks to pursue higher effectiveness for CF. We propose a novel and general neural collaborative filtering framework—namely, ConvNCF, which is featured with two designs: (1) applying outer product on user embedding and item embedding to explicitly model the pairwise correlations between embedding dimensions, and (2) employing convolutional neural network above the outer product to learn the high-order correlations among embedding dimensions. To justify our proposal, we present three instantiations of ConvNCF by using different inputs to represent a user and conduct experiments on two real-world datasets. Extensive results verify the utility of modeling embedding dimension correlations with ConvNCF, which outperforms several competitive CF methods.

References

  1. Ting Bai, Ji-Rong Wen, Jun Zhang, and Wayne Xin Zhao. 2017. A neural collaborative filtering model with interaction-based neighborhood. In Proceedings of the CIKM. 1979--1982. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. Immanuel Bayer, Xiangnan He, Bhargav Kanagal, and Steffen Rendle. 2017. A generic coordinate descent framework for learning from implicit feedback. In Proceedings of the WWW. 1341--1350. Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. James Bennett, Stan Lanning et al. 2007. The Netflix prize. In Proceedings of the KDD Cup and Workshop, Vol. 2007. New York, NY, 35.Google ScholarGoogle Scholar
  4. Alex Beutel, Paul Covington, Sagar Jain, Can Xu, Jia Li, Vince Gatto, and Ed H. Chi. 2018. Latent cross: Making use of context in recurrent recommender systems. In Proceedings of the WSDM. 46--54. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. Da Cao, Xiangnan He, Liqiang Nie, Xiaochi Wei, Xia Hu, Shunxiang Wu, and Tat-Seng Chua. 2017. Cross-platform app recommendation by jointly modeling ratings and texts. ACM Trans. Inform. Syst. 35, 4 (July 2017), 37:1--37:27. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Da Cao, Liqiang Nie, Xiangnan He, Xiaochi Wei, Shunzhi Zhu, and Tat-Seng Chua. 2017. Embedding factorization models for jointly recommending items and user generated lists. In Proceedings of the SIGIR. ACM, 585--594. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. Jingyuan Chen, Hanwang Zhang, Xiangnan He, Liqiang Nie, Wei Liu, and Tat-Seng Chua. 2017. Attentive collaborative filtering: Multimedia recommendation with item- and component-level attention. In Proceedings of the SIGIR. 335--344. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Tianqi Chen, Weinan Zhang, Qiuxia Lu, Kailong Chen, Zhao Zheng, and Yong Yu. 2012. SVDFeature: A toolkit for feature-based collaborative filtering. J. Machine Learn. Res. 13, Dec. (2012), 3619--3622. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Xu Chen, Yongfeng Zhang, Hongteng Xu, Zheng Qin, and Hongyuan Zha. 2018. Adversarial distillation for efficient recommendation with external knowledge. ACM Trans. Inform. Syst. 37, 1 (2018), 12. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. Zhiyong Cheng, Xiaojun Chang, Lei Zhu, Rose C. Kanjirathinkal, and Mohan Kankanhalli. 2019. MMALFM: Explainable recommendation by leveraging reviews and images. ACM Trans. Inform. Syst. 37, 2 (2019), 16. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. Zhiyong Cheng, Ying Ding, Xiangnan He, Lei Zhu, Xuemeng Song, and Mohan S. Kankanhalli. 2018. A3NCF: An adaptive aspect attention model for rating prediction. In Proceedings of the IJCAI. 3748--3754. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Zhiyong Cheng, Jialie Shen, Lei Zhu, Mohan S. Kankanhalli, and Liqiang Nie. 2017. Exploiting music play sequence for music recommendation. In Proceedings of the IJCAI. 3654--3660. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. Paul Covington, Jay Adams, and Emre Sargin. 2016. Deep neural networks for YouTube recommendations. In Proceedings of the RecSys. 191--198. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. George Cybenko. 1989. Approximation by superpositions of a sigmoidal function. Math. Contr., Sig. Syst. 2, 4 (1989), 303--314.Google ScholarGoogle ScholarCross RefCross Ref
  15. Shuiguang Deng, Longtao Huang, Guandong Xu, Xindong Wu, and Zhaohui Wu. 2017. On deep learning for trust-aware recommendations in social networks. IEEE Trans. Neural Netw. Learn. Syst. 28, 5 (2017), 1164--1177.Google ScholarGoogle ScholarCross RefCross Ref
  16. Jingtao Ding, Fuli Feng, Xiangnan He, Guanghui Yu, Yong Li, and Depeng Jin. 2018. An improved sampler for Bayesian personalized ranking by leveraging view data. In Proceedings of the WWW. 13--14. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. Xiangnan He, Xiang Wang, Cheng Luo, Yiqun Liu, Feng Fuli, and Tat-Seng Chua. 2019. Temporal relational ranking for stock prediction. ACM Trans. Inform. Syst. 37, 2 (2019), 27. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. Matt W. Gardner and S. R. Dorling. 1998. Artificial neural networks (the multilayer perceptron)—A review of applications in the atmospheric sciences. Atmos. Environ. 32, 14–15 (1998), 2627--2636.Google ScholarGoogle ScholarCross RefCross Ref
  19. Carlos A. Gomez-Uribe and Neil Hunt. 2016. The Netflix recommender system: Algorithms, business value, and innovation. ACM Trans. Manag. Inform. Syst. 6, 4 (2016), 13.Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. Xinyu Guan, Zhiyong Cheng, Xiangnan He, Yongfeng Zhang, Zhibo Zhu, Qinke Peng, and Tat-Seng Chua. 2019. Attentive aspect modeling for review-aware recommendation. ACM Trans. Inform. Syst. 37, 3 (2019), 28. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. Yangyang Guo, Zhiyong Cheng, Liqiang Nie, Yinglong Wang, Jun Ma, and Mohan Kankanhalli. 2019. Attentive long short-term preference modeling for personalized product search. ACM Trans. Inform. Syst. 37, 2 (2019), 19. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In Proceedings of the CVPR. 770--778.Google ScholarGoogle ScholarCross RefCross Ref
  23. Ruining He and Julian McAuley. 2016. VBPR: Visual Bayesian personalized ranking from implicit feedback. In Proceedings of the AAAI. 144--150. Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. Xiangnan He and Tat-Seng Chua. 2017. Neural factorization machines for sparse predictive analytics. In Proceedings of the SIGIR. 355--364. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. Xiangnan He, Xiaoyu Du, Xiang Wang, Feng Tian, Jinhui Tang, and Tat-Seng Chua. 2018. Outer product-based neural collaborative filtering. In Proceedings of the IJCAI. Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. Xiangnan He, Zhankui He, Xiaoyu Du, and Tat-Seng Chua. 2018. Adversarial personalized ranking for item recommendation. In Proceedings of the SIGIR. Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. Xiangnan He, Zhenkui He, Jingkuan Song, Zhenguang Liu, Yu-Gang Jiang, and Tat-Seng Chua. 2018. NAIS: Neural attentive item similarity model for recommendation. IEEE Trans. Knowl. Data Eng. 30, 12 (2018), 2354--2366.Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. Xiangnan He, Lizi Liao, Hanwang Zhang, Liqiang Nie, Xia Hu, and Tat-Seng Chua. 2017. Neural collaborative filtering. In Proceedings of the WWW. 173--182.Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. Xiangnan He, Jinhui Tang, Xiaoyu Du, Richang Hong, Tongwei Ren, and Tat-Seng Chua. 2019. Fast matrix factorization with non-uniform weights on missing data. IEEE Trans. Neural Netw. Learn. Syst. https://ieeexplore.ieee.org/abstract/document/8624600.Google ScholarGoogle ScholarCross RefCross Ref
  30. Xiangnan He, Hanwang Zhang, Min-Yen Kan, and Tat-Seng Chua. 2016. Fast matrix factorization for online recommendation with implicit feedback. In Proceedings of the SIGIR. 549--558. Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural Comput. 9, 8 (1997), 1735--1780. Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. Kurt Hornik. 1991. Approximation capabilities of multilayer feedforward networks. Neural Netw. 4, 2 (1991), 251--257. Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. Cheng-Kang Hsieh, Longqi Yang, Yin Cui, Tsung-Yi Lin, Serge Belongie, and Deborah Estrin. 2017. Collaborative metric learning. In Proceedings of the WWW. 193--201. Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. Gao Huang, Zhuang Liu, Kilian Q. Weinberger, and Laurens van der Maaten. 2017. Densely connected convolutional networks. In Proceedings of the CVPR. 4700--4708.Google ScholarGoogle Scholar
  35. Santosh Kabbur, Xia Ning, and George Karypis. 2013. FISM: Factored item similarity models for top-n recommender systems. In Proceedings of the SIGKDD. ACM, 659--667. Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. Donghyun Kim, Chanyoung Park, Jinoh Oh, Sungyoung Lee, and Hwanjo Yu. 2016. Convolutional matrix factorization for document context-aware recommendation. In Proceedings of the RecSys. ACM, 233--240. Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. Yehuda Koren. 2008. Factorization meets the neighborhood: A multifaceted collaborative filtering model. In Proceedings of the SIGKDD. ACM, 426--434. Google ScholarGoogle ScholarDigital LibraryDigital Library
  38. Yehuda Koren, Robert Bell, and Chris Volinsky. 2009. Matrix factorization techniques for recommender systems. Computer 42, 8 (2009). Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton. 2012. Imagenet classification with deep convolutional neural networks. In Proceedings of the NIPS. 1097--1105. Google ScholarGoogle ScholarDigital LibraryDigital Library
  40. Dawen Liang, Laurent Charlin, James McInerney, and David M. Blei. 2016. Modeling user exposure in recommendation. In Proceedings of the WWW. 951--961. Google ScholarGoogle ScholarDigital LibraryDigital Library
  41. Zhouhan Lin, Minwei Feng, Cicero Nogueira dos Santos, Mo Yu, Bing Xiang, Bowen Zhou, and Yoshua Bengio. 2017. A structured self-attentive sentence embedding. In Proceedings of the ICLR.Google ScholarGoogle Scholar
  42. Zhongqi Lu, Zhicheng Dou, Jianxun Lian, Xing Xie, and Qiang Yang. 2015. Content-based collaborative filtering for news topic recommendation. In Proceedings of the AAAI. 217--223. Google ScholarGoogle ScholarDigital LibraryDigital Library
  43. Xin Luo, MengChu Zhou, Shuai Li, Zhuhong You, Yunni Xia, and Qingsheng Zhu. 2016. A nonnegative latent factor model for large-scale sparse matrices in recommender systems via alternating direction method. IEEE Trans. Neural Netw. Learn. Syst. 27, 3 (2016), 579--592.Google ScholarGoogle ScholarCross RefCross Ref
  44. Zhanyu Ma, Yuping Lai, W. Bastiaan Kleijn, Yi-Zhe Song, Liang Wang, and Jun Guo. 2018. Variational Bayesian learning for Dirichlet process mixture of inverted Dirichlet distributions in non-Gaussian image feature modeling. IEEE Trans. Neural Netw. Learn. Syst. 99 (2018), 1--15.Google ScholarGoogle ScholarCross RefCross Ref
  45. Zhanyu Ma, Jing-Hao Xue, Arne Leijon, Zheng-Hua Tan, Zhen Yang, and Jun Guo. 2018. Decorrelation of neutral vector variables: Theory and applications. IEEE Trans. Neural Netw. Learn. Syst. 29, 1 (2018), 129--143.Google ScholarGoogle ScholarCross RefCross Ref
  46. Weike Pan, Qiang Yang, Wanling Cai, Yaofeng Chen, Qing Zhang, Xiaogang Peng, and Zhong Ming. 2019. Transfer to rank for heterogeneous one-class collaborative filtering. ACM Trans. Inform. Syst. 37, 1 (2019), 10. Google ScholarGoogle ScholarDigital LibraryDigital Library
  47. Tieyun Qian, Bei Liu, Quoc Viet Hung Nguyen, and Hongzhi Yin. 2019. Spatiotemporal representation learning for translation-based POI recommendation. ACM Trans. Inform. Syst. 37, 2 (2019), 18. Google ScholarGoogle ScholarDigital LibraryDigital Library
  48. Bohui Fang, Weinan Zhang, Ruiming Tang, Minzhe Niu, Huifeng Guo, Yong Yu, Yanru Qu, and Xiuqiang He. 2018. Product-based neural networks for user response prediction over multi-field categorical data. ACM Trans. Inform. Syst. 37, 1 (2018), 5. Google ScholarGoogle ScholarDigital LibraryDigital Library
  49. Steffen Rendle. 2010. Factorization machines. In Proceedings of the ICDM. 995--1000. Google ScholarGoogle ScholarDigital LibraryDigital Library
  50. Steffen Rendle, Christoph Freudenthaler, Zeno Gantner, and Lars Schmidt-Thieme. 2009. BPR: Bayesian personalized ranking from implicit feedback. In Proceedings of the UAI. 452--461. Google ScholarGoogle ScholarDigital LibraryDigital Library
  51. Amit Sharma, Jake M. Hofman, and Duncan J. Watts. 2015. Estimating the causal impact of recommendation systems from observational data. In Proceedings of the EC. ACM, 453--470. Google ScholarGoogle ScholarDigital LibraryDigital Library
  52. Yi Tay, Luu Anh Tuan, and Siu Cheung Hui. 2018. Latent relational metric learning via memory-based attention for collaborative ranking. In Proceedings of the WWW. 729--739. Google ScholarGoogle ScholarDigital LibraryDigital Library
  53. Hongwei Wang, Fuzheng Zhang, Xing Xie, and Minyi Guo. 2018. DKN: Deep knowledge-aware network for news recommendation. In Proceedings of the WWW. 1835--1844. Google ScholarGoogle ScholarDigital LibraryDigital Library
  54. Suhang Wang, Jiliang Tang, Yilin Wang, and Huan Liu. 2015. Exploring implicit hierarchical structures for recommender systems. In Proceedings of the IJCAI. 1813--1819. Google ScholarGoogle ScholarDigital LibraryDigital Library
  55. Xiang Wang, Xiangnan He, Liqiang Nie, and Tat-Seng Chua. 2017. Item Silk Road: Recommending items from information domains to social users. In Proceedings of the SIGIR. 185--194. Google ScholarGoogle ScholarDigital LibraryDigital Library
  56. Libing Wu, Cong Quan, Chenliang Li, Qian Wang, Bolong Zheng, and Xiangyang Luo. 2019. A context-aware user-item representation learning for item recommendation. ACM Trans. Inform. Syst. 37, 2 (2019), 22. Google ScholarGoogle ScholarDigital LibraryDigital Library
  57. Yao Wu, Christopher DuBois, Alice X. Zheng, and Martin Ester. 2016. Collaborative denoising auto-encoders for top-n recommender systems. In Proceedings of the WSDM. ACM, 153--162. Google ScholarGoogle ScholarDigital LibraryDigital Library
  58. Hong-Jian Xue, Xinyu Dai, Jianbing Zhang, Shujian Huang, and Jiajun Chen. 2017. Deep matrix factorization models for recommender systems. In Proceedings of the IJCAI. 3203--3209. Google ScholarGoogle ScholarDigital LibraryDigital Library
  59. Fisher Yu and Vladlen Koltun. 2015. Multi-scale context aggregation by dilated convolutions. Retrieved from: arXiv preprint arXiv:1511.07122 (2015).Google ScholarGoogle Scholar
  60. Wenhui Yu, Huidi Zhang, Xiangnan He, Xu Chen, Li Xiong, and Zheng Qin. 2018. Aesthetic-based clothing recommendation. In Proceedings of the WWW. 649--658. Google ScholarGoogle ScholarDigital LibraryDigital Library
  61. Fajie Yuan, Guibing Guo, Joemon M. Jose, Long Chen, Haitao Yu, and Weinan Zhang. 2016. LambdaFM: Learning optimal ranking with factorization machines using lambda surrogates. In Proceedings of the CIKM. ACM, 227--236. Google ScholarGoogle ScholarDigital LibraryDigital Library
  62. Fajie Yuan, Alexandros Karatzoglou, Ioannis Arapakis, Joemon M. Jose, and Xiangnan He. 2019. A simple convolutional generative network for next item recommendation. In Proceedings of the WSDM. Google ScholarGoogle ScholarDigital LibraryDigital Library
  63. Fajie Yuan, Xin Xin, Xiangnan He, Guibing Guo, Weinan Zhang, Chua Tat-Seng, and Joemon M. Jose. 2018. FBGD: Learning embeddings from positive unlabeled data with BGD. In Proceedings of UAI 2018. http://auai.org/uai2018/schedule.php.Google ScholarGoogle Scholar
  64. Yongfeng Zhang, Qingyao Ai, Xu Chen, and W. Bruce Croft. 2017. Joint representation learning for top-n recommendation with heterogeneous information sources. In Proceedings of the CIKM. 1449--1458. Google ScholarGoogle ScholarDigital LibraryDigital Library
  65. Yongfeng Zhang, Min Zhang, Yiqun Liu, Shaoping Ma, and Shi Feng. 2013. Localized matrix factorization for recommendation based on matrix block diagonal forms. In Proceedings of the WWW. ACM, 1511--1520. Google ScholarGoogle ScholarDigital LibraryDigital Library
  66. Wayne Xin Zhao, Wenhui Zhang, Yulan He, Xing Xie, and Ji-Rong Wen. 2018. Automatically learning topics and difficulty levels of problems in online judge systems. ACM Trans. Inform. Syst. 36, 3 (2018), 27. Google ScholarGoogle ScholarDigital LibraryDigital Library
  67. Zhou Zhao, Hanqing Lu, Deng Cai, Xiaofei He, and Yueting Zhuang. 2016. User preference learning for online social recommendation. IEEE Trans. Knowl. Data Eng. 28, 9 (2016), 2522--2534. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Modeling Embedding Dimension Correlations via Convolutional Neural Collaborative Filtering

        Recommendations

        Comments

        Login options

        Check if you have access through your login credentials or your institution to get full access on this article.

        Sign in

        Full Access

        • Published in

          cover image ACM Transactions on Information Systems
          ACM Transactions on Information Systems  Volume 37, Issue 4
          October 2019
          299 pages
          ISSN:1046-8188
          EISSN:1558-2868
          DOI:10.1145/3357218
          Issue’s Table of Contents

          Copyright © 2019 ACM

          Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

          Publisher

          Association for Computing Machinery

          New York, NY, United States

          Publication History

          • Published: 13 September 2019
          • Accepted: 1 August 2019
          • Revised: 1 June 2019
          • Received: 1 April 2019
          Published in tois Volume 37, Issue 4

          Permissions

          Request permissions about this article.

          Request Permissions

          Check for updates

          Qualifiers

          • research-article
          • Research
          • Refereed

        PDF Format

        View or Download as a PDF file.

        PDF

        eReader

        View online with eReader.

        eReader

        HTML Format

        View this article in HTML Format .

        View HTML Format