Anchen Li
ABSTRACT
Hypercomplex algebras are well-developed in the area of mathematics. Recently, several hypercomplex recommendation approaches have been proposed and yielded great success. However, two vital issues have not been well-considered in existing hypercomplex recommenders. First, these methods are only designed for specific and low-dimensional hypercomplex algebras (e.g., complex and quaternion algebras), ignoring the exploration and utilization of high-dimensional ones. Second, most recommenders treat every user-item interaction as an isolated data instance, without considering high-order collaborative relationships.
To bridge these gaps, in this paper, we propose a novel recommendation framework named HyperComplex Graph Collaborative Filtering (HCGCF). To study the high-dimensional hypercomplex algebras, we introduce Cayley–Dickson construction which utilizes a recursive process to define hypercomplex algebras and their mathematical operations. Based on Cayley–Dickson construction, we devise a hypercomplex graph convolution operator to learn user and item representations. Specifically, the operator models both the neighborhood summary and interaction relations with neighbors in hypercomplex spaces, effectively exploiting the high-order connectivity in the user-item bipartite graph. To the best of our knowledge, it is the first time that Cayley-Dickson construction and graph convolution techniques have been explicitly discussed and used in hypercomplex recommender systems. Compared with several state-of-the-art recommender baselines, HCGCF achieves superior performance in both click-through rate prediction and top-K recommendation on three real-world datasets.
- Daniel Alfsmann. 2006. On families of 2 N-dimensional hypercomplex algebras suitable for digital signal processing. In EUSIPCO. 1–4.Google Scholar
- John Baez. 2002. The octonions. Bulletin of the american mathematical society 39, 2 (2002), 145–205.Google Scholar
- Rianne van den Berg, Thomas N Kipf, and Max Welling. 2017. Graph convolutional matrix completion. In KDD.Google Scholar
- Jiangxia Cao, Xixun Lin, Shu Guo, Luchen Liu, Tingwen Liu, and Bin Wang. 2021. Bipartite graph embedding via mutual information maximization. In WSDM. 635–643.Google Scholar
- Jie Chen, Tengfei Ma, and Cao Xiao. 2018. Fastgcn: fast learning with graph convolutional networks via importance sampling. In ICLR.Google Scholar
- Lei Chen, Le Wu, Richang Hong, Kun Zhang, and Meng Wang. 2020. Revisiting graph based collaborative filtering: A linear residual graph convolutional network approach. In AAAI. 27–34.Google Scholar
- Tong Chen, Hongzhi Yin, Xiangliang Zhang, Zi Huang, Yang Wang, and Meng Wang. 2021. Quaternion Factorization Machines: A Lightweight Solution to Intricate Feature Interaction Modelling. TNNLS (2021).Google Scholar
- Weiyu Cheng, Yanyan Shen, Yanmin Zhu, and Linpeng Huang. 2018. DELF: A Dual-Embedding based Deep Latent Factor Model for Recommendation.. In IJCAI. 3329–3335.Google Scholar
- Colin Cooper, Sang Hyuk Lee, Tomasz Radzik, and Yiannis Siantos. 2014. Random walks in recommender systems: exact computation and simulations. In WWW. 811–816.Google Scholar
- Craig Culbert. 2007. Cayley-Dickson algebras and loops. Journal of Forensic Biomechanics 1, 1 (2007), 1–17.Google Scholar
- Leonard E Dickson. 1919. On quaternions and their generalization and the history of the eight square theorem. Annals of Mathematics(1919), 155–171.Google Scholar
- Yaxing Fang, Pengpeng Zhao, Guanfeng Liu, Yanchi Liu, Victor S Sheng, Lei Zhao, and Xiaofang Zhou. 2021. Quaternion-Based Graph Convolution Network for Recommendation. arXiv preprint arXiv:2111.10536(2021).Google Scholar
- Chase J Gaudet and Anthony S Maida. 2020. Generalizing Complex/Hyper-complex Convolutions to Vector Map Convolutions. arXiv preprint arXiv:2009.04083(2020).Google Scholar
- Xavier Glorot and Yoshua Bengio. 2010. Understanding the difficulty of training deep feedforward neural networks. In AISTATS. 249–256.Google Scholar
- Marco Gori, Augusto Pucci, V Roma, and I Siena. 2007. Itemrank: A random-walk based scoring algorithm for recommender engines.. In IJCAI. 2766–2771.Google Scholar
- Huifeng Guo, Ruiming Tang, Yunming Ye, Zhenguo Li, and Xiuqiang He. 2017. DeepFM: a factorization-machine based neural network for CTR prediction. In IJCAI.Google Scholar
- Xiangnan He, Kuan Deng, Xiang Wang, Yan Li, Yongdong Zhang, and Meng Wang. 2020. Lightgcn: Simplifying and powering graph convolution network for recommendation. In SIGIR. 639–648.Google ScholarDigital Library
- Xiangnan He, Ming Gao, Min-Yen Kan, and Dingxian Wang. 2016. Birank: Towards ranking on bipartite graphs. IEEE Transactions on Knowledge and Data Engineering 29, 1(2016), 57–71.Google ScholarDigital Library
- Xiangnan He, Zhankui He, Jingkuan Song, Zhenguang Liu, Yu-Gang Jiang, and Tat-Seng Chua. 2018. Nais: Neural attentive item similarity model for recommendation. IEEE Transactions on Knowledge and Data Engineering 30, 12(2018), 2354–2366.Google ScholarDigital Library
- Xiangnan He, Lizi Liao, Hanwang Zhang, Liqiang Nie, Xia Hu, and Tat-Seng Chua. 2017. Neural collaborative filtering. In WWW. 173–182.Google Scholar
- Thomas N Kipf and Max Welling. 2017. Semi-supervised classification with graph convolutional networks. In ICLR.Google Scholar
- Yehuda Koren, Robert Bell, and Chris Volinsky. 2009. Matrix factorization techniques for recommender systems. Computer (2009), 30–37.Google ScholarDigital Library
- Kantor I L, Kantor I L, and Solodovnikov A S. 1989. Hypercomplex numbers: an elementary introduction to algebras. Springer.Google Scholar
- Srdan Lazendic, Aleksandra Pizurica, and Hendrik De Bie. 2018. Hypercomplex algebras for dictionary learning. In AGACSE. 57–64.Google Scholar
- Anchen Li, Bo Yang, Hongxu Chen, and Guandong Xu. 2021. Hyperbolic Neural Collaborative Recommender. arXiv preprint arXiv:2104.07414(2021).Google Scholar
- Anchen Li, Bo Yang, Huan Huo, and Farookh Khadeer Hussain. 2021. Leveraging implicit relations for recommender systems. Information Sciences 579(2021), 55–71.Google ScholarCross Ref
- Zhaopeng Li, Qianqian Xu, Yangbangyan Jiang, Xiaochun Cao, and Qingming Huang. 2020. Quaternion-Based Knowledge Graph Network for Recommendation. In MM. 880–888.Google Scholar
- Tu Dinh Nguyen, Dinh Phung, 2021. Quaternion graph neural networks. In ACML. PMLR, 236–251.Google Scholar
- Titouan Parcollet, Mirco Ravanelli, Mohamed Morchid, Georges Linarès, Chiheb Trabelsi, Renato De Mori, and Yoshua Bengio. 2019. Quaternion recurrent neural networks. In ICLR.Google Scholar
- Dario Pavllo, Christoph Feichtenhofer, Michael Auli, and David Grangier. 2020. Modeling human motion with quaternion-based neural networks. IJCV 128, 4 (2020), 855–872.Google ScholarDigital Library
- Metod Saniga, Frédéric Holweck, and Petr Pracna. 2014. Cayley-Dickson algebras and finite geometry. arXiv preprint arXiv:1405.6888(2014).Google Scholar
- Chuan Shi, Binbin Hu, Wayne Xin Zhao, and S Yu Philip. 2018. Heterogeneous information network embedding for recommendation. IEEE Transactions on Knowledge and Data Engineering 31, 2(2018), 357–370.Google ScholarDigital Library
- Jianing Sun, Yingxue Zhang, Wei Guo, Huifeng Guo, Ruiming Tang, Xiuqiang He, Chen Ma, and Mark Coates. 2020. Neighbor interaction aware graph convolution networks for recommendation. In SIGIR. 1289–1298.Google Scholar
- Zhiqing Sun, Zhi-Hong Deng, Jian-Yun Nie, and Jian Tang. 2019. Rotate: Knowledge graph embedding by relational rotation in complex space. In ICLR.Google Scholar
- Jiaxi Tang and Ke Wang. 2018. Personalized top-n sequential recommendation via convolutional sequence embedding. In WSDM. 565–573.Google Scholar
- Yi Tay, Aston Zhang, Luu Anh Tuan, Jinfeng Rao, Shuai Zhang, Shuohang Wang, Jie Fu, and Siu Cheung Hui. 2019. Lightweight and efficient neural natural language processing with quaternion networks. In ACL. 1494–1503.Google Scholar
- Chiheb Trabelsi, Olexa Bilaniuk, Ying Zhang, Dmitriy Serdyuk, Sandeep Subramanian, Joao Felipe Santos, Soroush Mehri, Negar Rostamzadeh, Yoshua Bengio, and Christopher J Pal. 2018. Deep complex networks. In ICLR.Google Scholar
- Thanh Tran, Di You, and Kyumin Lee. 2020. Quaternion-based self-attentive long short-term user preference encoding for recommendation. In CIKM. 1455–1464.Google Scholar
- Théo Trouillon, Johannes Welbl, Sebastian Riedel, Éric Gaussier, and Guillaume Bouchard. 2016. Complex embeddings for simple link prediction. In ICML. 2071–2080.Google Scholar
- Laurens Van der Maaten and Geoffrey Hinton. 2008. Visualizing data using t-SNE.Journal of machine learning research 9, 11 (2008).Google Scholar
- Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. 2018. Graph attention networks. In ICLR.Google Scholar
- Guilherme Vieira and Marcos Eduardo Valle. 2020. Extreme Learning Machines on Cayley-Dickson Algebra Applied for Color Image Auto-Encoding. In IJCNN. 1–8.Google Scholar
- Guilherme Vieira and Marcos Eduardo Valle. 2021. A General Framework for Hypercomplex-valued Extreme Learning Machines. arXiv preprint arXiv:2101.06166(2021).Google Scholar
- Xiang Wang, Xiangnan He, Meng Wang, Fuli Feng, and Tat-Seng Chua. 2019. Neural graph collaborative filtering. In SIGIR. 165–174.Google Scholar
- Hong-Jian Xue, Xinyu Dai, Jianbing Zhang, Shujian Huang, and Jiajun Chen. 2017. Deep Matrix Factorization Models for Recommender Systems.. In IJCAI. 3203–3209.Google Scholar
- Jheng-Hong Yang, Chih-Ming Chen, Chuan-Ju Wang, and Ming-Feng Tsai. 2018. HOP-rec: high-order proximity for implicit recommendation. In RecSys. 140–144.Google Scholar
- Baolin Yi, Xiaoxuan Shen, Hai Liu, Zhaoli Zhang, Wei Zhang, Sannyuya Liu, and Naixue Xiong. 2019. Deep matrix factorization with implicit feedback embedding for recommendation system. IEEE Transactions on Industrial Informatics 15, 8 (2019), 4591–4601.Google ScholarCross Ref
- Shuai Zhang, Yi Tay, Lina Yao, and Qi Liu. 2019. Quaternion knowledge graph embeddings. In NeurIPS. 2731–2741.Google Scholar
- Shuai Zhang, Lina Yao, Lucas Vinh Tran, Aston Zhang, and Yi Tay. 2019. Quaternion Collaborative Filtering for Recommendation. In IJCAI. 4313–4319.Google Scholar
- Xuanyu Zhu, Yi Xu, Hongteng Xu, and Changjian Chen. 2018. Quaternion convolutional neural networks. In ECCV. 631–647.Google Scholar
Index Terms
- Hypercomplex Graph Collaborative Filtering
Recommendations
Typicality-Based Collaborative Filtering Recommendation
Collaborative filtering (CF) is an important and popular technology for recommender systems. However, current CF methods suffer from such problems as data sparsity, recommendation inaccuracy, and big-error in predictions. In this paper, we borrow ideas ...
Kernelized probabilistic matrix factorization for collaborative filtering: exploiting projected user and item graph
RecSys '18: Proceedings of the 12th ACM Conference on Recommender SystemsMatrix Factorization (MF) techniques have already shown its strong foundation in collaborative filtering (CF), particularly for rating prediction problem. In the basic MF model, the use of additional information such as social network, item tags along ...
Item enhanced graph collaborative network for collaborative filtering recommendation
AbstractLearning vector embeddings of users and items is the core of modern recommender systems. Recently the collaborative filtering recommender systems based on graph convolutional networks, which integrates the bipartite graph of user-item interaction ...
Comments