Mingyi Liu
Skip Abstract Section
Abstract
Dynamic link prediction has become a trending research subject because of its wide applications in the web, sociology, transportation, and bioinformatics. Currently, the prevailing approach for dynamic link prediction is based on graph neural networks, in which graph representation learning is the key to perform dynamic link prediction tasks. However, there are still great challenges because the structure of graphs evolves over time. A common approach is to represent a dynamic graph as a collection of discrete snapshots, in which information over a period is aggregated through summation or averaging. This way results in some fine-grained time-related information loss, which further leads to a certain degree of performance degradation. We conjecture that such fine-grained information is vital because it implies specific behavior patterns of nodes and edges in a snapshot. To verify this conjecture, we propose a novel fine-grained behavior-aware network (BehaviorNet) for dynamic network link prediction. Specifically, BehaviorNet adapts a transformer-based graph convolution network to capture the latent structural representations of nodes by adding edge behaviors as an additional attribute of edges. GRU is applied to learn the temporal features of given snapshots of a dynamic network by utilizing node behaviors as auxiliary information. Extensive experiments are conducted on several real-world dynamic graph datasets, and the results show significant performance gains for BehaviorNet over several state-of-the-art (SOTA) discrete dynamic link prediction baselines. Ablation study validates the effectiveness of modeling fine-grained edge and node behaviors.
- [1] . 2018. FastGCN: Fast learning with graph convolutional networks via importance sampling. In 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30–May 3, 2018, Conference Track Proceedings. OpenReview.net. https://openreview.net/forum?id=rytstxWAW.Google Scholar
- [2] . 2021. GC-LSTM: Graph convolution embedded LSTM for dynamic network link prediction. Applied Intelligence (2021), 1–16.Google Scholar
- [3] . 2020. Simple and deep graph convolutional networks. In Proceedings of the 37th International Conference on Machine Learning, ICML 2020, 13–18 July 2020, Virtual Event(
Proceedings of Machine Learning Research , Vol. 119). PMLR, 1725–1735. http://proceedings.mlr.press/v119/chen20v.html.Google Scholar - [4] . 2017. Gate-variants of Gated Recurrent Unit (GRU) neural networks. In IEEE 60th International Midwest Symposium on Circuits and Systems, MWSCAS 2017, Boston, MA, USA, August 6–9, 2017. IEEE, 1597–1600.
DOI: Google ScholarCross Ref
- [5] . 2019. Fast graph representation learning with PyTorch geometric. In ICLR Workshop on Representation Learning on Graphs and Manifolds.Google Scholar
- [6] . 2020. Attentional graph convolutional networks for knowledge concept recommendation in MOOCs in a heterogeneous view. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval. 79–88.Google ScholarDigital Library
- [7] . 2019. Exploiting edge features for graph neural networks. In IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2019, Long Beach, CA, USA, June 16–20, 2019. Computer Vision Foundation / IEEE, 9211–9219.
DOI: Google ScholarCross Ref - [8] . 2018. DynGEM: Deep embedding method for dynamic graphs. CoRR abs/1805.11273 (2018).
arXiv:1805.11273 http://arxiv.org/abs/1805.11273.Google Scholar - [9] . 2016. node2vec: Scalable feature learning for networks. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Francisco, CA, USA, August 13–17, 2016, , , , , , and (Eds.). ACM, 855–864.
DOI: Google ScholarDigital Library - [10] . 2017. Inductive representation learning on large graphs. In Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4–9, 2017, Long Beach, CA, USA, , , , , , , and (Eds.). 1024–1034. https://proceedings.neurips.cc/paper/2017/hash/5dd9db5e033da9c6fb5ba83c7a7ebea9-Abstract.html.Google Scholar
- [11] . 2020. Graph Hawkes neural network for forecasting on temporal knowledge graphs. In Conference on Automated Knowledge Base Construction, AKBC 2020, Virtual, June 22–24, 2020, , , , and (Eds.).
DOI: Google ScholarCross Ref - [12] . 2011. What’s in a crowd? Analysis of face-to-face behavioral networks. Journal of Theoretical Biology 271, 1 (2011), 166–180.Google ScholarCross Ref
- [13] . 2015. Correlation network analysis for multi-dimensional data in stocks market. Physica A: Statistical Mechanics and its Applications 429 (2015), 62–75.Google ScholarCross Ref
- [14] . 2015. Adam: A method for stochastic optimization. In 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7–9, 2015, Conference Track Proceedings, and (Eds.). http://arxiv.org/abs/1412.6980.Google Scholar
- [15] . 2017. Semi-supervised classification with graph convolutional networks. (2017). https://openreview.net/forum?id=SJU4ayYgl.Google Scholar
- [16] . 2019. Predict then propagate: Graph neural networks meet personalized PageRank. In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6–9, 2019. OpenReview.net. https://openreview.net/forum?id=H1gL-2A9Ym.Google Scholar
- [17] . 2004. The Enron corpus: A new dataset for email classification research. In Machine Learning: ECML 2004, 15th European Conference on Machine Learning, Pisa, Italy, September 20–24, 2004, Proceedings(
Lecture Notes in Computer Science , Vol. 3201), , , , and (Eds.). Springer, 217–226.DOI: Google ScholarDigital Library - [18] . 2021. Learning temporal attention in dynamic graphs with bilinear interactions. PLOS ONE 16, 3 (
03 2021), 1–18.DOI: Google ScholarCross Ref - [19] . 2018. REV2: Fraudulent user prediction in rating platforms. In Proceedings of the Eleventh ACM International Conference on Web Search and Data Mining, WSDM 2018, Marina Del Rey, CA, USA, February 5–9, 2018, , , , and (Eds.). ACM, 333–341.
DOI: Google ScholarDigital Library - [20] . 2019. Predicting dynamic embedding trajectory in temporal interaction networks. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, KDD 2019, Anchorage, AK, USA, August 4–8, 2019, , , , , , and (Eds.). ACM, 1269–1278.
DOI: Google ScholarDigital Library - [21] . 2019. DeepGCNs: Can GCNs go as deep as CNNs?. In 2019 IEEE/CVF International Conference on Computer Vision, ICCV 2019, Seoul, Korea (South), October 27 - November 2, 2019. IEEE, 9266–9275.
DOI: Google ScholarCross Ref - [22] . 2016. Discriminative deep random walk for network classification. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, ACL 2016, August 7–12, 2016, Berlin, Germany, Volume 1: Long Papers. The Association for Computer Linguistics.
DOI: Google ScholarCross Ref - [23] . 2020. K-Core based temporal graph convolutional network for dynamic graphs. IEEE Transactions on Knowledge and Data Engineering01 (2020), 1–1.
DOI: Google ScholarCross Ref - [24] . 2020. Towards fine-grained temporal network representation via time-reinforced random walk. In The Thirty-Fourth AAAI Conference on Artificial Intelligence, AAAI 2020, The Thirty-Second Innovative Applications of Artificial Intelligence Conference, IAAI 2020, The Tenth AAAI Symposium on Educational Advances in Artificial Intelligence, EAAI 2020, New York, NY, USA, February 7–12, 2020. AAAI Press, 4973–4980. https://aaai.org/ojs/index.php/AAAI/article/view/5936.Google ScholarCross Ref
- [25] . 2020. Streaming graph neural networks. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2020, Virtual Event, China, July 25–30, 2020, , , , , , , and (Eds.). ACM, 719–728.
DOI: Google ScholarDigital Library - [26] . 2014. Inductive matrix completion for predicting gene-disease associations. Bioinform. 30, 12 (2014), 60–68.
DOI: Google ScholarCross Ref - [27] . 2019. AGL-score: Algebraic graph learning score for protein-ligand binding scoring, ranking, docking, and screening. J. Chem. Inf. Model. 59, 7 (2019), 3291–3304.
DOI: Google ScholarCross Ref - [28] . 2018. Continuous-time dynamic network embeddings. In Companion of The Web Conference 2018 on The Web Conference 2018, WWW 2018, Lyon , France, April 23–27, 2018, , , , and (Eds.). ACM, 969–976.
DOI: Google ScholarDigital Library - [29] . 2003. Graph-based anomaly detection. In Proceedings of the Ninth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Washington, DC, USA, August 24–27, 2003, , , , and (Eds.). ACM, 631–636.
DOI: Google ScholarDigital Library - [30] . 2009. Clustering in weighted networks. Soc. Networks 31, 2 (2009), 155–163.
DOI: Google ScholarCross Ref - [31] . 2009. Patterns and dynamics of users’ behavior and interaction: Network analysis of an online community. Journal of the American Society for Information Science and Technology 60, 5 (2009), 911–932.Google ScholarDigital Library
- [32] . 2020. EvolveGCN: Evolving graph convolutional networks for dynamic graphs. In The Thirty-Fourth AAAI Conference on Artificial Intelligence, AAAI 2020, The Thirty-Second Innovative Applications of Artificial Intelligence Conference, IAAI 2020, The Tenth AAAI Symposium on Educational Advances in Artificial Intelligence, EAAI 2020, New York, NY, USA, February 7–12, 2020. AAAI Press, 5363–5370. https://aaai.org/ojs/index.php/AAAI/article/view/5984.Google ScholarCross Ref
- [33] . 2020. Dynamic network embedding via incremental skip-gram with negative sampling. Science China Information Sciences 63, 10 (2020), 1–19.Google ScholarCross Ref
- [34] . 2021. Lime: Low-cost and incremental learning for dynamic heterogeneous information networks. IEEE Trans. Comput. 71, 3 (2021), 628–642.Google ScholarDigital Library
- [35] . 2014. DeepWalk: Online learning of social representations. In The 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD’14, New York, NY, USA - August 24–27, 2014, , , , , and (Eds.). ACM, 701–710.
DOI: Google ScholarDigital Library - [36] . 2020. Temporal graph networks for deep learning on dynamic graphs. CoRR abs/2006.10637.
arXiv:2006.10637 https://arxiv.org/abs/2006.10637.Google Scholar - [37] . 2015. The network data repository with interactive graph analytics and visualization. In Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence, January 25–30, 2015, Austin, Texas, USA, and (Eds.). AAAI Press, 4292–4293. http://www.aaai.org/ocs/index.php/AAAI/AAAI15/paper/view/9553.Google ScholarCross Ref
- [38] . 2021. PyTorch geometric temporal: Spatiotemporal signal processing with neural machine learning models. In Proceedings of the 30th ACM International Conference on Information and Knowledge Management. 4564–4573.Google ScholarDigital Library
- [39] . 2020. DySAT: Deep neural representation learning on dynamic graphs via self-attention networks. In WSDM’20: The Thirteenth ACM International Conference on Web Search and Data Mining, Houston, TX, USA, February 3–7, 2020, , , , and (Eds.). ACM, 519–527.
DOI: Google ScholarDigital Library - [40] . 2018. Structured sequence modeling with graph convolutional recurrent networks. In Neural Information Processing - 25th International Conference, ICONIP 2018, Siem Reap, Cambodia, December 13–16, 2018, Proceedings, Part I(
Lecture Notes in Computer Science , Vol. 11301), , , and (Eds.). Springer, 362–373.DOI: Google ScholarDigital Library - [41] . 2020. Fundamentals of recurrent neural network (RNN) and long short-term memory (LSTM) network. Physica D: Nonlinear Phenomena 404 (2020), 132306.Google ScholarCross Ref
- [42] . 2021. Masked label prediction: Unified message passing model for semi-supervised classification. (2021), 1548–1554.
DOI: Google ScholarCross Ref - [43] . 2020. Foundations and modelling of dynamic networks using dynamic graph neural networks: A survey. CoRR abs/2005.07496 (2020).
arXiv:2005.07496 https://arxiv.org/abs/2005.07496.Google Scholar - [44] . 2021. Hyperbolic variational graph neural network for modeling dynamic graphs. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 35. 4375–4383.Google ScholarCross Ref
- [45] . 2019. Predictive temporal embedding of dynamic graphs. In ASONAM’19: International Conference on Advances in Social Networks Analysis and Mining, Vancouver, British Columbia, Canada, 27–30 August, 2019, , , and (Eds.). ACM, 57–64.
DOI: Google ScholarDigital Library - [46] . 2009. Relational learning via latent social dimensions. In Proceedings of the 15th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Paris, France, June 28–July 1, 2009, , , , and (Eds.). ACM, 817–826.
DOI: Google ScholarDigital Library - [47] . 2021. Bootstrapped representation learning on graphs. CoRR abs/2102.06514 (2021).
arXiv:2102.06514 https://arxiv.org/abs/2102.06514.Google Scholar - [48] . 2019. DyRep: Learning representations over dynamic graphs. In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6–9, 2019. OpenReview.net. https://openreview.net/forum?id=HyePrhR5KX.Google Scholar
- [49] . 2008. Visualizing data using t-SNE. Journal of Machine Learning Research 9, 11 (2008).Google Scholar
- [50] . 2017. Attention is all you need. In Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4–9, 2017, Long Beach, CA, USA, , , , , , , and (Eds.). 5998–6008. https://proceedings.neurips.cc/paper/2017/hash/3f5ee243547dee91fbd053c1c4a845aa-Abstract.html.Google Scholar
- [51] . 2018. Graph attention networks. In 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30–May 3, 2018, Conference Track Proceedings. OpenReview.net. https://openreview.net/forum?id=rJXMpikCZ.Google Scholar
- [52] . 2020. Seven-layer model in complex networks link prediction: A survey. Sensors 20, 22 (2020).
DOI: Google ScholarCross Ref - [53] . 2017. Knowledge graph embedding: A survey of approaches and applications. IEEE Trans. Knowl. Data Eng. 29, 12 (2017), 2724–2743.
DOI: Google ScholarCross Ref - [54] . 2019. Simplifying graph convolutional networks. In Proceedings of the 36th International Conference on Machine Learning, ICML 2019, 9–15 June 2019, Long Beach, California, USA(
Proceedings of Machine Learning Research , Vol. 97), and (Eds.). PMLR, 6861–6871. http://proceedings.mlr.press/v97/wu19e.html.Google Scholar - [55] . 2016. An efficient supervised training algorithm for multilayer spiking neural networks. PloS One 11, 4 (2016), e0150329.Google ScholarCross Ref
- [56] . 2021. InfoGCL: Information-aware graph contrastive learning. Advances in Neural Information Processing Systems 34 (2021).Google Scholar
- [57] . 2015. Network representation learning with rich text information. In Proceedings of the Twenty-Fourth International Joint Conference on Artificial Intelligence, IJCAI 2015, Buenos Aires, Argentina, July 25–31, 2015, and (Eds.). AAAI Press, 2111–2117. http://ijcai.org/Abstract/15/299.Google Scholar
- [58] . 2020. Revisiting “Over-smoothing” in deep GCNs. CoRR abs/2003.13663 (2020).
arXiv:2003.13663 https://arxiv.org/abs/2003.13663.Google Scholar - [59] . 2020. Evolution of cooperation in a conformity-driven evolving dynamic social network. Appl. Math. Comput. 379 (2020), 125251.
DOI: Google ScholarCross Ref - [60] . 2019. Graph transformer networks. (2019), 11960–11970. https://proceedings.neurips.cc/paper/2019/hash/9d63484abb477c97640154d40595a3bb-Abstract.html.Google Scholar
- [61] . 2019. Graph convolutional networks: A comprehensive review. Computational Social Networks 6, 1 (2019), 1–23.Google ScholarCross Ref
- [62] . 2020. PairNorm: Tackling oversmoothing in GNNs. In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26–30, 2020. OpenReview.net. https://openreview.net/forum?id=rkecl1rtwB.Google Scholar
- [63] . 2019. AddGraph: Anomaly detection in dynamic graph using attention-based temporal GCN. In Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, IJCAI 2019, Macao, China, August 10–16, 2019, (Ed.). ijcai.org, 4419–4425.
DOI: Google ScholarCross Ref - [64] . 2018. Dynamic network embedding by modeling triadic closure process. In Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence, (AAAI-18), the 30th Innovative Applications of Artificial Intelligence (IAAI-18), and the 8th AAAI Symposium on Educational Advances in Artificial Intelligence (EAAI-18), New Orleans, Louisiana, USA, February 2–7, 2018, and (Eds.). AAAI Press, 571–578. https://www.aaai.org/ocs/index.php/AAAI/AAAI18/paper/view/16572.Google ScholarCross Ref
- [65] . 2021. Progresses and challenges in link prediction. CoRR abs/2102.11472 (2021).
arXiv:2102.11472 https://arxiv.org/abs/2102.11472.Google Scholar - [66] . 2018. High-order proximity preserved embedding for dynamic networks. IEEE Trans. Knowl. Data Eng. 30, 11 (2018), 2134–2144.
DOI: Google ScholarDigital Library - [67] . 2016. Scalable temporal latent space inference for link prediction in dynamic social networks. IEEE Trans. Knowl. Data Eng. 28, 10 (2016), 2765–2777.
DOI: Google ScholarDigital Library
Index Terms
- BehaviorNet: A Fine-grained Behavior-aware Network for Dynamic Link Prediction
Recommendations
Graph Sequential Neural ODE Process for Link Prediction on Dynamic and Sparse Graphs
WSDM '23: Proceedings of the Sixteenth ACM International Conference on Web Search and Data MiningLink prediction on dynamic graphs is an important task in graph mining. Existing approaches based on dynamic graph neural networks (DGNNs) typically require a significant amount of historical data (interactions over time), which is not always available ...
Community-enhanced Link Prediction in Dynamic Networks
The growing popularity of online social networks is quite evident nowadays and provides an opportunity to allow researchers in finding solutions for various practical applications. Link prediction is the technique of understanding network structure and ...
Few-shot Link Prediction in Dynamic Networks
WSDM '22: Proceedings of the Fifteenth ACM International Conference on Web Search and Data MiningDynamic link prediction, which aims at forecasting future edges of a node in a dynamic network, is an important problem in network science and has a wide range of real-world applications. A key property of dynamic networks is that new nodes and links ...
Comments