Jianhuan Zhuo
ABSTRACT
Knowledge graph (KG) entity typing focuses on inferring possible entity type instances, which is a significant subtask of knowledge graph completion (KGC). Existing entity typing methods usually exploit the entity representation to model the transmission between entities and their types, which cannot fully explore the fine-grained entity typing on identifying the semantic type of an entity. To address these issues, we propose Neighborhood-Attention Neural Fine-Grained Entity Typing (AttEt), which considers the neighborhood information of the entities from KGs to bridge entities and their types together. In this paper, AttEt first develops a type-specific attention mechanism to aggregate the neighborhood knowledge of the given entity with type-specific weights. These weights are beneficial to capture various characteristics for different types of the entity, and further imply the complex correlation among these fine-grained types. Then, AttEt adaptively integrates the aggregated neighbor-level representation with entity inherent embedding to calculate the matching score between the entity and its candidate type. Besides, many entities are sparse in their relations with other entities in KGs, which makes the entity typing task more challenging. To solve this problem, we present a smooth strategy on relation-sparsity entities to improve the robustness of the model. Extensive experiments on two real-world datasets (Freebase and YAGO) show that AttEt significantly outperforms state-of-the-art baselines in the HITS@1 by 2.11% on Freebase and by 8.42% on YAGO, respectively.
Supplemental Material
- Kurt Bollacker, Colin Evans, Praveen Paritosh, Tim Sturge, and Jamie Taylor. 2008a. Freebase:a collaboratively created graph database for structuring human knowledge, In In Proceedings of KDD. In processings of the 2008 ACM SIGMOD international Conference on Management of Data , 1247--1250.Google Scholar
- Kurt D. Bollacker, Colin Evans, Praveen Paritosh, Tim Sturge, and Jamie Taylor. 2008b. Freebase: a collaboratively created graph database for structuring human knowledge. In Sigmod Conference .Google ScholarDigital Library
- A. Bordes, N. Usunier, and A. Garcia-Dur!'"aan. 2013. Translating Embeddings for Modeling Multi-relational Data. In In Proceedings of NIPS. 2787--2795.Google Scholar
- Tongfei Chen, Yunmo Chen, and Benjamin Van Durme. 2020. Hierarchical Entity Typing via Multi-level Learning to Rank. In Proceedings of ACL . Association for Computational Linguistics, Online, 8465--8475.Google ScholarCross Ref
- Eunsol Choi, Omer Levy, Yejin Choi, and Luke Zettlemoyer. 2018. Ultra-Fine Entity Typing. In Proceedings of ACL . Association for Computational Linguistics, Melbourne, Australia, 87--96.Google ScholarCross Ref
- Wanyun Cui, Yanghua Xiao, Haixun Wang, Yangqiu Song, Seung-won Hwang, and Wei Wang. 2017. KBQA: learning question answering over QA corpora and knowledge bases. Proceedings of the VLDB Endowment , Vol. 10, 5 (Jan. 2017), 565--576. https://doi.org/10.14778/3055540.3055549Google ScholarDigital Library
- John Duchi, Elad Hazan, and Yoram Singer. 2011. Adaptive subgradient methods for online learning and stochastic optimization. Journal of machine learning research , Vol. 12, 7 (2011).Google ScholarDigital Library
- Hannaneh Hajishirzi, Leila Zilles, Daniel S. Weld, and Luke Zettlemoyer. 2013. Joint Coreference Resolution and Named-Entity Linking with Multi-Pass Sieves. In Proceedings of EMNLP . Association for Computational Linguistics, Seattle, Washington, USA, 289--299.Google Scholar
- Xiao Huang, Jingyuan Zhang, Dingcheng Li, and Ping Li. 2019. Knowledge Graph Embedding Based Question Answering. In Proceedings of WSDM (WSDM '19). Association for Computing Machinery, New York, NY, USA, 105--113.Google ScholarDigital Library
- Prachi Jain, Pankaj Kumar, Soumen Chakrabarti, et al. 2018. Type-sensitive knowledge base inference without explicit type supervision. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers). 75--80.Google ScholarCross Ref
- G. Ji, S. He, L. Xu, K. Liu, and J. Zhao. 2015. Knowledge graph embedding via dynamic mapping matrix. In In Proceedings of ACL . 687--696.Google Scholar
- Hailong Jin, Lei Hou, Juanzi Li, and Tiansi Dong. 2018. Attributed and Predictive Entity Embedding for Fine-Grained Entity Typing in Knowledge Bases. In Proceedings of ACL , , Emily M. Bender, Leon Derczynski, and Pierre Isabelle (Eds.). Association for Computational Linguistics, 282--292.Google Scholar
- Hailong Jin, Lei Hou, Juanzi Li, and Tiansi Dong. 2019. Fine-Grained Entity Typing via Hierarchical Multi Graph Convolutional Networks. In Proceedings of EMNLP, Kentaro Inui, Jing Jiang, Vincent Ng, and Xiaojun Wan (Eds.). Association for Computational Linguistics, 4968--4977.Google ScholarCross Ref
- Thomas N. Kipf and Max Welling. 2017. Semi-Supervised Classification with Graph Convolutional Networks. In Proceedings of ICLR . OpenReview.net.Google Scholar
- J. Lehmann, R. Isele, and M. Jakob. 2015. DBpedia: A large-scale, multilingual knowledge base extracted from wikipedia. Semantic Web , Vol. 6, 2 (2015), 167--195.Google ScholarCross Ref
- Jian Li, Yong Liu, Rong Yin, Hua Zhang, Lizhong Ding, and Weiping Wang. 2018. Multi-Class Learning: From Theory to Algorithm. NeurIPS , Vol. 31 (2018), 1593--1602.Google Scholar
- Yankai Lin, Zhiyuan Liu, Xuan Zhu, Xuan Zhu, and Xuan Zhu. 2015. Learning entity and relation embeddings for knowledge graph completion. In Proceedings of AAAI. 2181--2187.Google ScholarDigital Library
- Shuman Liu, Hongshen Chen, Zhaochun Ren, Yang Feng, Qun Liu, and Dawei Yin. 2018. Knowledge Diffusion for Neural Dialogue Generation. In Proceedings of ACL . Association for Computational Linguistics, Melbourne, Australia, 1489--1498.Google ScholarCross Ref
- Yong Liu, Shizhong Liao, Shali Jiang, Lizhong Ding, Hailun Lin, and Weiping Wang. 2020. Fast Cross-Validation for Kernel-Based Algorithms. IEEE Transactions on Pattern Analysis and Machine Intelligence , Vol. 42, 5 (2020), 1083--1096. https://doi.org/10.1109/TPAMI.2019.2892371Google Scholar
- Changsung Moon, Paul Jones, and Nagiza F Samatova. 2017. Learning entity type embeddings for knowledge graph completion. In Proceedings of CIKM . 2215--2218.Google ScholarDigital Library
- Dai Quoc Nguyen, Thanh Vu, Tu Dinh Nguyen, Dat Quoc Nguyen, and Dinh Phung. 2019. A Capsule Network-based Embedding Model for Knowledge Graph Completion and Search Personalization. In Proceedings of NAACL . Association for Computational Linguistics, Minneapolis, Minnesota, 2180--2189.Google ScholarCross Ref
- Maximilian Nickel, Lorenzo Rosasco, and Tomaso Poggio. 2016. Holographic Embeddings of Knowledge Graphs. In Proceedings of AAAI (AAAI'16). AAAI Press, 1955--1961.Google ScholarCross Ref
- Maximilian Nickel, Volker Tresp, and Hans-Peter Kriegel. 2011a. A three-way model for collective learning on multi-relational data.. In Proceedings of ICML , Vol. 11. 809--816.Google Scholar
- Maximilian Nickel, Volker Tresp, and Hans Peter Kriegel. 2011b. A Three-Way Model for Collective Learning on Multi-Relational Data. In International Conference on Machine Learning, ICML 2011. 809--816.Google Scholar
- Yasumasa Onoe, Michael Boratko, and Greg Durrett. 2021. Modeling Fine-Grained Entity Types with Box Embeddings. CoRR , Vol. abs/2101.00345 (2021).Google Scholar
- Patrick Pantel, Thomas Lin, and Michael Gamon. 2012. Mining Entity Types from Query Logs via User Intent Modeling. In Proceedings of ACL. Association for Computational Linguistics, Jeju Island, Korea, 563--571.Google Scholar
- Heiko Paulheim and Christian Bizer. 2013. Type Inference on Noisy RDF Data. In Proceedings of the 12th International Semantic Web Conference - Part I (ISWC '13). Springer-Verlag, 510--525. https://doi.org/10.1007/978--3--642--41335--3_32Google ScholarCross Ref
- Qingyu Guo, Fuzhen Zhuang, Chuan Qin, Hengshu Zhu, Xing Xie, Hui Xiong, and Qing He. 2020. A Survey on Knowledge Graph-Based Recommender Systems . arXiv:2003.00911 [cs, stat] (Feb. 2020). http://arxiv.org/abs/2003.00911 arXiv: 2003.00911.Google Scholar
- Fabian M. Suchanek, Gjergji Kasneci, and Gerhard Weikum. 2007. Yago: a core of semantic knowledge. In Proceedings of WWW. 697--706.Google ScholarDigital Library
- Thé o Trouillon and Maximilian Nickel. 2017. Complex and Holographic Embeddings of Knowledge Graphs: A Comparison. CoRR , Vol. abs/1707.01475 (2017).Google Scholar
- Thé o Trouillon, Johannes Welbl, Sebastian Riedel, É ric Gaussier, and Guillaume Bouchard. 2016. Complex Embeddings for Simple Link Prediction. In Proceedings of ICML. 2071--2080.Google Scholar
- Petar Velickovic, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò , and Yoshua Bengio. 2017. Graph Attention Networks. CoRR , Vol. abs/1710.10903 (2017).Google Scholar
- Hongwei Wang, Hongyu Ren, and Jure Leskovec. 2020. Entity Context and Relational Paths for Knowledge Graph Completion. CoRR , Vol. abs/2002.06757 (2020). arxiv: 2002.06757Google Scholar
- Xiang Wang, Dingxian Wang, Canran Xu, Xiangnan He, Yixin Cao, and Tat-Seng Chua. 2018. Explainable Reasoning over Knowledge Graphs for Recommendation . arXiv:1811.04540 [cs] (Nov. 2018). http://arxiv.org/abs/1811.04540 arXiv: 1811.04540.Google Scholar
- Z. Wang, J. Zhang, J. Feng, and Z. Chen. 2014. Knowledge graph embedding by translating on hyperplanes. In In Proceedings of AAAI . 1112--1119.Google Scholar
- H. Xiao, M. Huang, and X. Zhu. 2016. TransG: A Generative Model for Knowledge Graph Embedding. In Proceedings of ACL . 2316--2325.Google Scholar
- Ruobing Xie, Zhiyuan Liu, Jia Jia, Huanbo Luan, and Maosong Sun. 2016. Representation Learning of Knowledge Graphs with Entity Descriptions. In Proceedings of AAAI . AAAI Press, 2659--2665.Google ScholarCross Ref
- Ji Xin, Yankai Lin, Zhiyuan Liu, and Maosong Sun. 2018. Improving Neural Fine-Grained Entity Typing With Knowledge Attention. In Proceedings of AAAI , , Sheila A. McIlraith and Kilian Q. Weinberger (Eds.). AAAI Press, 5997--6004.Google ScholarCross Ref
- Bo Xu, Yi Zhang, Jiaqing Liang, Yanghua Xiao, Seung-won Hwang, and Wei Wang. 2016. Cross-Lingual Type Inference. In Proceedings of DASFAA (Lecture Notes in Computer Science), Vol. 9642. Springer, 447--462.Google ScholarCross Ref
- Peng Xu and Denilson Barbosa. 2018. Neural Fine-Grained Entity Type Classification with Hierarchy-Aware Loss. In Proceedings of ACL . Association for Computational Linguistics, New Orleans, Louisiana, 16--25.Google ScholarCross Ref
- Yadollah Yaghoobzadeh, Heike Adel, and Hinrich Schü tze. 2018. Corpus-Level Fine-Grained Entity Typing. J. Artif. Intell. Res. , Vol. 61 (2018), 835--862.Google ScholarCross Ref
- Bishan Yang, Wen-tau Yih, Xiaodong He, Jianfeng Gao, and Li Deng. 2014. Embedding Entities and Relations for Learning and Inference in Knowledge Bases. CoRR , Vol. abs/1412.6575 (2014).Google Scholar
- Limin Yao, Sebastian Riedel, and Andrew McCallum. 2013. Universal schema for entity type prediction. In Proceedings of workshop on Automated knowledge base construction (AKBC '13). Association for Computing Machinery, New York, NY, USA, 79--84.Google ScholarDigital Library
- Dani Yogatama, Daniel Gillick, and Nevena Lazic. 2015. Embedding Methods for Fine Grained Entity Type Classification. In Proceedings of ACL . Association for Computational Linguistics, Beijing, China, 291--296.Google ScholarCross Ref
- Chuxu Zhang, Huaxiu Yao, Chao Huang, Meng Jiang, Zhenhui Li, and Nitesh V. Chawla. 2020. Few-Shot Knowledge Graph Completion . Proceedings of AAAI Conference on Artificial Intelligence , Vol. 34, 03 (April 2020), 3041--3048. Number: 03.Google Scholar
- Richong Zhang, Fanshuang Kong, Chenyue Wang, and Yongyi Mao. 2018. Embedding of Hierarchically Typed Knowledge Bases. In Proceedings of AAAI , , Sheila A. McIlraith and Kilian Q. Weinberger (Eds.). AAAI Press, 2046--2053.Google Scholar
- Yu Zhao, Anxiang Zhang, Ruobing Xie, Kang Liu, and Xiaojie Wang. 2020. Connecting Embeddings for Knowledge Graph Entity Typing. In Proceedings of ACL . Association for Computational Linguistics, Online.Google ScholarCross Ref
- Ben Zhou, Daniel Khashabi, Chen-Tse Tsai, and Dan Roth. 2018. Zero-Shot Open Entity Typing as Type-Compatible Grounding. In Proceedings of ACL. Association for Computational Linguistics, Brussels, Belgium, 2065--2076.Google Scholar
Index Terms
- A Neighborhood-Attention Fine-grained Entity Typing for Knowledge Graph Completion
Recommendations
Learning Entity Type Embeddings for Knowledge Graph Completion
CIKM '17: Proceedings of the 2017 ACM on Conference on Information and Knowledge ManagementMissing data is a severe problem for algorithms that operate over knowledge graphs (KGs). Most previous research in KG completion has focused on the problem of inferring missing entities and missing relation types between entities. However, in addition ...
Fine-Grained Entity Typing for Relation-Sparsity Entities
Database Systems for Advanced ApplicationsAbstractThis paper works on fine-grained entity typing without using external knowledge for Knowledge Graphs (KGs). Aiming at identifying the semantic type of an entity, this task has been studied predominantly in KGs. Provided with dense enough relations ...
It’s All in the Name: Entity Typing Using Multilingual Language Models
The Semantic Web: ESWC 2022 Satellite EventsAbstractThe entity type information in Knowledge Graphs (KGs) of different languages plays an important role in a wide range of Natural Language Processing applications. However, the entity types in KGs are often incomplete. Multilingual entity typing is ...
Comments