Abstract
An N-gram language model aims at capturing statistical word order dependency information from corpora. Although the concept of language models has been applied extensively to handle a variety of NLP problems with reasonable success, the standard model does not incorporate semantic information, and consequently limits its applicability to semantic problems such as word sense disambiguation. We propose a framework that integrates semantic information into the language model schema, allowing a system to exploit both syntactic and semantic information to address NLP problems. Furthermore, acknowledging the limited availability of semantically annotated data, we discuss how the proposed model can be learned without annotated training examples. Finally, we report on a case study showing how the semantics-enhanced language model can be applied to unsupervised word sense disambiguation with promising results.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Banerjee, S., Pedersen, T.: Extended gloss overlaps as a measure of semantic relatedness. In: Proceedings of the Eighteenth International Joint Conference on Artificial Intelligence, pp. 805–810 (2003)
Baum, L.E.: An Inequality and Associated Maximization in Statistical Estimation for Probabilistic Functions of Markov Processes. Inequalities 627(3), 1–8 (1972)
Bellegarda, J.: Exploiting latent semantic information in statistical language modeling. Proceedings of IEEE 88(8), 1279–1296 (2000)
Brody, S., Navigli, R., Lapata, M.: Ensemble Methods for Unsupervised WSD. In: Proceedings of the ACL/COLING, pp. 97–104 (2006)
Brown, P.F., et al.: Class-based n-gram models of natural language. Computational Linguistics 18(4), 467–479 (1992)
Chueh, C.H., Wang, H.M., Chien, J.T.: A Maximum Entropy Approach for Semantic Language Modeling. Computational Linguistics and Chinese Language Processing 11(1), 37–56 (2006)
Cutting, D., et al.: A practical part-of-speech tagger. In: Proceedings of ANLP-1992, Trento, Italy, pp. 133–140 (1992)
Dempster, A.D., Laird, N.M., Rubin, D.B.: Maximum likelihood for incomplete data via the EM algorithm. Journal of Royal Statistical Society Series B 39, 1–38 (1977)
Galley, M., McKeown, K.: Improving Word Sense Disambiguation in Lexical Chaining. In: Proceedings of the Eighteenth International Joint Conference on Artificial Intelligence, pp. 1486–1488 (2003)
Griffiths, T., et al.: Integrating Topics and Syntax. In: Proceedings of the Advances in Neural Information Processing Systems (2004)
Hoste, V., et al.: Parameter optimization for machine-learning of word sense disambiguation. Language Engineering 8(4), 311–325 (2002)
Knight, K., et al.: Unsupervised Analysis for Decipherment Problems. In: Proceedings of the ACL-COLING (2006)
Koehn, P., Knight, K.: Estimating word translation probabilities from unrelated monolingual corpora using the EM algorithm. In: Proceedings of the AAAI, pp. 711–715 (2000)
Lin, S.d., Knight, K.: Discovering the linear writing order of a two-dimensional ancient hieroglyphic script. Artificial Intelligence 170(4-5) (2006)
McCarthy, D., et al.: Finding predominant word senses in untagged text. In: Proceedings of the 42nd Annual Meeting of the Association for Computational Linguistics, Barcelona, Spain (2004)
Mihalcea, R.: Unsupervised Large-Vocabulary Word Sense Disambiguation with Graph-based Algorithms for Sequence Data Labeling. In: Proceedings of the Joint Conference on Human Language Technology / Empirical Methods in Natural Language Processing (HLT/EMNLP) (2005)
Navigli, R., Velardi, P.: Structural Semantic Interconnections: a Knowledge-Based Approach to Word Sense Disambiguation. IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI) 27(7), 1063–1074 (2005)
Rapaport, W.J.: Holism, Conceptual-Role Semantics, and Syntactic Semantics. Minds and Machines 12(1), 3–59 (2002)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2008 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Lin, Sd., Verspoor, K. (2008). A Semantics-Enhanced Language Model for Unsupervised Word Sense Disambiguation. In: Gelbukh, A. (eds) Computational Linguistics and Intelligent Text Processing. CICLing 2008. Lecture Notes in Computer Science, vol 4919. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-78135-6_24
Download citation
DOI: https://doi.org/10.1007/978-3-540-78135-6_24
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-78134-9
Online ISBN: 978-3-540-78135-6
eBook Packages: Computer ScienceComputer Science (R0)