Dmitriy Bespalov
ABSTRACT
In this paper, we propose an efficient embedding for modeling higher-order (n-gram) phrases that projects the n-grams to low-dimensional latent semantic space, where a classification function can be defined. We utilize a deep neural network to build a unified discriminative framework that allows for estimating the parameters of the latent space as well as the classification function with a bias for the target classification task at hand. We apply the framework to large-scale sentimental classification task. We present comparative evaluation of the proposed method on two (large) benchmark data sets for online product reviews. The proposed method achieves superior performance in comparison to the state of the art.
- A. Agresti. Analysis of Ordinal Categorical Data. John Wiley and Sons Inc., 2010.Google Scholar
Cross Ref
- B. Bai, J. Weston, R. Collobert, D. Grangier, K. Sadamasa, Y. Qi, O. Chapelle, and K. Weinberger. Supervised semantic indexing. In Proceeding of the 18th ACM Conference on Information and Knowledge Management, pages 187--196. ACM, 2009. Google ScholarDigital Library
- Y. Bengio. Learning Deep Architectures for AI. Now Publishers Inc., Hanover, MA, USA, 2009. Google ScholarDigital Library
- Y. Bengio, R. Ducharme, P. Vincent, and D. D. E. R. Operationnelle. A neural probabilistic language model. Journal of Machine Learning Research, 3:1137--1155, 2000. Google ScholarDigital Library
- D. Blei, A. Ng, and M. Jordan. Latent dirichlet allocation. The Journal of Machine Learning Research, 3:993--1022, 2003. Google ScholarDigital Library
- J. Blitzer, M. Dredze, and F. Pereira. Biographies, bollywood, boomboxes and blenders: Domain adaptation for sentiment classification. In In ACL, pages 187--205, 2007.Google Scholar
- L. Bottou. Stochastic learning. In O. Bousquet and U. von Luxburg, editors, Advanced Lectures on Machine Learning, Lecture Notes in Artificial Intelligence, LNAI 3176, pages 146--168. Springer Verlag, Berlin, 2004.Google Scholar
- S. Brody and N. Elhadad. An unsupervised aspect-sentiment model for online reviews. In Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics, HLT '10, pages 804--812, Stroudsburg, PA, USA, 2010. Association for Computational Linguistics. Google ScholarDigital Library
- R. Collobert and J. Weston. A unified architecture for natural language processing: Deep neural networks with multitask learning. In International Conference on Machine Learning, ICML, 2008. Google ScholarDigital Library
- H. Cui, V. Mittal, and M. Datar. Comparative experiments on sentiment classification for online product reviews. In proceedings of the 21st national conference on Artificial intelligence - Volume 2, pages 1265--1270. AAAI Press, 2006. Google ScholarDigital Library
- S. Deerwester, S. T. Dumais, G. W. Furnas, T. K. Landauer, and R. Harshman. Indexing by latent semantic analysis. Journal of The American Society for Information Science, 41(6):391--407, 1990.Google ScholarCross Ref
- G. Ganu, N. Elhadad, and A. Marian. Beyond the stars: improving rating predictions using review text content. 2009.Google Scholar
- V. Hatzivassiloglou and K. R. McKeown. Predicting the semantic orientation of adjectives. In Proceedings of the 35th Annual Meeting of the Association for Computational Linguistics and Eighth Conference of the European Chapter of the Association for Computational Linguistics, ACL '98, pages 174--181, Stroudsburg, PA, USA, 1997. Association for Computational Linguistics. Google ScholarDigital Library
- T. Hofmann. Probabilistic latent semantic indexing. In Proceedings of the 22nd annual international ACM SIGIR conference on Research and development in information retrieval, pages 50--57. ACM Press New York, NY, USA, 1999. Google ScholarDigital Library
- P. Kuksa, P.-H. Huang, and V. Pavlovic. Scalable algorithms for string kernels with inexact matching. In NIPS, 2008. Spotlight Presentation. Acceptance rate: 123/1022 (12%).Google Scholar
- C. S. Leslie, E. Eskin, J. Weston, and W. S. Noble. Mismatch string kernels for SVM protein classification. In NIPS, pages 1417--1424, 2002.Google ScholarDigital Library
- Y. Mansour, M. Mohri, and A. Rostamizadeh. Domain adaptation with multiple sources. In NIPS'08, pages 1041--1048, 2008.Google Scholar
- F. Morin. Hierarchical probabilistic neural network language model. aistats--05. In AISTATS, pages 246--252, 2005.Google Scholar
- G. Paltoglou and M. Thelwall. A study of information retrieval weighting schemes for sentiment analysis. In Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics, ACL '10, pages 1386--1395, Stroudsburg, PA, USA, 2010. Association for Computational Linguistics. Google ScholarDigital Library
- B. Pang and L. Lee. Opinion mining and sentiment analysis. Foundations and Trends in Information Retrieval, 2(1--2):1--135, 2008. Google ScholarDigital Library
- R. Salakhutdinov and G. Hinton. Semantic hashing. Int. J. Approx. Reasoning, 50:969--978, July 2009. Google ScholarDigital Library
- C. Sauper, A. Haghighi, and R. Barzilay. Incorporating content structure into text analysis applications. In EMNLP'10, pages 377--387, 2010. Google ScholarDigital Library
- P. Turney and M. Littman. Measuring praise and criticism: Inference of semantic orientation from association. ACM Transactions on Information Systems, 21:315--346, 2003. Google ScholarDigital Library
- H. Wang, Y. Lu, and C. Zhai. Latent aspect rating analysis on review text data: a rating regression approach. In Proceedings of the 16th ACM SIGKDD international conference on Knowledge discovery and data mining, KDD '10, pages 783--792, New York, NY, USA, 2010. ACM. Google ScholarDigital Library
- J. Weston, S. Bengio, and N. Usunier. Large scale image annotation: Learning to rank with joint word-image embeddings. Machine learning, 81(1):21--35, 2010. Google ScholarDigital Library
- J. Weston, C. Leslie, E. Ie, D. Zhou, A. Elisseeff, and W. S. Noble. Semi-supervised protein classification using cluster kernels. Bioinformatics, 21(15):3241--3247, 2005. Google ScholarDigital Library
Index Terms
- Sentiment classification based on supervised latent n-gram analysis
Recommendations
Joint sentiment/topic model for sentiment analysis
CIKM '09: Proceedings of the 18th ACM conference on Information and knowledge managementSentiment analysis or opinion mining aims to use automated tools to detect subjective information such as opinions, attitudes, and feelings expressed in text. This paper proposes a novel probabilistic modeling framework based on Latent Dirichlet ...
Exploring weakly supervised latent sentiment explanations for aspect-level review analysis
CIKM '13: Proceedings of the 22nd ACM international conference on Information & Knowledge ManagementIn sentiment analysis, aspect-level review analysis has been an important task because it can catalogue, aggregate, or summarize various opinions according to a product's properties. In this paper, we explore a new concept for aspect-level review ...
Semi-supervised probabilistic sentiment analysis: merging labeled sentences with unlabeled reviews to identify sentiment
ASIST '13: Proceedings of the 76th ASIS&T Annual Meeting: Beyond the Cloud: Rethinking Information BoundariesDocument level sentiment analysis, the task of determining whether the sentiment expressed in a document is positive or negative, is commonly performed by supervised methods. As with all supervised tasks, obtaining training data for these methods can be ...
Comments