Erik Cambria
Frank Z. Xing
ABSTRACT
Deep learning has unlocked new paths towards the emulation of the peculiarly-human capability of learning from examples. While this kind of bottom-up learning works well for tasks such as image classification or object detection, it is not as effective when it comes to natural language processing. Communication is much more than learning a sequence of letters and words: it requires a basic understanding of the world and social norms, cultural awareness, commonsense knowledge, etc.; all things that we mostly learn in a top-down manner. In this work, we integrate top-down and bottom-up learning via an ensemble of symbolic and subsymbolic AI tools, which we apply to the interesting problem of polarity detection from text. In particular, we integrate logical reasoning within deep learning architectures to build a new version of SenticNet, a commonsense knowledge base for sentiment analysis.
Supplemental Material
- Sanders Analytics. 2015. Sanders Dataset. (2015). http://www.sananalytics.com/labGoogle Scholar
- Stefano Baccianella, Andrea Esuli, and Fabrizio Sebastiani. 2010. SentiWordNet 3.0: an enhanced lexical resource for sentiment analysis and opinion mining. In LREC, Vol. 10. 2200--2204.Google Scholar
- Marco Baroni, Silvia Bernardini, Adriano Ferraresi, and Eros Zanchetta. 2009. The WaCky wide web: a collection of very large linguistically processed web-crawled corpora. Language resources and evaluation, Vol. 43, 3 (2009), 209--226.Google Scholar
- James Bergstra and Yoshua Bengio. 2012. Random search for hyper-parameter optimization. The Journal of Machine Learning Research, Vol. 13, 1 (2012), 281--305.Google ScholarDigital Library
- M Bradley and P Lang. 1999. Affective Norms for English Words (ANEW): Stimuli, Instruction Manual and Affective Ratings. Technical Report. The Center for Research in Psychophysiology, University of Florida.Google Scholar
- Erik Cambria, Jie Fu, Federica Bisio, and Soujanya Poria. 2015. AffectiveSpace 2: Enabling Affective Intuition for Concept-Level Sentiment Analysis. In AAAI. Austin, 508--514.Google Scholar
- Erik Cambria, Thomas Mazzocco, Amir Hussain, and Chris Eckl. 2011. Sentic Medoids: Organizing Affective Common Sense Knowledge in a Multi-Dimensional Vector Space. LNCS 6677.Google Scholar
- Erik Cambria, Soujanya Poria, Alexander Gelbukh, and Mike Thelwall. 2017. Sentiment Analysis is a Big Suitcase. IEEE Intelligent Systems, Vol. 32, 6 (2017), 74--80.Google ScholarCross Ref
- Erik Cambria, Soujanya Poria, Devamanyu Hazarika, and Kenneth Kwok. 2018. SenticNet 5: Discovering conceptual primitives for sentiment analysis by means of context embeddings. In AAAI. 1795--1802.Google Scholar
- Erik Cambria, Yangqiu Song, Haixun Wang, and Newton Howard. 2014. Semantic Multi-Dimensional Scaling for Open-Domain Sentiment Analysis. IEEE Intelligent Systems, Vol. 29, 2 (2014), 44--51.Google ScholarCross Ref
- Sabrina Cerini, Valentina Compagnoni, Alice Demontis, Maicol Formentelli, and G Gandini. 2007. Micro-WNOp: A gold standard for the evaluation of automatically compiled lexical resources for opinion mining. Language resources and linguistic theory: Typology, second language acquisition, English linguistics (2007), 200--210.Google Scholar
- Zhuang Chen and Tieyun Qian. 2019. Transfer Capsule Network for Aspect Level Sentiment Classification. In ACL. 547--556.Google Scholar
- Lingjia Deng and Janyce Wiebe. 2015. MPQA 3.0: An entity/event-level sentiment corpus. In Proceedings of the 2015 conference of the North American chapter of the association for computational linguistics: human language technologies. 1323--1328.Google ScholarCross Ref
- Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In NAACL-HLT. 4171--4186.Google Scholar
- Cicero Nogueira dos Santos and Maira Gatti. 2014. Deep convolutional neural networks for sentiment analysis of short texts. In COLING. 69--78.Google Scholar
- Umberto Eco. 1984. Semiotics and Philosophy of Language .Indiana University Press.Google Scholar
- Umberto Eco. 1997. Kant and the Platypus: Essays on Language and Cognition .Bompiani.Google Scholar
- Andrea Esuli and Fabrizio Sebastiani. 2006. Sentiwordnet: A publicly available lexical resource for opinion mining. In Proceedings of LREC, Vol. 6. 417--422.Google Scholar
- R Evans and E Grefenstette. 2018. Learning explanatory rules from noisy data. Journal of Artificial Intelligence Research, Vol. 61 (2018), 1--64.Google ScholarCross Ref
- Marco Ferrarotti, Sergio Decherchi, and Walter Rocchia. 2019. Finding Principal Paths in Data Space. IEEE Transactions on Neural Networks and Learning Systems, Vol. 30, 8 (2019), 2449--2462.Google ScholarCross Ref
- Alec Go, Richa Bhayani, and Lei Huang. 2009. Twitter sentiment classification using distant supervision. CS224N project report, Stanford, Vol. 1, 12 (2009).Google Scholar
- Minqing Hu and Bing Liu. 2004. Mining and summarizing customer reviews. In Proceedings of the tenth ACM SIGKDD international conference on Knowledge discovery and data mining. ACM, 168--177.Google ScholarDigital Library
- Clayton J Hutto and Eric GIlbert. 2014. VADER: A parsimonious rule-based model for sentiment analysis of social media text. In ICWSM. 216--225.Google Scholar
- Ray Jackendoff. 1976. Toward an explanatory semantic representation. Linguistic Inquiry, Vol. 7, 1 (1976), 89--150.Google Scholar
- Ray Jackendoff. 1983. Semantics and cognition .MIT Press.Google Scholar
- J.J. Katz and J.A. Fodor. 1963. The structure of a Semantic Theory. Language, Vol. 39 (1963), 170--210.Google ScholarCross Ref
- Gerhard Kremer, Katrin Erk, Sebastian Padó, and Stefan Thater. 2014. What Substitutes Tell Us-Analysis of an" All-Words" Lexical Substitution Corpus. In EACL. 540--549.Google Scholar
- Qiao Liu, Haibin Zhang, Yifu Zeng, Ziqi Huang, and Zufeng Wu. 2018. Content Attention Model for Aspect Based Sentiment Analysis. In WWW. 1023--1032.Google Scholar
- Yukun Ma, Haiyun Peng, and Erik Cambria. 2018. Targeted aspect-based sentiment analysis via embedding commonsense knowledge into an attentive LSTM. In AAAI. 5876--5883.Google Scholar
- Diana McCarthy and Roberto Navigli. 2007. SemEval-2007 task 10: English lexical substitution task. In SemEval. 48--53.Google Scholar
- Oren Melamud, Omer Levy, Ido Dagan, and Israel Ramat-Gan. 2015. A Simple Word Embedding Model for Lexical Substitution. In VS@ HLT-NAACL. 1--7.Google Scholar
- Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. 2013. Distributed representations of words and phrases and their compositionality. In NIPS. 3111--3119.Google Scholar
- Marvin Minsky. 1975. A framework for representing knowledge. The psychology of computer vision, Patrick Winston (Ed.). McGraw-Hill, New York.Google Scholar
- Saif M Mohammad and Peter D Turney. 2013. Crowdsourcing a word--emotion association lexicon. Computational Intelligence, Vol. 29, 3 (2013), 436--465.Google ScholarCross Ref
- Preslav Nakov, Alan Ritter, Sara Rosentha, Fabrizio Sebastiani, and Veselin Stoyanov. 2016. SemEval-2016 Task 4: Sentiment Analysis in Twitter. In Proceedings of the 9th International Workshop on Semantic Evaluation (SemEval 2016). 1--18.Google ScholarCross Ref
- Preslav Nakov, Sara Rosenthal, Zornitsa Kozareva, Veselin Stoyanov, Alan Ritter, and Theresa Wilson. 2013. SemEval-2013 Task 2: Sentiment Analysis in Twitter. In Second Joint Conference on Lexical and Computational Semantics (*SEM), Volume 2: Proceedings of the Seventh International Workshop on Semantic Evaluation (SemEval 2013). Association for Computational Linguistics, 312--320.Google Scholar
- Finn Nielsen. 2011. A new ANEW: Evaluation of a word list for sentiment analysis in microblogs. CoRR, Vol. abs/1103.2903 (2011). arxiv: 1103.2903 http://arxiv.org/abs/1103.2903Google Scholar
- Bo Pang, Lillian Lee, and Shivakumar Vaithyanathan. 2002. Thumbs up?: Sentiment classification using machine learning techniques. In EMNLP, Vol. 10. 79--86.Google ScholarDigital Library
- Soujanya Poria, Erik Cambria, and Alexander Gelbukh. 2016. Aspect Extraction for Opinion Mining with a Deep Convolutional Neural Network. Knowledge-Based Systems, Vol. 108 (2016), 42--49.Google ScholarDigital Library
- Soujanya Poria, Erik Cambria, Alexander Gelbukh, Federica Bisio, and Amir Hussain. 2015. Sentiment Data Flow Analysis by Means of Dynamic Linguistic Patterns. IEEE Computational Intelligence Magazine, Vol. 10, 4 (2015), 26--36.Google ScholarDigital Library
- Sara Rosenthal, Preslav Nakov, Svetlana Kiritchenko, Saif Mohammad, Alan Ritter, and Veselin Stoyanov. 2015. SemEval-2015 Task 10: Sentiment Analysis in Twitter. In Proceedings of the 9th International Workshop on Semantic Evaluation (SemEval 2015). 451--463.Google ScholarCross Ref
- David Rumelhart and Andrew Ortony. 1977. The representation of knowledge in memory. Schooling and the acquisition of knowledge. Erlbaum, Hillsdale, NJ.Google Scholar
- Ivan Sag, Timothy Baldwin, Francis Bond, Ann Copestake, and Dan Flickinger. 2002. Multiword Expressions: A Pain in the Neck for NLP. In CICLing. 1--15.Google Scholar
- Hassan Saif, Miriam Fernandez, Yulan He, and Harith Alani. 2013. Evaluation datasets for Twitter sentiment analysis: a survey and a new dataset, the STS-Gold. In AI*IA.Google Scholar
- Roger Schank. 1972. Conceptual dependency: A theory of natural language understanding. Cognitive Psychology, Vol. 3 (1972), 552--631.Google ScholarCross Ref
- Richard Socher, Danqi Chen, Christopher D Manning, and Andrew Ng. 2013a. Reasoning with neural tensor networks for knowledge base completion. In NIPS. 926--934.Google Scholar
- Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D Manning, Andrew Y Ng, and Christopher Potts. 2013b. Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of the 2013 conference on empirical methods in natural language processing. 1631--1642.Google Scholar
- Richard Socher, Alex Perelygin, Jean Y Wu, Jason Chuang, Christopher D Manning, Andrew Y Ng, and Christopher Potts. 2013c. Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank. In EMNLP. 1642--1654.Google Scholar
- Robert Speer and Catherine Havasi. 2012. ConceptNet 5: A Large Semantic Network for Relational Knowledge. Theory and Applications of Natural Language Processing. Chapter 6.Google Scholar
- Carlo Strapparava and Alessandro Valitutti. 2004. WordNet-Affect: An Affective Extension of WordNet. In LREC. 1083--1086.Google Scholar
- Yosephine Susanto, Andrew Livingstone, Bee Chin Ng, and Erik Cambria. 2020. The Hourglass Model Revisited. IEEE Intelligent Systems, Vol. 35, 5 (2020).Google Scholar
- Maite Taboada, Julian Brooke, Milan Tofiloski, Kimberly Voll, and Manfred Stede. 2011. Lexicon-based methods for sentiment analysis. Computational linguistics, Vol. 37, 2 (2011), 267--307.Google Scholar
- Duyu Tang, Furu Wei, Bing Qin, Ting Liu, and Ming Zhou. 2014. Coooolll: A deep learning system for twitter sentiment classification. In SemEval. 208--212.Google Scholar
- Mike Thelwall, Kevan Buckley, Georgios Paltoglou, Di Cai, and Arvid Kappas. 2010. Sentiment strength detection in short informal text. Journal of the American society for information science and technology, Vol. 61, 12 (2010), 2544--2558.Google ScholarCross Ref
- Po-Wei Wang, Priya Donti, Bryan Wilder, and Zico Kolter. 2019. SATNet: Bridging deep learning and logical reasoning using a differentiable satisfiability solver. https://arxiv.org/abs/1905.12149. In ICML. 6545--6554.Google Scholar
- Janyce Wiebe, Theresa Wilson, and Claire Cardie. 2005. Annotating Expressions of Opinions and Emotions in Language. Language Resources and Evaluation, Vol. 39, 2 (2005), 165--210.Google ScholarCross Ref
- Anna Wierzbicka. 1996. Semantics: Primes and Universals .Oxford University Press.Google ScholarCross Ref
- Theresa Wilson, Paul Hoffmann, Swapna Somasundaran, Jason Kessler, Janyce Wiebe, Yejin Choi, Claire Cardie, Ellen Riloff, and Siddharth Patwardhan. 2005. OpinionFinder: A system for subjectivity analysis. Proceedings of HLT/EMNLP 2005 Interactive Demonstrations. 34--35.Google ScholarDigital Library
- Lei Xu. 1997. Bayesian Ying--Yang machine, clustering and number of clusters. Pattern Recognition Letters, Vol. 18, 11 (1997), 1167--1178.Google ScholarDigital Library
- F Yang, Z Yang, and W Cohen. 2017. Differentiable learning of logical rules for knowledge base reasoning. In NIPS. 2319--2328.Google Scholar
- Wei Zhao, Haiyun Peng, Steffen Eger, Erik Cambria, and Min Yang. 2019. Towards scalable and reliable capsule networks for challenging NLP applications. In ACL. 1549--1559.Google Scholar
- Xiaodan Zhu, Svetlana Kiritchenko, and Saif Mohammad. 2014. NRC-canada-2014: Recent improvements in the sentiment analysis of tweets. In Proceedings of the 8th international workshop on semantic evaluation (SemEval 2014). 443--447.Google ScholarCross Ref
Index Terms
- SenticNet 6: Ensemble Application of Symbolic and Subsymbolic AI for Sentiment Analysis
Recommendations
Extracting domain-specific opinion words for sentiment analysis
MICAI'12: Proceedings of the 11th Mexican international conference on Advances in Computational Intelligence - Volume Part IIIn this paper, we consider opinion word extraction, one of the key problems in sentiment analysis. Sentiment analysis (or opinion mining) is an important research area within computational linguistics. Opinion words, which form an opinion lexicon, ...
Joint sentiment/topic model for sentiment analysis
CIKM '09: Proceedings of the 18th ACM conference on Information and knowledge managementSentiment analysis or opinion mining aims to use automated tools to detect subjective information such as opinions, attitudes, and feelings expressed in text. This paper proposes a novel probabilistic modeling framework based on Latent Dirichlet ...
Sentiment-Specific Representation Learning for Document-Level Sentiment Analysis
WSDM '15: Proceedings of the Eighth ACM International Conference on Web Search and Data MiningIn this paper, we propose a representation learning research framework for document-level sentiment analysis. Given a document as the input, document-level sentiment analysis aims to automatically classify its sentiment/opinion (such as thumbs up or ...
Comments