A multi-task learning model for Chinese-oriented aspect polarity classification and aspect term extraction
Graphical abstract
Introduction
Aspect-based sentiment analysis [1], [2], [3] (ABSA) is a fine-grained task compared with traditional sentiment analysis, which aims to automatically extract the aspect terms and predict the polarities of them. Cambria et al. [4] discussed the importance of sentiment analysis and introduced its application scenarios in many fields. For example, given a restaurant review: “The dessert at this restaurant is delicious but the service is poor.” the full-designed model for ABSA needs to extract the aspects “dessert”, “service” and correctly inferring about their polarity. In this review, the consumers’ opinions on “dessert” and “service” are not consistent, with positive and negative sentiment polarity respectively.
Generally, aspect terms and their polarity need to be manually labeled before running the APC task. However, most of the proposed models for aspect-based sentiment analysis tasks only focus on improving the classification accuracy of aspect polarity and ignore the research of the Chinese ATE subtask. Therefore, when conducting transfer learning on aspect-based sentiment analysis, those proposed models often fall into the dilemma of lacking aspect extraction method on targeted tasks because there is not enough research support.
The aspect term extraction (ATE) and aspect polarity classification (APC) are the subtasks of ABSA. The APC is a kind of text classification task. There are a large number of deep learning-based models have been proposed to solve the APC subtasks, such as the models [5], [6], [7], [8], [9], [10] based on long short-term memory (LSTM) and the methodologies [11], [12] based on the Transformer network [13]. The purpose of the APC is to predict the exact sentiment polarity of different aspect terms, rather than fuzzily analyz the overall sentiment polarity on the sentence-level or document-level. In the APC task, the polarities are most usually classified into three categories: positive, negative, and neutral. The sentiment polarity classified based on aspects can better mine the fine-grained emotional tendency in reviews or tweets, thus providing a more accurate reference for decision-makers.
Consistent with the named entity recognition [14] (NER) task, the ATE is recognized as a subtask sequence tagging, and aims to extract aspect terms from the reviews or tweets. In most approaches [15], [16], [17], the ATE is studied independently, away from the APC task. The ATE models first segment a review into separate tokens and then infer whether the tokens belong to any aspect. The tokens may be labeled in different forms in different studies, but most of the works adopts the IOB2 3 labels to annotate tokens. Table 1 are several cases of joint task of ATE and APC.
To extract aspect terms from the text efficiently and analyze the sentiment polarity of aspects collaboratively, this paper proposes a multi-task learning model for aspect-based sentiment analysis. Multilingual processing is an important research orientation of natural language processing. The LCF-ATEPC4 model is a novel multilingual and multi-task model. Apart from achieving state-of-the-art performance in commonly used SemEval-2014 task4 datasets, the experimental results in four Chinese review datasets also validate that this model has a strong ability to be satisfy the demands of multilingual ABSA. The proposed model is based on multi-head self-attention (MHSA), integrating the bidirectional encoder representations from transformers (BERT) [18] and the local context focus mechanism. Training on a small amount of annotated data of aspect terms and their polarities, the model can be adapted to a large-scale dataset, automatically extracting the aspect terms and predicting the sentiment polarities. In this way, the model can discover the unknown aspects and avoids the tedious and huge cost of manually annotating all aspects and polarities. It is of great significance for the field-specific ABSA.
The main contributions of this article are as follows:
- 1.
For the first time, we study the multi-task learning for APC and ATE using Chinese and multilingual reviews, and proposes a novel model to solve the APC and ATE synchronously.
- 2.
We adapt the self-attention and local context focus techniques to improve collaborative training of ATE and APC, and experimental results on Chinese and multilingual datasets demonstrate our model significantly outperforms state-of-the-art performance compare to existing approaches.
- 3.
The proposed model integrates domain-adapted BERT to improve both the performance of ATE and APC. The experiments indicate the great potential of domain-adapted pretrained model and bring considerable effect especially the F1 score of ATE task.
Section snippets
Related works
Many existing approaches regarded the ATE and APC as independent tasks and studied separately. Accordingly, this section will introduce the related works of ATE, APC, and multi-task learning works in this section.
Methodology
The methodology of LCF-ATEPC is based on self-attention and the local context focus mechanism. Moreover, the domain-adapted BERT model integrated into the LCF-ATEPC provides an enhancement for model performance. This section introduces the architecture and methodology of LCF-ATEPC. The modules for APC and ATE are introduced independently. The contents are organized by the hierarchy of the network layer.
Datasets and hyperparameters setting
We prepare seven ABSA datasets to comprehensively evaluate the performance of LCF-ATEPC, involving four Chinese review datasets [48], [49], [31] (Car, Phone, Notebook, Camera) and three most commonly used English ABSA datasets (the Laptops and Restaurant datasets of SemEval-2014 Task4 subtask2 [1], and an ACL Twitter social dataset [50]). The polarity of each aspect term on the Laptops, Restaurants, and Twitter datasets may be “positive”, “neutral”, or “negative”, the conflicting labels of
Conclusion and future works
The ATE and APC were usually treated as independent tasks in previous studies since multi-task learning of ATE and APC failed to attract enough attention. Besides, the studies about Chinese language-oriented ABSA are not sufficient and urgent to be proposed and developed. To address the above problems, this paper proposes a multi-task learning model LCF-ATEPC, which is based on self-attention and local context focus and applies the pretrained BERT to the Chinese ABSA for the first time. Not
CRediT authorship contribution statement
Heng Yang: Conceptualization, Methodology, Software, Writing - original draft, Writing - review & editing. Biqing Zeng: Conceptualization, Data curation, Writing - original draft. Jianhao Yang: Investigation, Resources, Visualization, Validation. Youwei Song: Data curation, Writing - review & editing. Ruyang Xu: Formal analysis, Validation.
Declaration of Competing Interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
Acknowledgement
Thanks to the anonymous reviewers and the scholars who helped us. This research is funded by National Natural Science Foundation of China, Multi-modal Brain-Computer Interface and Its Application in Patients with Consciousness Disorder, Project approval number: 61876067; The Guangdong General Colleges and Universities Special Projects in Key Areas of Artificial Intelligence of China, Research and Application of Key Techniques of Sentiment analysis, project number: 2019KZDZX1033. And this
Heng Yang is pursuing a master’s degree in computer science and technology from South China Normal University. His research interests are natural language processing, sentiment classification, and entity extraction.
References (52)
- et al.
Improving sentiment analysis via sentence type classification using bilstm-crf and cnn
Expert Systems with Applications
(2017) - et al.
Aspect extraction for opinion mining with a deep convolutional neural network
Knowledge-Based Systems
(2016) - et al.
Textual sentiment analysis via three different attention convolutional neural networks and cross-modality consistent regression
Neurocomputing
(2018) - et al.
Learning multi-grained aspect target sequence for chinese sentiment analysis
Knowledge-Based Systems
(2018) - et al.
Improving aspect-based sentiment analysis via aligning aspect embedding
Neurocomputing
(2020) - et al.
Knowledge-enhanced neural networks for sentiment analysis of chinese reviews
Neurocomputing
(2019) - et al.
Principal curvature for infrared small target detection
Infrared Physics & Technology
(2015) - et al.
SemEval-2014 task 4: Aspect based sentiment analysis
- et al.
SemEval-2015 task 12: Aspect based sentiment analysis
- M. Pontiki, D. Galanis, H. Papageorgiou, I. Androutsopoulos, S. Manandhar, M. AL-Smadi, M. Al-Ayyoub, Y. Zhao, B. Qin,...
Affective computing and sentiment analysis
IEEE Intelligent Systems
Attention-based lstm for aspect-level sentiment classification
Interactive attention networks for aspect-level sentiment classification, in
Multi-grained attention network for aspect-level sentiment classification, in
Lcf: A local context focus mechanism for aspect-based sentiment classification
Applied Sciences
Recurrent attention network on memory for aspect sentiment analysis
Cited by (90)
Construction of an aspect-level sentiment analysis model for online medical reviews
2023, Information Processing and ManagementFuzzy Local Information C -Means based clustering and Fractional Dwarf Mongoose optimization enabled deep learning for relevant document retrieval
2023, Engineering Applications of Artificial IntelligenceReconstructing graph networks by using new target representation for aspect-based sentiment analysis
2023, Knowledge-Based SystemsA prompt model with combined semantic refinement for aspect sentiment analysis
2023, Information Processing and ManagementLeveraging hierarchical language models for aspect-based sentiment analysis on financial data
2023, Information Processing and ManagementA new hazard event classification model via deep learning and multifractal
2023, Computers in Industry
Heng Yang is pursuing a master’s degree in computer science and technology from South China Normal University. His research interests are natural language processing, sentiment classification, and entity extraction.
Biqing Zeng is a professor of the School of Software at South China Normal University. He obtained his Ph.D. in Computer Science at Central South University. His interests are natural language processing, artificial intelligence, big data.
Jianhao Yang is pursuing a master’s degree in the Software School of South China Normal University. His research interests are natural language processing, sentiment analysis, and recommendation systems.
Youwei Song obtained a master’s degree in the School of Data and Computer science at Sun Yat-Sen University. His research interests are sentiment analysis and natural language processing.
Ruyang Xu is pursuing a master’s degree in the School of Software at South China Normal University. His research interests include natural language processing, artificial intelligence.