Elsevier

Neurocomputing

Volume 419, 2 January 2021, Pages 344-356
Neurocomputing

A multi-task learning model for Chinese-oriented aspect polarity classification and aspect term extraction

https://doi.org/10.1016/j.neucom.2020.08.001Get rights and content

Highlights

  • Proposing a model for the joint task of aspect-based sentiment analysis.

  • The model has the ability to handle both Chinese and English reviews.

  • The model also integrates the domain-adapted BERT model for an enhancement.

  • The model achieves state-of-the-art performance on seven ABSA datasets.

Abstract

Aspect-based sentiment analysis (ABSA) task is a fine-grained task of natural language processing and consists of two subtasks: aspect term extraction (ATE) and aspect polarity classification (APC). Most of the related works merely focus on the subtask of Chinese aspect term polarity inferring and fail to emphasize the research of Chinese-oriented ABSA multi-task learning. Based on the local context focus (LCF) mechanism, this paper firstly proposes a multi-task learning model for Chinese-oriented aspect-based sentiment analysis, namely LCF-ATEPC. Compared with other models, this model equips the capability of extracting aspect term and inferring aspect term polarity synchronously. The experimental results on four Chinese review datasets outperform state-of-the-art performance on the ATE and APC subtask. And by integrating the domain-adapted BERT model, LCF-ATEPC achieves the state-of-the-art performance of ATE and APC in the most commonly used SemEval-2014 task4 Restaurant and Laptop datasets. Moreover, this model is effective to analyze both Chinese and English reviews collaboratively and the experimental results on a multilingual mixed dataset prove its effectiveness.

Introduction

Aspect-based sentiment analysis [1], [2], [3] (ABSA) is a fine-grained task compared with traditional sentiment analysis, which aims to automatically extract the aspect terms and predict the polarities of them. Cambria et al. [4] discussed the importance of sentiment analysis and introduced its application scenarios in many fields. For example, given a restaurant review: “The dessert at this restaurant is delicious but the service is poor.” the full-designed model for ABSA needs to extract the aspects “dessert”, “service” and correctly inferring about their polarity. In this review, the consumers’ opinions on “dessert” and “service” are not consistent, with positive and negative sentiment polarity respectively.

Generally, aspect terms and their polarity need to be manually labeled before running the APC task. However, most of the proposed models for aspect-based sentiment analysis tasks only focus on improving the classification accuracy of aspect polarity and ignore the research of the Chinese ATE subtask. Therefore, when conducting transfer learning on aspect-based sentiment analysis, those proposed models often fall into the dilemma of lacking aspect extraction method on targeted tasks because there is not enough research support.

The aspect term extraction (ATE) and aspect polarity classification (APC) are the subtasks of ABSA. The APC is a kind of text classification task. There are a large number of deep learning-based models have been proposed to solve the APC subtasks, such as the models [5], [6], [7], [8], [9], [10] based on long short-term memory (LSTM) and the methodologies [11], [12] based on the Transformer network [13]. The purpose of the APC is to predict the exact sentiment polarity of different aspect terms, rather than fuzzily analyz the overall sentiment polarity on the sentence-level or document-level. In the APC task, the polarities are most usually classified into three categories: positive, negative, and neutral. The sentiment polarity classified based on aspects can better mine the fine-grained emotional tendency in reviews or tweets, thus providing a more accurate reference for decision-makers.

Consistent with the named entity recognition [14] (NER) task, the ATE is recognized as a subtask sequence tagging, and aims to extract aspect terms from the reviews or tweets. In most approaches [15], [16], [17], the ATE is studied independently, away from the APC task. The ATE models first segment a review into separate tokens and then infer whether the tokens belong to any aspect. The tokens may be labeled in different forms in different studies, but most of the works adopts the IOB2 3 labels to annotate tokens. Table 1 are several cases of joint task of ATE and APC.

To extract aspect terms from the text efficiently and analyze the sentiment polarity of aspects collaboratively, this paper proposes a multi-task learning model for aspect-based sentiment analysis. Multilingual processing is an important research orientation of natural language processing. The LCF-ATEPC4 model is a novel multilingual and multi-task model. Apart from achieving state-of-the-art performance in commonly used SemEval-2014 task4 datasets, the experimental results in four Chinese review datasets also validate that this model has a strong ability to be satisfy the demands of multilingual ABSA. The proposed model is based on multi-head self-attention (MHSA), integrating the bidirectional encoder representations from transformers (BERT) [18] and the local context focus mechanism. Training on a small amount of annotated data of aspect terms and their polarities, the model can be adapted to a large-scale dataset, automatically extracting the aspect terms and predicting the sentiment polarities. In this way, the model can discover the unknown aspects and avoids the tedious and huge cost of manually annotating all aspects and polarities. It is of great significance for the field-specific ABSA.

The main contributions of this article are as follows:

  • 1.

    For the first time, we study the multi-task learning for APC and ATE using Chinese and multilingual reviews, and proposes a novel model to solve the APC and ATE synchronously.

  • 2.

    We adapt the self-attention and local context focus techniques to improve collaborative training of ATE and APC, and experimental results on Chinese and multilingual datasets demonstrate our model significantly outperforms state-of-the-art performance compare to existing approaches.

  • 3.

    The proposed model integrates domain-adapted BERT to improve both the performance of ATE and APC. The experiments indicate the great potential of domain-adapted pretrained model and bring considerable effect especially the F1 score of ATE task.

Section snippets

Related works

Many existing approaches regarded the ATE and APC as independent tasks and studied separately. Accordingly, this section will introduce the related works of ATE, APC, and multi-task learning works in this section.

Methodology

The methodology of LCF-ATEPC is based on self-attention and the local context focus mechanism. Moreover, the domain-adapted BERT model integrated into the LCF-ATEPC provides an enhancement for model performance. This section introduces the architecture and methodology of LCF-ATEPC. The modules for APC and ATE are introduced independently. The contents are organized by the hierarchy of the network layer.

Datasets and hyperparameters setting

We prepare seven ABSA datasets to comprehensively evaluate the performance of LCF-ATEPC, involving four Chinese review datasets [48], [49], [31] (Car, Phone, Notebook, Camera) and three most commonly used English ABSA datasets (the Laptops and Restaurant datasets of SemEval-2014 Task4 subtask2 [1], and an ACL Twitter social dataset [50]). The polarity of each aspect term on the Laptops, Restaurants, and Twitter datasets may be “positive”, “neutral”, or “negative”, the conflicting labels of

Conclusion and future works

The ATE and APC were usually treated as independent tasks in previous studies since multi-task learning of ATE and APC failed to attract enough attention. Besides, the studies about Chinese language-oriented ABSA are not sufficient and urgent to be proposed and developed. To address the above problems, this paper proposes a multi-task learning model LCF-ATEPC, which is based on self-attention and local context focus and applies the pretrained BERT to the Chinese ABSA for the first time. Not

CRediT authorship contribution statement

Heng Yang: Conceptualization, Methodology, Software, Writing - original draft, Writing - review & editing. Biqing Zeng: Conceptualization, Data curation, Writing - original draft. Jianhao Yang: Investigation, Resources, Visualization, Validation. Youwei Song: Data curation, Writing - review & editing. Ruyang Xu: Formal analysis, Validation.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Acknowledgement

Thanks to the anonymous reviewers and the scholars who helped us. This research is funded by National Natural Science Foundation of China, Multi-modal Brain-Computer Interface and Its Application in Patients with Consciousness Disorder, Project approval number: 61876067; The Guangdong General Colleges and Universities Special Projects in Key Areas of Artificial Intelligence of China, Research and Application of Key Techniques of Sentiment analysis, project number: 2019KZDZX1033. And this

Heng Yang is pursuing a master’s degree in computer science and technology from South China Normal University. His research interests are natural language processing, sentiment classification, and entity extraction.

References (52)

  • E. Cambria

    Affective computing and sentiment analysis

    IEEE Intelligent Systems

    (2016)
  • D.-T. Vo, Y. Zhang, Target-dependent twitter sentiment classification with rich automatic features, in: Proceedings of...
  • J. Wagner, P. Arora, S. Cortes, U. Barman, D. Bogdanova, J. Foster, L. Tounsi, Dcu: Aspect-based polarity...
  • D. Tang, B. Qin, X. Feng, T. Liu, Effective LSTMs for target-dependent sentiment classification, in: Proceedings of...
  • Y. Wang et al.

    Attention-based lstm for aspect-level sentiment classification

  • D. Ma et al.

    Interactive attention networks for aspect-level sentiment classification, in

  • F. Fan et al.

    Multi-grained attention network for aspect-level sentiment classification, in

  • Y. Song, J. Wang, T. Jiang, Z. Liu, Y. Rao, Attentional encoder network for targeted sentiment classification, arXiv...
  • B. Zeng et al.

    Lcf: A local context focus mechanism for aspect-based sentiment classification

    Applied Sciences

    (2019)
  • A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, I. Polosukhin, Attention is all you...
  • E. F. T. K. Sang, F. De Meulder, Introduction to the conll-2003 shared task: language-independent named entity...
  • P. Chen et al.

    Recurrent attention network on memory for aspect sentiment analysis

  • W. Xue, T. Li, Aspect based sentiment analysis with gated convolutional networks, in: Proceedings of the 56th Annual...
  • J. Devlin, M.-W. Chang, K. Lee, K. Toutanova, BERT: Pre-training of deep bidirectional transformers for language...
  • S. Poria, E. Cambria, L.-W. Ku, C. Gui, A. Gelbukh, A rule-based approach to aspect extraction from product reviews,...
  • Q. Liu, Z. Gao, B. Liu, Y. Zhang, Automated rule selection for aspect extraction in opinion mining, in: Proceedings of...
  • Cited by (90)

    View all citing articles on Scopus

    Heng Yang is pursuing a master’s degree in computer science and technology from South China Normal University. His research interests are natural language processing, sentiment classification, and entity extraction.

    Biqing Zeng is a professor of the School of Software at South China Normal University. He obtained his Ph.D. in Computer Science at Central South University. His interests are natural language processing, artificial intelligence, big data.

    Jianhao Yang is pursuing a master’s degree in the Software School of South China Normal University. His research interests are natural language processing, sentiment analysis, and recommendation systems.

    Youwei Song obtained a master’s degree in the School of Data and Computer science at Sun Yat-Sen University. His research interests are sentiment analysis and natural language processing.

    Ruyang Xu is pursuing a master’s degree in the School of Software at South China Normal University. His research interests include natural language processing, artificial intelligence.

    1

    ORCID: 0000-0002-6831-196X

    2

    ORCID: 0000-0001-9088-4759

    View full text