Skip to main content
SpringerLink
Log in

An attention-based CNN-LSTM model for subjectivity detection in opinion-mining

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Opinion-mining generally refers to analyzing opinions on various topics available in the form of text. It is an essential operation of natural language processing since it enables efficient decision-making and planning for users and businesses. Opinion-mining can be made more comfortable and more effective by initially performing subjectivity detection, i.e., identifying the text as subjective or objective. An opinion-mining model can better identify the opinions present in the remaining subjective statements by removing objective statements. With this reasoning, we present an efficient subjectivity detection model for improved accuracy in Opinion-mining. The model uses a strategic combination of convolutional neural network (CNN) and long short-term memory (LSTM). CNN and LSTM are state-of-the-art deep learning models that can efficiently process textual data and identify inherent connections and patterns with varying abstraction levels. The proposed work combines the strengths of both these models in an ensemble model. Effectiveness of the model is enhanced with the incorporation of an attention network. In the present task, the sentences are represented as word embeddings that include sentiment information and part-of-speech. The proposed model is applied on two movie review datasets, and its performance is evaluated compared with state-of-the-art methods. Various performance indexes have validated the superiority of the proposed model in the opinion-mining task.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Ravi K, Ravi V (2015) A survey on opinion-mining and sentiment analysis: tasks, approaches and applications. Knowl-Based Syst 89:14–46

    Article  Google Scholar 

  2. Sagnika S, Pattanaik A, Mishra BSP, Meher SK (2020) A review on multi-lingual sentiment analysis by machine learning methods. J Eng Sci Technol Rev 13(2):154–166

    Article  Google Scholar 

  3. Chaturvedi I, Cambria E, Welsch RE, Herrera F (2018) Distinguishing between facts and opinions for sentiment analysis: Survey and challenges. Inf Fusion 44:65–77

    Article  Google Scholar 

  4. Satapathy R, Chaturvedi I, Cambria E, Ho SS, Na JC (2017) Subjectivity detection in nuclear energy tweets. Comput Sist 21(4):657–664

    Google Scholar 

  5. Bhunia AK, Konwer A, Bhunia AK, Bhowmick A, Roy PP, Pal U (2019) Script identification in natural scene image and video frames using an attention based Convolutional-LSTM network. Pattern Recogn 85:172–184

    Article  Google Scholar 

  6. Fang Y, Gao J, Huang C, Peng H, Wu R (2019) Self multi-head attention-based convolutional neural networks for fake news detection. PLoS ONE 14(9):e0222713

    Article  Google Scholar 

  7. Keshavarz HR, Saniee Abadeh M (2018) MHSubLex: Using metaheuristic methods for subjectivity classification of microblogs. J AI Data Min 6(2):341–353

    Google Scholar 

  8. Xuan HNT, Le AC (2012) Linguistic features for subjectivity classification. In: 2012 international conference on asian language processing, pp 17–20. IEEE

  9. Banea C, Mihalcea R, Wiebe J (2014) Sense-level subjectivity in a multilingual setting. Comput Speech Lang 28(1):7–19

    Article  Google Scholar 

  10. Amini I, Karimi S, Shakery A (2019) Cross-lingual subjectivity detection for resource lean languages. In: Proceedings of the tenth workshop on computational approaches to subjectivity, sentiment and social media analysis, pp 81–90

  11. Kamal A (2013) Subjectivity classification using machine learning techniques for mining feature-opinion pairs from web opinion sources. arXiv preprint arXiv:1312.6962

  12. Wiebe J, Riloff E (2005) Creating subjective and objective sentence classifiers from unannotated texts. International conference on intelligent text processing and computational linguistics. Springer, Berlin, pp 486–497

    Google Scholar 

  13. Lin C, He Y, Everson R (2011) Sentence subjectivity detection with weakly-supervised learning. In: Proceedings of 5th international joint conference on natural language processing, pp 1153–1161

  14. Chaturvedi I, Cambria E, Poria S, Bajpai R (2016) Bayesian deep convolution belief networks for subjectivity detection. In: 2016 IEEE 16th international conference on data mining workshops (ICDMW), pp 916–923. IEEE

  15. Chaturvedi I, Ragusa E, Gastaldo P, Zunino R, Cambria E (2018) Bayesian network based extreme learning machine for subjectivity detection. J Franklin Inst 355(4):1780–1797

    Article  MathSciNet  Google Scholar 

  16. Rustamov, S. (2018). A hybrid system for subjectivity analysis. Advances in fuzzy systems, 2018.

  17. Zhao Z, Wu Y (2016) Attention-based convolutional neural networks for sentence classification. In: INTERSPEECH, pp 705–709

  18. Yang M, Tu W, Wang J, Xu F, Chen X (2017) Attention based LSTM for target dependent sentiment classification. In: Thirty-First AAAI conference on artificial intelligence

  19. Zhang S, Xu X, Pang Y, Han J (2020) Multi-layer attention based CNN for target-dependent sentiment classification. Neural Process Lett 51(3):2089–2103

    Article  Google Scholar 

  20. Gan C, Wang L, Zhang Z, Wang Z (2020) Sparse attention based separable dilated convolutional neural network for targeted sentiment analysis. Knowledge-Based Syst 188:104827

    Article  Google Scholar 

  21. Lei Z, Yang Y, Yang M (2018) Sentiment lexicon enhanced attention-based LSTM for sentiment classification. In: Thirty-second AAAI conference on artificial intelligence

  22. Zhang X, Chen F, Huang R (2018) A combination of RNN and CNN for attention-based relation classification. Proc Comput Sci 131:911–917

    Article  Google Scholar 

  23. Liu J, Yang Y, Lv S, Wang J, Chen H (2019) Attention-based BiGRU-CNN for Chinese question classification. J Ambient Intell Hum Comput, pp 1–12

  24. Jain D, Kumar A, Garg G (2020) Sarcasm detection in mash-up language using soft-attention based bi-directional LSTM and feature-rich CNN. Appl Soft Comput, p 106198

  25. Cho K, Van Merriënboer B, Gulcehre C, Bahdanau D, Bougares F, Schwenk H, Bengio Y (2014) Learning phrase representations using RNN encoder-decoder for statistical machine translation. arXiv preprint arXiv:1406.1078

  26. Attention Mechanism in Deep Learning | Attention Model Keras. Available: https://www.analyticsvidhya.com/blog/2019/11/comprehensive-guide-attention-mechanism-deep-learning/

  27. Bahdanau D, Cho K, Bengio Y (2014) Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473

  28. Intuitive Understanding of Attention Mechanism in Deep Learning | by Harshall Lamba | Towards Data Science. Available: https://towardsdatascience.com/intuitive-understanding-of-attention-mechanism-in-deep-learning-6c9482aecf4f

  29. LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521(7553):436–444

    Article  Google Scholar 

  30. Liu P, Qiu X, Huang X (2016) Recurrent neural network for text classification with multi-task learning. arXiv preprint arXiv:1605.05101

  31. Arel I, Rose DC, Karnowski TP (2010) Deep machine learning-a new frontier in artificial intelligence research [research frontier]. IEEE Comput Intell Mag 5(4):13–18

    Article  Google Scholar 

  32. Sagnika S, Mishra BSP, Meher SK (2020) Improved method of word embedding for efficient analysis of human sentiments. Multim Tools Appl 79:32389–32413

    Article  Google Scholar 

  33. Pang B, Lee L (2004) A sentimental education: Sentiment analysis using subjectivity summarization based on minimum cuts. arXiv preprint cs/0409058

  34. Kim Y (2014) Convolutional neural networks for sentence classification. arXiv preprint arXiv:1408.5882

  35. Wang S, Manning C (2013) Fast dropout training. In: International conference on machine learning, pp 118–126

  36. Das N, Sagnika S (2020) A subjectivity detection-based approach to sentiment analysis. Machine learning and information processing. Springer, Singapore, pp 149–160

    Chapter  Google Scholar 

  37. Sindhu C, Sasmal B, Gupta R, Prathipa J (2021) Subjectivity detection for sentiment analysis on twitter data. Artificial intelligence techniques for advanced computing applications. Springer, Singapore, pp 467–476

    Chapter  Google Scholar 

  38. Islam MZ, Islam MM, Asraf A (2020) A combined deep CNN-LSTM network for the detection of novel coronavirus (COVID-19) using X-ray images. Inf Med Unlocked 20:100412

    Article  Google Scholar 

Download references

Funding

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

Author information

Authors and Affiliations

  1. School of Computer Engineering, Kalinga Institute of Industrial Technology Deemed To Be University, Bhubaneswar, Odisha, 751024, India

    Santwana Sagnika & Bhabani Shankar Prasad Mishra

  2. Systems Science and Informatics Unit, Indian Statistical Institute, Bangalore Centre, 8th Mile, Mysore Road, RVCE Post, Bangalore, Karnataka, 560059, India

    Saroj K. Meher

Authors
  1. Santwana Sagnika
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bhabani Shankar Prasad Mishra
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Saroj K. Meher
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Santwana Sagnika.

Ethics declarations

Conflict of interest

The authors have no conflicts of interest to declare that are relevant to the content of this article.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Sagnika, S., Mishra, B.S.P. & Meher, S.K. An attention-based CNN-LSTM model for subjectivity detection in opinion-mining. Neural Comput & Applic 33, 17425–17438 (2021). https://doi.org/10.1007/s00521-021-06328-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-021-06328-5

Keywords

Search

Navigation