Abstract
Online forums and social media platforms are increasingly being used to discuss topics of varying polarities where different people take different stances. Several methodologies for automatic stance detection from text have been proposed in literature. To our knowledge, there has not been any systematic investigation towards their reproducibility, and their comparative performances. In this work, we explore the reproducibility of several existing stance detection models, including both neural models and classical classifier-based models. Through experiments on two datasets – (i) the popular SemEval microblog dataset, and (ii) a set of health-related online news articles – we also perform a detailed comparative analysis of various methods and explore their shortcomings.
S. Ghosh, P. Singhania and S. Singh—Equal contribution by authors.
K. Rudra—The work was done when the author was a Research Associate at IIT Kharagpur.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Agrawal, A., An, A.: Unsupervised emotion detection from text using semantic and syntactic relations. In: Proceedings of the IEEE/WIC/ACM Joint Conferences on Web Intelligence and Intelligent Agent Technology (2012)
Bodenreider, O.: The unified medical language system (UMLS): integrating biomedical terminology. Nucleic Acids Res. 32(Suppl. 1), D267–D270 (2004)
Chen, W.F., Ku, L.W.: UTCNN: a deep learning model of stance classification on social media text. arXiv preprint arXiv:1611.03599 (2016)
Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018)
Dey, K., Shrivastava, R., Kaushik, S.: Twitter stance detection - a subjectivity and sentiment polarity inspired two-phase approach. In: Proceedings of the IEEE Conference on Data Mining Workshops (ICDMW) (2017)
Dey, K., Shrivastava, R., Kaushik, S.: Topical stance detection for Twitter: a two-phase LSTM model using attention. In: Pasi, G., Piwowarski, B., Azzopardi, L., Hanbury, A. (eds.) ECIR 2018. LNCS, vol. 10772, pp. 529–536. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-76941-7_40
Du, J., Xu, R., He, Y., Gui, L.: Stance classification with target-specific neural attention networks. In: Proceedings of the International Joint Conferences on Artificial Intelligence (IJCAI) (2017)
Faulkner, A.: Automated classification of stance in student essays: an approach using stance target information and the Wikipedia link-based measure. Science 376(12), 86 (2014)
Ferreira, W., Vlachos, A.: Emergent: a novel data-set for stance classification. In: Proceedings of the NAACL-HLT (2016)
HaCohen-kerner, Y., Ido, Z., Ya’akobov, R.: Stance classification of tweets using skip char ngrams. In: Altun, Y., et al. (eds.) ECML PKDD 2017. LNCS (LNAI), vol. 10536, pp. 266–278. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-71273-4_22
Han, B., Baldwin, T.: Lexical normalisation of short text messages: makn sens a# twitter. In: Proceedings of the Annual Meeting of the Association for Computational Linguistics: Human Language Technologies (2011)
Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)
Jang, M., Allan, J.: Improving automated controversy detection on the web. In: Proceedings of the ACM SIGIR Conference (2016)
Kim, Y.: Convolutional neural networks for sentence classification. CoRR abs/1408.5882 (2014). http://arxiv.org/abs/1408.5882
Küçük, D., Can, F.: Stance detection on tweets: an SVM-based approach. arXiv preprint arXiv:1803.08910 (2018)
Lai, M., Patti, V., Ruffo, G., Rosso, P.: Stance evolution and Twitter interactions in an italian political debate. In: Silberztein, M., Atigui, F., Kornyshova, E., Métais, E., Meziane, F. (eds.) NLDB 2018. LNCS, vol. 10859, pp. 15–27. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-91947-8_2
Liu, H., Li, S., Zhou, G.: Two-target stance detection with target-related zone modeling. In: Zhang, S., Liu, T.-Y., Li, X., Guo, J., Li, C. (eds.) CCIR 2018. LNCS, vol. 11168, pp. 170–182. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01012-6_14
Mohammad, S., Kiritchenko, S., Sobhani, P., Zhu, X., Cherry, C.: Semeval-2016 task 6: detecting stance in tweets. In: Proceedings of the International Workshop on Semantic Evaluation (SemEval-2016), pp. 31–41 (2016)
Padó, S., Noh, T.G., Stern, A., Wang, R., Zanoli, R.: Design and realization of a modular architecture for textual entailment. Natural Lang. Eng. 21(2), 167–200 (2015)
Sen, A., Sinha, M., Mannarswamy, S., Roy, S.: Stance classification of multi-perspective consumer health information. In: Proceedings of the ACM India Joint Conference on Data Science and Management of Data (2018)
Sobhani, P., Inkpen, D., Zhu, X.: A dataset for multi-target stance detection. In: Proceedings of the Conference of the European Chapter of the Association for Computational Linguistics (2017)
Sobhani, P., Inkpen, D., Zhu, X.: Exploring deep neural networks for multitarget stance detection. Comput. Intell. 35(1), 82–97 (2019)
Sulis, E., Farías, D.I.H., Rosso, P., Patti, V., Ruffo, G.: Figurative messages and affect in Twitter: differences between# irony,# sarcasm and# not. Knowl.-Based Syst. 108, 132–143 (2016)
Taulé, M., et al.: Overview of the task on stance and gender detection in tweets on Catalan independence at IberEval 2017. In: Proceedings of the Workshop on Evaluation of Human Language Technologies for Iberian Languages (2017)
Wang, R., Zhou, D., Jiang, M., Si, J., Yang, Y.: A survey on opinion mining: from stance to product aspect. IEEE Access 7, 41101–41124 (2019)
Wei, P., Lin, J., Mao, W.: Multi-target stance detection via a dynamic memory-augmented network. In: Proceedings of the ACM SIGIR Conference (2018)
Wei, W., Zhang, X., Liu, X., Chen, W., Wang, T.: pkudblab at semeval-2016 task 6: a specific convolutional neural network system for effective stance detection. In: SemEval@NAACL-HLT (2016)
Zarrella, G., Marsh, A.: MITRE at semeval-2016 task 6: transfer learning for stance detection. arXiv preprint arXiv:1606.03784 (2016)
Zubiaga, A., Aker, A., Bontcheva, K., Liakata, M., Procter, R.: Detection and resolution of rumours in social media: a survey. ACM Comput. Surv. (CSUR) 51(2), 32 (2018)
Acknowledgement
The work is partially supported by a project titled “Building Healthcare Informatics Systems Utilising Web Data” funded by Department of Science & Technology, Government of India.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Ghosh, S., Singhania, P., Singh, S., Rudra, K., Ghosh, S. (2019). Stance Detection in Web and Social Media: A Comparative Study. In: Crestani, F., et al. Experimental IR Meets Multilinguality, Multimodality, and Interaction. CLEF 2019. Lecture Notes in Computer Science(), vol 11696. Springer, Cham. https://doi.org/10.1007/978-3-030-28577-7_4
Download citation
DOI: https://doi.org/10.1007/978-3-030-28577-7_4
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-28576-0
Online ISBN: 978-3-030-28577-7
eBook Packages: Computer ScienceComputer Science (R0)