Assessing Sentiment of the Expressed Stance on Social Media

Abstract

Stance detection is the task of inferring viewpoint towards a given topic or entity either being supportive or opposing. One may express a viewpoint towards a topic by using positive or negative language. This paper examines how the stance is being expressed in social media according to the sentiment polarity. There has been a noticeable misconception of the similarity between the stance and sentiment when it comes to viewpoint discovery, where negative sentiment is assumed to mean against stance, and positive sentiment means in-favour stance. To analyze the relation between stance and sentiment, we construct a new dataset with four topics and examine how people express their viewpoint with regards these topics. We validate our results by carrying a further analysis of the popular stance benchmark SemEval stance dataset. Our analyses reveal that sentiment and stance are not highly aligned, and hence the simple sentiment polarity cannot be used solely to denote a stance toward a given topic.

A Analysis of the Textual Patterns

To gauge the similarity between the vocabulary choice that has been used to express the sentiment and stance we analyzed the tweets in the two datasets using Jaccard similarity. We used Jaccard coefficient the widely adopted measure to capture the overlap between two sets [1, 3, 8]. In this analysis, for each sentiment and stance gold labels we combine all tweets and use Term Frequency-Inverse Document (TF-IDF), to find important words in each type of sentiment and stance. In order to compute the TF-IDF on tweet level we consider each tweet as document. Using TF-IDF helps in filtering out less significant words. The Jaccard similarity between the set of sentiment and stance words defined as following:

$$\begin{aligned} Jaccard(W_{sentiment},W_{stance}) = \frac{W_{sentiment} \cap W_{stance}}{W_{sentiment} \cup W_{stance}} \end{aligned}$$

(1)

where W$_{sentiment}$ and W$_{stance}$ denote the list of top N words by TF-IDF value for the tweets with specific sentiment and stance type.

Fig. 3.

Jaccard similarity of the top N-most frequent words between sentiment and stance.

Figure 3 shows that the similarity between the words that have been used to express favor stance has less than 20% of similarity with tweets that has a positive sentiment. That means users tend to express their Favor stance without using positive sentiment words. In contrast, the common words for against stance have the most significant similarity with against sentiment words. The Jacquard similarity become stable with growing N. As Fig. 4 shows that the overall agreement between the sentiment and the stance is minuscule in general. The tweets that have against-negative labels constitutes less than 33%. Similarly less than 8% of the data has positive sentiment and favor stance. This shows that in general negative words tend to be similar to the against words while the matching cases are minuscule. On the other-hand, the matching cases where the tweet express favor and positive sentiment constitute about 8.9% and 4% of the overall data of SemEval stance and CD stance dataset.

Fig. 4.

Tweets with matching and mixed stance and sentiment.