Article Preview
TopIntroduction
Online social media networks have become a rich source of news distribution about real-world events of all kinds. Twitter as a social networking site has emerged to be an endless dynamic real-time global stream of news. Many people use Twitter as a source of news content instead of sharing thoughts and emotions. Journalists are also increasingly adopting twitter as a professional tool for the process of news selection and presentation by continuous monitoring of emerging user-generated newsworthy stories from Twitter stream overloaded with the high amount of noise and redundant information. The typical monitoring method is to search the stream with event relevant keywords. However, the search results after satisfying such a Boolean query is formidable. For example, victory of Narendra Modi as a Prime Minister of India in general election 2019 induced millions of tweets on result declaration night. In such sea of tweets on a topic or related to any event, ranking has become an important issue in Twitter not just in Web search. While there exist an extensive research on ranking for web search (Brin and Page (1998), Agichtein et al. (2006), Xiang et al (2010), Aggarwal (2010)), there is little work done for ranking of tweets that generate the need to develop an efficient Tweets ranking model with the following goals:
Relevance: The top-ranked tweets constituting the summary of the event must be relevant to the specific event and contain some important information that can be used in event analysis.
Diversity: The resultant tweets presenting the summary should capture the diverse information and must not be similar in nature-wise.
There are various applications like Twitter Sentiment Analysis (Thelwall et al. (2011), Zhang et al. (2018), Tripathi et al. (2019)) and News Recommender System (Abel et al. (2011), Phelan et al. (2011), Kywe et al. (2012)) also required relevant tweets to process. Some popular text ranking methods like Lexrank (Erkan and Radev (2004)) and Textrank (Mihalcea and Tarau (2004)) are based on Google’s Pagerank algorithm (Brin and Page (1998)) to rank sentences which are suitable for traditional news data but fails in case of Twitter. This is primarily because, in contrast to traditional news documents, tweets are inherently noisy with high variance in word frequencies and low word count. Further, tweets have some other challenging features like high redundancy and large diversity in the vocabulary to convey the same information. Hence, extracting important tweets based on lexical similarity is not effective to handle vocabulary variation problem. For example:
These two tweets convey the same information means that there exists some relationship with no common vocabulary. In this case, similarity measures like the Cosine or Jaccard metric fails because of using the Vector Space Model based on the common occurrence of textual units between two documents. To resolve this problem, word embedding language models (Mikolov et al. (2013), Pennington et a. (2014), Song et al. (2019) ) come into picture that capture semantics or meaningful relationships. So that, we calculate the semantic similarity between tweets using Word Mover’s Distance (WMD) (Kusner et al. (2015)) based on Word2Vec model (Mikolov et al. (2013)). Main contributions of the proposed approach are as follows:
(1) We utilize several social influence features like re-tweet count, follower count and presence of URL to remove the noisy tweets like personal, fake and so on.
(2) We also utilize tweet content features to rank the tweets by using a semantic graph model where vertices represent tweets and edges represent the semantic similarity between tweets. Top-ranked vertices (tweets) are extracted to represent the event summary by utilizing the Pagerank algorithm.
(3) We compared the results of the proposed approach with four baseline approaches (Lexrank, Textrank, Re-tweet voting, and Follower voting). Our system outperforms the baseline approaches in accuracy measured by ROUGE metric and Human evaluation score.