Framework for the Discovery of Newsworthy Events in Social Media

Framework for the Discovery of Newsworthy Events in Social Media

Fernando José Fradique Duarte (University of Aveiro, Aveiro, Portugal), Óscar Mortágua Pereira (Instituto de Telecomunicações, DETI – University of Aveiro, Aveiro, Portugal) and Rui L. Aguiar (Instituto de Telecomunicações, DETI – University of Aveiro, Aveiro, Portugal)
DOI: 10.4018/IJOCI.2019070103
OnDemand PDF Download:
No Current Special Offers


The new communication paradigm established by social media along with its growing popularity in recent years contributed to attract an increasing interest of several research fields. One such research field is the field of event detection in social media. The contribution of this article is to implement a system to detect newsworthy events in Twitter. The proposed pipeline first splits the tweets into segments. These segments are then ranked. The top k segments in this ranking are then grouped together. Finally, the resulting candidate events are filtered in order to retain only those related to real-world newsworthy events. The implemented system was tested with three months of data, representing a total of 4,770,636 tweets written in Portuguese. In terms of performance, the proposed approach achieved an overall precision of 88% and a recall of 38%.
Article Preview


Social Media services have become a very popular medium of communication and users use these services for various different reasons. In the case of Twitter, a microblogging service, the main reasons found are (Java, Song, Finin, & Tseng, 2007): daily chatter, conversations, sharing information and reporting news. Microblogging services in particular have become very popular due to their portability, immediacy and ease of use, allowing users to respond and spread information more rapidly (Atefeh & Khreich, 2015). The popularity and real time nature of these services and the fact that the data generated reflect aspects of real-world societies and is publicly available have attracted the attention of researchers in several fields (Madani, Boussaid, & Zegour, 2014; Nicolaos, Ioannis, & Dimitrios, 2016). One such field is the field of event detection in Social Media.

Event detection in Social Media has many potential applications, some of which with significant social impact such as in the detection of natural disasters and to identify and track diseases and epidemics (Madani et al., 2014). Another relevant application can be found in the detection of news topics and events of interest or newsworthy, as real-world events are often discussed by users in these services before they are even reported in traditional Media (Papadopoulos, Corney, & Aiello, 2014; Sakaki, Okazaki, & Matsuo, 2010; Van Canneyt et al., 2014). These services however present some challenges, some of which are inherit to their design and usage (Atefeh & Khreich, 2015). In the case of Twitter, the use of informal and abbreviated words, the frequent occurrence of spelling and grammatical errors, data sparseness and lack of context due to the short length of the messages, are just a few examples of these challenges. The diversity and nature of the topics discussed may also pose additional challenges, more specifically in the case of event detection, as most of these topics are of little interest (e.g. daily chatter). The event detection process must therefore be able to filter out these topics in order to retain only those potentially related to events of interest.

The goal of this work and also its main contribution is the implementation of a fully functional system in order to detect newsworthy events using tweets, that is, any real-world event of sufficient interest to the general public in order to be reported by the Media. To achieve this goal a similar methodology already proposed in the literature, namely Twevent (C. Li, Sun, & Datta, 2012) is used as the base of the implementation. The event detection pipeline proposed consists of the following steps: first the tweets are segmented into a set of non-overlapping segments (i.e. n-grams). These segments are then ranked according to a weighting scheme. Only the top K segments are retained for further processing, thus obtaining the event segments. A variant of the Jarvis-Patrick clustering algorithm is then used in order to cluster these event segments into candidate event clusters. Finally, these candidate event clusters are filtered in order to retain only those related to real world events of interest. This filtering step is performed by a Random Forest model.

Complete Article List

Search this Journal:
Open Access Articles
Volume 12: 4 Issues (2022): 2 Released, 2 Forthcoming
Volume 11: 4 Issues (2021)
Volume 10: 4 Issues (2020)
Volume 9: 4 Issues (2019)
Volume 8: 4 Issues (2018)
Volume 7: 4 Issues (2017)
Volume 6: 4 Issues (2016)
Volume 5: 4 Issues (2015)
Volume 4: 4 Issues (2014)
Volume 3: 4 Issues (2012)
Volume 2: 4 Issues (2011)
Volume 1: 4 Issues (2010)
View Complete Journal Contents Listing