Analyzing Twitter User-Generated Content Changes

Introduction

In recent years, social networks and online communities have become a powerful source of knowledge. Social network users are used to publish and continuously update multimedia resources, posts, blogs, etc. Actions undertaken by Web users reflect their habits, personal interests, and professional skills. Hence, the analysis of the user-generated content coming from social networks has received an increasingly high attention in several application contexts. For instance, data mining techniques have already been applied to recommend personalized services and products based on social annotations (Wang et al., 2010; Shepitsen et al., 2008; Xue et al., 2009), organize and make social knowledge accessible (Kasneci et al., 2009), and perform email spamming based on social networks (Lam et al., 2007). In particular, data mining from UGC published on the popular Twitter microblogging Website has achieved promising results in the analysis of most notable user behaviors (Li, Guo, & Zhao, 2008; Mathioudakis & Koudas, 2010) and topic trends (Cheong & Lee, 2009).

Twitter user-generated content consists of a large collection of short textual messages (i.e., the tweets) posted by Web users and their contextual information (e.g., publication time and date). Since the Twitter user-generated content and contextual data continuously evolve over time, a relevant research issue is the application of data mining techniques to discover most significant pattern changes. Dynamic itemset mining (Agrawal & Psaila, 1995) entails discovering itemsets that (i) frequently occur in the analyzed data, and (ii) may change from one time period to another. The history of the main itemset quality indexes reflects the most relevant temporal data correlation changes. However, the sparseness of the analyzed data makes dynamic itemset mining from UGC a challenging task. In fact, potentially relevant itemsets discovered at a certain time period are likely to become infrequent (i.e., their support value becomes lower than a given threshold) in at least another one. Hence, the information associated with the discovered itemsets may be lost, unless lowering the support threshold and mining a huge amount of other (potentially redundant) itemsets.

This chapter presents the TwiChI (Twitter Change mIner) system that aims at supporting experts in the analysis of Twitter UGC changes targeted to user behavior and topic trend analysis. TwiChI exploits the Twitter Application Programming Interfaces (APIs) to retrieve both tweet textual contents and their contextual features (i.e., publication date, time, place). Data crawling is continuously executed using the Twitter Public stream endpoint to track the temporal evolution of the frequent itemsets occurring in the analyzed data. The retrieved data is analyzed by the proposed HiGen Miner algorithm (Cagliero, 2011), which discovers compact patterns, named the History Generalized Patterns (HiGens). HiGens represent the evolution of frequent itemsets across consecutive time periods. To avoid the discarding of rare but potentially relevant knowledge, itemsets that become infrequent in a certain time period with respect to the minimum support threshold are generalized at a higher level of abstraction by exploiting a taxonomy (i.e., a set of is-a hierarchies built on data items). A generalized version of a traditional itemset is an itemset that represents the same knowledge at a higher level of aggregation according to a given taxonomy (Agrawal & Srikant, 1995). Hence, the knowledge associated with itemsets that rarely occur at certain time periods is still maintained by replacing the low level itemset versions with their frequent generalizations with least abstraction level.