Object-of-Interest Retrieval in Social Media Image Databases for e-Crime Forum Detection

Object-of-Interest Retrieval in Social Media Image Databases for e-Crime Forum Detection

Xinpeng L. Liao (University of Alabama at Birmingham, Birmingham, AL, USA), Pradip Chitrakar (University of Alabama at Birmingham, Birmingham, AL, USA), Chengcui Zhang (Department of Computer and Information Sciences, University of Alabama at Birmingham, Birmingham, AL, USA) and Gary Warner (University of Alabama at Birmingham, Birmingham, AL, USA)
DOI: 10.4018/ijmdem.2015070103


Using object-of-interest matching to detect presence of e-Crime activities in low-duplicate social media images is an interesting yet challenging problem that involves many complications due to the dataset's inherent diversity. SURF-based (Speeded Up Robust Features) object matching, though claimed to be scale and rotation invariant, is not effective as expected in this domain. This paper approaches this problem by an extended paradigm of Generalized Hough Transform using shape matching applied to two types of object-of-interest, Guy Fawkes Mask and Credit Card. We propose an extended GHT that updates the best matching score and the sum up score simultaneously, combined with a face detector and circular magnitude ranker, for detecting Guy Fawkes; also proposed is an extended GHT capable of mining the directional property in Hough space, combined with optical character recognition and an edge density filter, for detecting credit cards. Experiments on two real world datasets indicate that our approach outperforms the baseline GHT and the SURF.
Article Preview

1. Introduction

In light of the information sharing power of social media, there is a trend developing among criminal groups to use social media as their tools to conduct criminal activities and share experience. Signs of such criminal activities could include things such as the Guy Fawkes masks (Figure. 1(a)) and Visa logos (Figure. 1(c)) in profile pictures indicating the involvement of criminal hacker group, the plain text posting of credit card numbers, clear evidence of selling stolen merchandise, evidence of stolen login credentials being shared, or screen shots displaying evidence of controlling a botnet. At the computer forensics center in our university, we are monitoring several criminal groups that are using Facebook to promote e-Crime Forums. Also, such kind of images are rarely seen in non-criminal groups. The question now arises – “How can we find such images from the vast pool of social media images?” Logo matching (Bagdanov, Ballan, Bertini & Bimbo, 2007) (Romberg, August, Ries & Lienhart, 2012) (Romberg & Lienhart, 2013) (Romberg, Pueyo, Lienhart & Zwol, 2011) may be one of the promising directions to mine social media images for detecting such criminal forums. However, there are many objects-of-interests in this context that are not commercial logos, such as Guy Fawkes masks and guns that do not always appear with a consistent look and thus exhibit a greater variety in their visual appearance. Credit cards could be detected using logo matching, but it is not practical to detect each different type of objects with a totally different method in this context. Our goal in this paper, however, is motivated by the fact that most objects of interest have a relatively rigid and defining shape (but not necessarily defining color or texture features), therefore we set out to develop a constant shape-based object detection framework, which uses shape matching to automatically label the social media image with objects-of-interests in order to detect e-Crime forums.

Figure 1.

Motivation for using shape matching: (a) and (c) SURF; (b) and (d) shape matching

Although identifying an identical, near-duplicate (Chum, Philbin & Zisserman, 2008) (Lee, Ke & Isard, 2010) (Wu, Ke, Isard & Sun, 2009) object in a database of images has been studied extensively, it is still a challenging problem because the visual appearances of a target object may be very different due to different viewpoint, lighting, and partial occlusion (Baumberg, 2000) (Sivic & Zisserman, 2003). Typically, a representation of the visual appearance is captured by a feature descriptor, e.g., SIFT-like features (SIFT: Scale-Invariant Feature Transform (Lowe, 2004)). The feature descriptor of a target object (e.g., Guy Fawkes mask) will be compared with that of every other image in the database. Recognition of an object is achieved by nearest neighbor ratio test (Muja & Lowe, 2009) and a geometrical verification process. The similarity between a reference/target image and a test image can be formulated as simple as the number of matched points after geometrical verification (Zhou, Li, Lu & Tian, 2013).

Complete Article List

Search this Journal:
Open Access Articles: Forthcoming
Volume 10: 4 Issues (2019): Forthcoming, Available for Pre-Order
Volume 9: 4 Issues (2018): 3 Released, 1 Forthcoming
Volume 8: 4 Issues (2017)
Volume 7: 4 Issues (2016)
Volume 6: 4 Issues (2015)
Volume 5: 4 Issues (2014)
Volume 4: 4 Issues (2013)
Volume 3: 4 Issues (2012)
Volume 2: 4 Issues (2011)
Volume 1: 4 Issues (2010)
View Complete Journal Contents Listing