Object Architected Design and Efficient Dynamic Adjustment Mechanism of Distributed Web Crawlers

Object Architected Design and Efficient Dynamic Adjustment Mechanism of Distributed Web Crawlers

Cheng-Hung Tsai (Institute for Information Industry, Innovative DigiTech-Enabled Applications & Service Institute, Taipei, Taiwan), Tsun Ku (Institute for Information Industry, Innovative DigiTech-Enabled Applications & Service Institute, Taipei, Taiwan) and Wu-Fan Chien (Institute for Information Industry, Innovative DigiTech-Enabled Applications & Service Institute, Taipei, Taiwan)
DOI: 10.4018/ijitn.2015010105
OnDemand PDF Download:
$30.00
List Price: $37.50

Abstract

As the global socialnomics rise, big data makes enterprises face the tremendous tide of data at any time. How to efficiently process, analyze these unstructured data and dig useful information from them has been an issue for every level of enterprises to face and settle. Gartner conducted a survey (Gartner CIO Agenda) over 2300 CIOs worldwide and found out that the business intelligence based on big data has been the primary issue (IBM, 2013). Hence, by understanding the above developing trend of social media, this research is mainly based on the authors' previously proposed paper in ises face the tremendous ti:Design and Implementation of a Web Crawlers Based in Social Networkst any time. How to efficiently process, analyze these unstructured data and dig useful infed architecture of web crawler. This new architecture is added with the concept of object structure for the design and implementation of the whole system. The authors will also investigate the improved object structure that brings the convenience of system maintenance.
Article Preview

According to the survey of eMarketer at 2013, the number of global social media users has reached 1.73 billion, which covers the 67.7% of the population of internet users. Among them the monthly active users of Facebook are 1.2 billion, the ones of Twitter are over 0.55 billion, the ones of Weibo and Tencent originated in China are over 0.4 billion (IDEAS, III, 2013). Currently the social media users are mostly young people. There are over 90% of internet users aging from 18 to 29 use social networking sites, but the penetration rate of those over 65 years old is only 40%. However, the penetration rate of senior citizens on social networking sites is increasing rapidly.

According to the statistics of III (2013), domestic enterprises have only 15% structured data, yet the other 85% exists as an unstructured data, such as email, web links, logs/journals of information systems. On the other hand, according to the research of IBM (2013), after the internet becomes universal, the data produced on the internet every two days at present is equal to the sum of data from the year of invention of computer to 2003. Under the rapid and massive increase of digital data, formal information systems/platforms are unable to stand. This causes the big data to become the widely concerned issue.

Complete Article List

Search this Journal:
Reset
Open Access Articles: Forthcoming
Volume 9: 4 Issues (2017)
Volume 8: 4 Issues (2016)
Volume 7: 4 Issues (2015)
Volume 6: 4 Issues (2014)
Volume 5: 4 Issues (2013)
Volume 4: 4 Issues (2012)
Volume 3: 4 Issues (2011)
Volume 2: 4 Issues (2010)
Volume 1: 4 Issues (2009)
View Complete Journal Contents Listing