Big Data Overview

Big Data Definition

Big data is defined in Wikipedia that as the “data sets that are too large for storage, management, processing and analysis, it present challenges beyond traditional IT techniques.” (Wikipedia, 2012) BIG is a term that is relative to the size of the data, and the scope of the IT infrastructure that is in place. Transforming big data could benefit scientific discovery, environmental and biomedical research, and national security. In order to do that, big data requires the use of new technical architectures and analytics tools, to generate business value from the huge volume of data, in order to create insights.

Big Data comes in all kinds of forms: from highly structured ERP (Enterprise Resource Planning) data, or CRM (Customer Relation Management) data, to multi-million rows of text file, to video files and machine generated sensor data. The common feature is the high data volume and data complexity. Most of big data is unstructured or semi-structured, and require new techniques and tools to analyze.

Big Data examples are everywhere in our lives. With the popularity of mobile computing and the self-expression tolls, everyone has the ability to share their thoughts and ideas worldwide. Smart phones carry sensors like GPS, accelerometer, microphone, camera and Bluetooth which can collect huge amounts of data, and allow research on behavioral and social science, with the large scale mobile data to characterize and understand real-life phenomena. In 2011, there have been 6 billion mobile phone subscribers, growing 45 percent annually for the past four years. (ITU, 2011) A quarter of them use smartphones. By 2014, mobile internet use should overtake desktop internet use.

There are more than 845 million active Facebook users by the end of 2011, 50 percent log onto Facebook every day; 30 billion pieces of content are shared every month. Every 60 seconds, there are 510,000 posted comments, 293,000 status updates and 135,000 uploaded photos. 20 million Facebook applications are installed per day. In just 20 minutes, over 1 million links are shared. (Protalinski, 2012)

Twitter has 100 million active users around the world; more than half of them log in to twitter each day to follow their interest. The average user has 115 followers. An average of 190,000,000 tweets are sent per day. Tweeter handles 1.6 billion queries per day. 34% of marketers have generated leads using Twitter and 20% have closed deals. (Twitter, 2011)