Big Data: An Emerging Field of Data Engineering

Big Data: An Emerging Field of Data Engineering

Piyush Kumar Shukla (University Institute of Technology RGPV, India) and Madhuvan Dixit (Millennium Institute of Technology, India)
Copyright: © 2015 |Pages: 19
DOI: 10.4018/978-1-4666-8387-7.ch016
OnDemand PDF Download:
No Current Special Offers


In this chapter, Big Data provide large-volume, complex structure, heterogeneous and irregular growing data sets include multiple and autonomous different resources. In this chapter, With the growing improvement of networking sites, image information storing capacity become big issue too, Big Data concept are most growing expanding in all technical area and knowledge engineering domains, including physical, medical and paramedical sciences. Here a data-driven method consist demand-driven aggregation of information and knowledge mining and analysis, user interest prototyping, security and privacy aspects has been presented.
Chapter Preview


Big data is concept more uses from various companies. Some examples are related with oil and gas refineries and mining industries, online social networks, multimedia data and business related transactions. More amount of data collected from different increasingly efficient data storing various devices as well as stored on fast-growing mass storage, people are keep to search to find solutions to collect and process the information more efficiently, and to find various values from the mass at the same time. When referring to big data research strategy problems, people often support the 4 v's -- volume, velocity, variety, and value. These pose support more brand-new challenges to computer scientists nowadays (Ahmed & Karypis, 2012).

In 2004, Wal-Mart claimed to have the very large data warehouse with approx 500 terabytes storage. In 2009, eBay announce storage amounted to eight PB. Two years later, the Yahoo warehouse totaled 170 PB. Since the rise of digitization, various technical enterprises have amassed burgeoning amounts of digital information, including trillions of bytes of data about their customers, suppliers and operations (Aral & Walker, 2012). Information volume is also growing exponentially due to the spread of machine-oriented data (data records, web-log files, sensors information) and from growing human involvement within the social networks.

Basically the growth of information will never stop. As per the 2011 IDC Digital Universe Study, 130 EB of data were created and stored in 2006. The amount grew to 1,327 EB in 2011 and is projected to grow at 43.6% to 7,913 EB in 2016. The growth of information constitutes the “Big Data” concept – a technological concept bring about by the rapid rate of data expands and parallel generalization in technology that have given rise to an contribution of software and hardware products that are enabling users to analyze this information to produce new and more granular desire levels of information (Machanavajjhala & Reiter, 2012). A decade of Digital Universe Growth: Storage in Exabytes has been illustrated in figure 1.

Figure 1.

A decade of Digital Universe Growth: Storage in Exabytes (Machanavajjhala & Reiter, 2012)


According to McKinsey, Big Data, perform to datasets whose sizes are beyond the ability of database software product to acquire, store, manage and study. There is no explicit definition of how big an image dataset should be in order to be considered large Big Data. New technology has to be manages this Big Data concept. Basically, IDC defines Big Data technologies considered as a new generation of database technologies and architectures designed to extract data value from large volumes of a wide variety of data. According to Reilly, “Big data is information which is exceeds the processing capacity of traditional database products. Basically, information is too big, moves too fast, or does not assemble the structures of desired database architectures. To gain value from these databases resource, there must be different way to process it” (Machanavajjhala & Reiter, 2012).


Characteristics Of Big Data

Big Data is not just about the volume of information but also includes information variety and velocity. Together, these three parameters form the three Vs of Big Data theory. Volume is synonymous form of the “big” in the term, “Big Data”. Volume is a relative term – some smaller-sized industries are likely to have mere gigabytes or terabytes of information storage as opposed to the PB or EB of data that big data global enterprises have. Data volume will continue to expand, regardless of the organization’s capacity. There is a natural definition for companies to store data of all categories: account data, biomedical data, research data and so on. Many of these organization datasets are within the terabytes range today but in early days they could be reach PB or even EB (Aral & Walker, 2012). Data can come from a variety of resources and in a variety of category. With the expansion of sensors, various smart devices such smart phone as well as social networking sites, information in an organization has become more complex because it includes not only structured and complex relational information, but also semi-structured, heterogeneous and unstructured information. The 3 Vs of Big Data has been illustrated in figure 2.

Key Terms in this Chapter

Data Mining: Data mining is the analysis of data for relationships that have not previously been discovered.

Autonomous Sources: Autonomous describes things that function separately or independently. Once you move out of your parents' house, and get your own job, you will be an autonomous member of the family.

Big Data Mining: Big data analytics enables organizations to analyze a mix of structured, semi-structured and unstructured data in search of valuable business information and insights.

Heterogeneity: Heterogeneity is a word that signifies diversity. A classroom consisting of people from lots of different backgrounds would be considered having the quality of heterogeneity.

Big Data: Big data is a broad term for data sets so large or complex that traditional data processing applications are inadequate.

Data Analytics: Data analytics refers to qualitative and quantitative techniques and processes used to enhance productivity and business gain.

Big Data Model: Big data is a collection of data from traditional and digital sources inside and outside your company that represents a source for ongoing discovery and analysis.

Complete Chapter List

Search this Book: