Data generation speed and amount of data has increased over the past 20 years in different fields. A report published in 2011 from International Data Corporation (IDC) states that, the overall generated and stored data size in the globe was 1.8ZB (≈ 1021B), which enlarged by almost 9 times within 5 years (Lopez et al., 2014). Due to the enormous growth of world data, the name of big data is essentially used to express massive datasets. In general, big data analytics is requires advance tools and techniques to store, process and analyze the huge volume of data. Big data consists of huge unstructured data that require advance real-time analysis (Lopez and Gunasekaran, 2015). Thus, nowadays many of the researchers are interested in developing advance technologies and algorithms to solve the issues when dealing with big data. In order to discover new opportunities and hidden values from big data, Yahoo developed the Hadoop based tools and technologies to store and process the big data. Nowadays, private organizations are also interested in the high prospective of big data, and numerous government agencies declared vital ideas to speed up the big data research and applications. Two leading scientific journals such as nature and science are also opened special issues to solve and discuss the challenges and impacts of big data. In recent years, big data plays a vital role in Internet companies such as Google, Facebook and Twitter. For example, Google handles nearly 100 Petabyte (PB) and Facebook produces log data of over 10 Petabyte per month. A popular Chinese company, Baidu, analyzes data of 10 Petabyte (PB), and Taobao, a subsidiary of Alibaba, produces data of 10 Terabyte (TB) for online trading per day. Sources of big data and the corresponding mining techniques are depicted in Table 1. State of-the-art tools and technologies to handle big data are depicted in Table 2.
Nowadays, Big Data has been playing a vital role in almost all environments such as healthcare, education, business organizations and scientific research. There is a strong relationship in Big Data and IoE (Internet of Everything). In general, IoE applications are used to capture or observe some specific values to find the hidden values and take better decisions. When the device is connected to the Internet, it always senses the specific metric and stores those metrics into a connected data stores. This would increase the size of the data stored in a data store. Hence, high end devices and scalable storage systems are needed to store such huge size of data. The amount of data to be stored and processed becomes an important problem in real life. Relational data base management system (RDBMS) is generally used to store the traditional data but day by day the volume, velocity and variety of sensor data is growing towards the Exabyte. This requires advanced tools and techniques to store, process and display such large amount of sensor data to the end users. Hence, Big Data tools are often used to process such huge amounts of data. This would increase the economy and market of the Big Data analytics. The report “Big Data Market by Component (Software and Services) states that “The Big Data market is expected to grow from USD 28.65 Billion in 2016 to USD 66.79 Billion by 2021 at a high Compound Annual Growth Rate (CAGR) of 18.45%”. For the purpose of the report, 2015 has been considered as the base year and 2016 as the estimated year for performing market estimation and forecasting. 10V’s of big data is shown in Figure 1. Various big data analytical algorithms are shown in Figure 4.
10V’s of big data