Iterative MapReduce: i-MapReduce on Medical Dataset Using Hadoop

Background

Big Data is also like normal data but with an enormous size. This is a term is generally used to describe a collection of data that is very huge in size and still growing exponentially with time. In short, such a large collection of data which is difficult to handle via traditional databases and other management tools is called ‘BigData’. Generic features of BigData are:

• •

  Volume

• •

  Velocity

• •

  Variability

• •

  Veracity

The main objective of this analysis is to find interesting patterns on the basis of conditional dependence on given attributes of a dataset. With such an increased rate of data generation it becomes very difficult to analyze the patterns in the dataset. Also the situation becomes more critical when we have sequential patterns in the dataset i.e. the order of dependency matters. Such behavior of data is very usual in day to day actions such as customer shopping behavior, medical symptoms leading to a future patient disease, financial stock market data predictions etc. Pattern mining of BigData using Hadoop faces a lot of issues in terms of data storage, data shuffling, data scanning, data processing units etc.

Figure 1.

Implications of BigData