Efficient High Dimensional Data Classification

Background

Big data refers to the data that is large in size or large in dimensionality or both. It becomes difficult to store, analyze, search, share and visualize such a large data. Big data is being generated due to the advancements in sensor technology, wireless sensor networks, information gathering mobile devices, remote sensing devices, various data capturing tools and sophisticated cameras. Big data are of high volume (i.e. both size and dimensionality may be large), of high velocity (i.e. making time sensitive decisions by processing big data as it streams into the enterprise), and/or high variety information assets (i.e. structured and unstructured data). The traditional data processing applications and conventional decision making algorithms become impractical in handling big data of this size and volume.

A wide variety of sensors enable collecting a large number of observations (patterns). They also enable collecting larger number of features that describe each pattern leading to high dimensional data. The classification of the high dimensional data has become increasingly important in the fields of engineering, computational biology, genomics and pattern recognition. In high dimensional data, generally the number of features is greater than the number of patterns. One of the major challenges in classifying high dimensional data is high variance and overfitting (due to the noise). The other concern is the curse of dimensionality that makes the conventional classification impractical. The curse of dimensionality refers to the demand for more number of observations with an increase in the dimensionality in order to get good estimates. The increase in dimensionality increases the noise, over fitting and the complexity of any learning algorithm. These problems can be solved and the generalization performance of the classifier can be improved either by 1) reducing the dimensionality of the data 2) minimizing the VC dimension of the classifier or 3) by providing a large number of training samples.

Dimensionality reduction involves feature extraction or feature selection. Feature extraction techniques such as Principal Component Analysis, Linear Discriminant Analysis, Random Projection, Independent Component Analysis etc., have been extensively explored (Van der Maaten Postma & Van den Herik, 2009; Subasi & Ismail Gursoy, 2010; Seetha, Murty & Saravanan, 2011a; 2011b; Deng et al., 2012). Feature selection techniques using wrappers and filters have also been well studied (Kohavi & John, 1997; Liu & Yu, 2005; Gao et al., 2011; Bermejo et al., 2012). The recent trend is towards pattern synthesis to combat the curse of dimensionality (Agrawal et al., 2005; Viswanath, Murty & Bhatnagar, 2006; Chen, Zhu & Nakagawa, 2011; Seetha, Saravanan & Murty, 2012).