Article Preview
Top1. Introduction
Many areas such as computer vision, signal processing and medical image analysis require the managing of data sets with a large number of features or dimensions. Therefore, dimensionality reduction may be necessary in order to discard redundancy and reduce the computational cost of further operations, Lee & Verleysen (2007).
We may distinguish two major classes of dimensionality reduction methods: linear and nonlinear. The former includes the classical principal component analysis (PCA), linear discriminant analysis (LDA) and multidimensional scaling (MDS), Engel et al. (2012), Hastie et al. (2001), Cox & Cox (2001). Linear techniques seek for new variables that obey some optimization criterium and can be expressed as linear combination of the original ones. That is why they fail if the input data has curved or nonlinear structures. These methods can be also classified as subspace learning methods in the sense that the output linear space has an optimum subspace for compact data representation.
In this paper we focus on nonlinear dimensionality reduction methods that can be classified into global and local categories. The kernel PCA (KPCA), kernel LDA (KLDA) and kernel Fisher discriminant analysis (KFD) are known global nonlinear dimensionality reduction methods that map the original input data into a feature space by a (global) non-linear mapping, where inner products in the feature space can be computed by a kernel function in the input space without explicitly knowing the non-linear mapping, Baudat & Anouar (2000), Park & Park (2005), Scholkopf et al. (1998). Laplacian Eigenmap and Isomap can be also considered as global techniques because they work on global structures computed through a graph associated with the whole database samples, Lee and Verleysen (2007), Belkin & Niyogi (2003), Tenenbaum et al. (2000). On the other hand, local methods attempt to preserve the structure of the data by seeking to map nearby data points into nearby points in the low-dimensional representation. Then, the global manifold information is recovered by minimizing the overall reconstruction error. Traditional manifold learning techniques like Locally Linear Embedding (LLE) and Local Tangent Space Alignment (LTSA) and Hessian Eigenmaps, as well as the more recent Local Riemannian Manifold Learning (LRML), belong to this category of nonlinear dimensionality reduction methods, Goldberg et al. (2008), Roweis & Saul (2000), Junior et al. (2013).
The main point behind manifold learning techniques is the assumption that the input data lies on a low-dimensional manifold embedded in a high dimensional space. Therefore, we need to learn the underlying intrinsic manifold geometry in order to address the problem of dimensionality reduction. Thus, instead of seeking for an optimum linear subspace the manifold learning methods try to discover an embedding procedure that describes the intrinsic similarities of the data. Manifold-based high dimensional data analysis has been applied in several problems related, for instance, to face analysis, pattern recognition, age estimation, character recognition, computer vision and hyperspectral data, Lunga et al. (2014), Lee & Verleysen (2007), Zhang et al. (2004), Lin & Zha (2008a).