Feature ordering is a significant data preprocessing method in Incremental Attribute Learning (IAL), a novel machine learning approach which gradually trains features according to a given order. Previous research has shown that, similar to feature selection, feature ordering is also important based on each feature's discrimination ability, and should be sorted in a descending order of their discrimination ability. However, such an ordering is crucial for the performance of IAL. As the number of feature dimensions in IAL is increasing, feature discrimination ability also should be calculated in the corresponding incremental way. Based on Single Discriminability (SD), where only the feature discrimination ability is computed, a new filter statistical feature discrimination ability predictive metric, called the Accumulative Discriminability (AD), is designed for the dynamical feature discrimination ability estimation. Moreover, a criterion that summarizes all the produced values of AD is employed with a GA (Genetic Algorithm)-based approach to obtain the optimum feature ordering for classification problems based on neural networks by means of IAL. Compared with the feature ordering obtained by other approaches, the method proposed in this paper exhibits better performance in the final classification results. Such a phenomenon indicates that, (i) the feature discrimination ability should be incrementally estimated in IAL, and (ii) the feature ordering derived by AD and its corresponding approaches are applicable with IAL.
TopIntroduction
In pattern recognition, accurate classification and regression results usually depend on discriminative features which are also known as attributes of datasets. As an approach for pattern recognition, machine learning plays a significant role in the improvement of classification and regression performance. Different from conventional machine learning approaches which train all features in one batch by some predictive algorithms like neural networks (NN) and genetic algorithms (GA), Incremental Attribute Learning (IAL) is a novel supervised machine learning approach which gradually trains one or more input features step by step. Such a strategy makes the features which have greater discrimination abilities be trained in an earlier step than others, and get rid of interference between features during classification. Previous studies showed that IAL can be independently employed and successfully applied based on many machine learning approaches, such as NN (Ang, Guan, Tan, & Al Mamun, 2008; S.-U. Guan & Li, 2004; S.-U. Guan & Li, 2001; S.-U. Guan & Liu, 2004, 2005; S.-U. Guan, Liu, & Qi, 2004; S. U. Guan & Liu, 2002), GA (Chen & Guan, 2004; S. U. Guan & Zhu, 2005; F. Zhu & Guan, 2004; F. M. Zhu & Guan, 2008a, 2008b), Decision Trees (DT) (Chao & Wong, 2009) and Support Vector Machines (SVM) (X. Liu, Zhang, Zhan, & Zhu, 2008). Apart from these, the final results produced by IAL also exhibit better performance in pattern recognition compared with the conventional approaches which import all features for training in one batch. One of the most important reasons why IAL can succeed in pattern recognition is that it trains features which have greater discrimination ability at an earlier stage in the process which can successfully avoid interference from other features whose discrimination abilities are weaker (Ang et al., 2008). Therefore, feature ordering should be regarded as one of the most important preprocesses of sequential feature training in IAL.
The work presented in this paper has four motivations which are related to each other. Firstly, this paper aims to make feature ordering an independent and necessary preprocess in IAL-based pattern recognition, because in the last decade, feature ordering was seldom studied independently. Rather, it was often researched as a byproduct of IAL, which is often ignored. For example, in Guan’s experiments (S.-U. Guan & Li, 2001), all the features were imported in the original order. In the research on the incremental decision tree learning methodology and the Incremental Feature Learning of SVM, researchers have not arranged new feature orderings for the training. The original feature ordering was directly employed in their experiments (Chao & Wong, 2009; X. Liu et al., 2008).