Article Preview
TopIntroduction
In pattern recognition, accurate classification and regression results usually depend on discriminative features which are also known as attributes of datasets. As an approach for pattern recognition, machine learning plays a significant role in the improvement of classification and regression performance. Different from conventional machine learning approaches which train all features in one batch by some predictive algorithms like neural networks (NN) and genetic algorithms (GA), Incremental Attribute Learning (IAL) is a novel supervised machine learning approach which gradually trains one or more input features step by step. Such a strategy makes the features which have greater discrimination abilities be trained in an earlier step than others, and get rid of interference between features during classification. Previous studies showed that IAL can be independently employed and successfully applied based on many machine learning approaches, such as NN (Ang, Guan, Tan, & Al Mamun, 2008; S.-U. Guan & Li, 2004; S.-U. Guan & Li, 2001; S.-U. Guan & Liu, 2004, 2005; S.-U. Guan, Liu, & Qi, 2004; S. U. Guan & Liu, 2002), GA (Chen & Guan, 2004; S. U. Guan & Zhu, 2005; F. Zhu & Guan, 2004; F. M. Zhu & Guan, 2008a, 2008b), Decision Trees (DT) (Chao & Wong, 2009) and Support Vector Machines (SVM) (X. Liu, Zhang, Zhan, & Zhu, 2008). Apart from these, the final results produced by IAL also exhibit better performance in pattern recognition compared with the conventional approaches which import all features for training in one batch. One of the most important reasons why IAL can succeed in pattern recognition is that it trains features which have greater discrimination ability at an earlier stage in the process which can successfully avoid interference from other features whose discrimination abilities are weaker (Ang et al., 2008). Therefore, feature ordering should be regarded as one of the most important preprocesses of sequential feature training in IAL.
The work presented in this paper has four motivations which are related to each other. Firstly, this paper aims to make feature ordering an independent and necessary preprocess in IAL-based pattern recognition, because in the last decade, feature ordering was seldom studied independently. Rather, it was often researched as a byproduct of IAL, which is often ignored. For example, in Guan’s experiments (S.-U. Guan & Li, 2001), all the features were imported in the original order. In the research on the incremental decision tree learning methodology and the Incremental Feature Learning of SVM, researchers have not arranged new feature orderings for the training. The original feature ordering was directly employed in their experiments (Chao & Wong, 2009; X. Liu et al., 2008).