The choice of kernel function and its parameter is very important for better performance of support vector machine. In this chapter, the authors proposed few new kernel functions which satisfy the Mercer’s conditions and a robust algorithm to automatically determine the suitable kernel function and its parameters based on AdaBoost to improve the performance of support vector machine. The performance of proposed algorithm is evaluated on several benchmark datasets from UCI repository. The experimental results for different datasets show that the Gaussian kernel is not always the best choice to achieve high generalization of support vector machine classifier. However, with the proper choice of kernel function and its parameters using proposed algorithm, it is possible to achieve maximum classification accuracy for all datasets.
TopIntroduction
In the past two decades valuable work has been carried out in the area of text categorization (Joachims, Thorsten, 1998; Thorsten et al., 2001), optical character recognition (Mori et al., 1992), intrusion detection (Mukkamala et al., 2002), speech recognition (Schmidt, 1996), handwritten digit recognition (Weston & Watkins, 1999) etc. All such real-world applications are essentially multi-class classification problems. Multi-class classification is intrinsically harder than binary classification problem because the classification has to learn to construct a greater number of separation boundaries or relations. Classification error rate is greater in multi-class problem than that of binary as there can be error in determination of any one of the decision boundaries or relations.
There are basically two types of multi-class classification algorithms. The first type deals directly with multiple values in the target field i.e. K- Nearest Neighbor, Naive Bayes, classification trees in the class etc. Intuitively, these methods can be interpreted as trying to construct a conditional probability density for each class, then classifying by selecting the class with maximum a posteriori probability. For data with high dimensional input space and very few samples per class, it is very difficult to construct accurate densities. While the other approaches decompose the multi -class problem into a set of binary problems and then combining them to make a final multi-class classifier. This group contains support vector machines, boosting and more generally, any binary classifier. In certain settings the later approach results in better performance then the multiple target approaches.
Support Vector Machine (SVM) is most commonly and popular classification technique used in literature. Support Vector Machines (SVMs) are based on statistical learning theory developed by Vapnik (Vapnik, 1998; Corts & Vapnik, 1995; Corts & Vapnik, 1995; Burges, 1998). It transforms the training vectors into a high-dimensional feature space, labeling each vector by its class. It classifies data by determining a set of support vectors, which are members of the set of training inputs that outline a hyperplane in feature space (Kittler & Hojjatoleslami, 1998). It is based on the idea of Structural risk minimization, which minimizes the generalization error. The number of free parameters used in the SVM depends on the margin that separates the data points and not on the number of input features. SVM provides a generic technique to fit the surface of the hyperplane to the data by employing an appropriate kernel function. Use of a kernel function enables the curse of dimensionality to be addressed, and the solution implicitly contains support vectors that provide a description of the significant data for classification (Scholkopf & smola, 2002).