Discriminant Criteria for Pattern Classification

Discriminant Criteria for Pattern Classification

David Zhang (Hong Kong Polytechnic University, Hong Kong), Fengxi Song (New Star Research Institute Of Applied Technology, China), Yong Xu (Harbin Institute of Technology, China) and Zhizhen Liang (Shanghai Jiao Tong University, China)
DOI: 10.4018/978-1-60566-200-8.ch003

Abstract

As mentioned in Chapter II, there are two kinds of LDA approaches: classification- oriented LDA and feature extraction-oriented LDA. In most chapters of this session of the book, we focus our attention on the feature extraction aspect of LDA for SSS problems. On the other hand,, with this chapter we present our studies on the pattern classification aspect of LDA for SSS problems. In this chapter, we present three novel classification-oriented linear discriminant criteria. The first one is large margin linear projection (LMLP) which makes full use of the characteristic of the SSS problems. The second one is the minimum norm minimum squared-error criterion which is a modification of the minimum squared-error discriminant criterion. The third one is the maximum scatter difference which is a modification of the Fisher discriminant criterion.
Chapter Preview
Top

Introduction

Linear Discriminant Function and Linear Classifier

Let 978-1-60566-200-8.ch003.m01 be a set of training samples from two classes 978-1-60566-200-8.ch003.m02 and 978-1-60566-200-8.ch003.m03 with 978-1-60566-200-8.ch003.m04 samples from 978-1-60566-200-8.ch003.m05, and let 978-1-60566-200-8.ch003.m06 be their corresponding class labels. Here 978-1-60566-200-8.ch003.m07 means that 978-1-60566-200-8.ch003.m08 belongs to 978-1-60566-200-8.ch003.m09 whereas 978-1-60566-200-8.ch003.m10 means that 978-1-60566-200-8.ch003.m11 belongs to 978-1-60566-200-8.ch003.m12. A linear discriminant function is a linear combination of the components of a feature vector 978-1-60566-200-8.ch003.m13 which can be written as:978-1-60566-200-8.ch003.m14, (1) where the vector 978-1-60566-200-8.ch003.m15 and the scalar 978-1-60566-200-8.ch003.m16 are called weight and bias respectively. The hyperplane 978-1-60566-200-8.ch003.m17 is a decision surface which is used to separate samples with positive class labels from samples with negative ones.

A linear discriminant criterion is an optimization model which is used to seek the weight for a linear discriminant function. The chief goal of classification-oriented LDA is to set up an appropriated linear discriminant criterion and to calculate the optimal projection direction, i.e. the weight. Here “optimal” means that after samples are projected onto the weight, the resultant projections of samples from two distinct classes 978-1-60566-200-8.ch003.m18 and 978-1-60566-200-8.ch003.m19 are fully separated.

Once the weight 978-1-60566-200-8.ch003.m20 has been derived from a certain linear discriminant criterion, the corresponding bias 978-1-60566-200-8.ch003.m21 can be computed using:978-1-60566-200-8.ch003.m22, (2) or978-1-60566-200-8.ch003.m23, (3) where 978-1-60566-200-8.ch003.m24 and 978-1-60566-200-8.ch003.m25 are respectively the mean training sample and the mean of training samples from the class 978-1-60566-200-8.ch003.m26. They are defined as978-1-60566-200-8.ch003.m27, (4) and

978-1-60566-200-8.ch003.m28. (5)

For simplicity, we calculate the bias using the Eq. (2) throughout this chapter.

Let 978-1-60566-200-8.ch003.m29 denote the mean of the projected training samples from the class 978-1-60566-200-8.ch003.m30. Thus, the binary linear classifier based on the weight 978-1-60566-200-8.ch003.m31 and the bias 978-1-60566-200-8.ch003.m32 is defined as follow:978-1-60566-200-8.ch003.m33, (6) which assigns a class label 978-1-60566-200-8.ch003.m34 to an unknown sample 978-1-60566-200-8.ch003.m35. Here, 978-1-60566-200-8.ch003.m36 is the sign function. That is, once the weight in a linear discriminant function has been worked out the corresponding binary linear classifier is fixed.

Complete Chapter List

Search this Book:
Reset