Fisher Linear Discriminant
Fisher linear discriminant (FLD) (Duda, Hart, & Stork, 2001) operates by learning a discriminant matrix which maps a d-dimensional input space into an r-dimensional feature space by maximizing the multiple Fisher discriminant criterion.Specifically, a Fisher discriminant matrix is an optimal solution of the following optimization model:
.
(1)Here is an arbitrary matrix, and and are the between- and within- class scatter matrices, and is the determinant of a square matrix.
The between-class scatter matrix SB and the within-class scatter matrix SW are defined as follows,, (2) and
.
(3)Here Ni and are respectively the number and the mean of samples from the ith class , the mean of samples from all classes, and l the number of classes.
It has been proved that if is nonsingular, the matrix composed of unit eigenvectors of the matrix corresponding to the first r largest eigenvalues is an optimal solution of the optimization model defined in Eq. (1) (Wilks, 1962). The matrix is the Fisher discriminant matrix commonly used in Fisher linear discriminant.
Since the matrix is usually asymmetric, Fisher discriminant vectors, i.e. column vectors of the Fisher discriminant matrix are unnecessary orthogonal to each other.