In this chapter the authors present an overview of pattern classification. In particular, they focus on the mathematical background of pattern classification rather than discussing the practical analysis of various pattern classification methods, and present the derivation of classification rules from a mathematical aspect. First, the authors define the pattern space without the loss of generality. Then, the categorisation of pattern classification is presented according to the design of classification systems. The mathematical formulation of each category of pattern classification is also given. Theoretical discussion using mathematical formulations is presented for distance-based pattern classification and statistical pattern classification. For statistical pattern classification, the standard assumption is made where patterns from each class follow normal distributions with different means and variances.
TopIntroduction
Many problems can be seen as pattern classification problems (Duda, Hart, and Stork, 2001). There are a lot of approaches to pattern classification such as Bayesian (Gelman, Carlin, Stern, and Rubin, 2003), fuzzy (Ishibuchi, Nakashima, and Nii, 2004), neural networks (Bishop, 1995), and support vector machines (Abe 2010). These are only a part of various approaches to pattern classification. Pattern classification is a process of mapping from an observed pattern into one of pre-specified concepts. A concept here is called a “class”. For example, in the case of medical diagnosis, the classes are often “benign” and “malignant”. A general framework of pattern classification is shown in Figure 1. Raw input data is first collated. From this information, a raw input vector is then generated. Often, efficient classification cannot be performed using such raw input patterns. Thus, it is necessary to reduce the dimensionality of patterns into a reasonably small number. The actual input vector to a classification system is generated by reducing the number of dimensionalities of pattern vectors. Relevant features for classification are extracted by using some feature selection method such as sequential feature selection (Kohavi and John, 1996) and principal component analysis (Fukunaga, 1990). A normalisation process can be also performed to generate pattern vectors for classification systems. For example, in Figure 1, a normalisation process is performed so that attribute values range in a unit interval of [0.0, 1.0] after the number of features is reduced to five. A classification system then decides which class an input vector belongs to. This chapter describes elements of pattern classification. Specifically, the fundamentals of distance-based and statistical pattern classification are given.
Figure 1. A general framework of classification for medical imaging
TopPattern Space
Let us denote that an input pattern for classification systems is an n-dimensional numerical vector. Without loss of generality, we also assume that each element of the n-dimensional vector is normalised into a unit interval [0.0, 1.0]. In this case, any n-dimensional input patterns reside in an n-dimensional hypercube . We refer to this hypercube as pattern space. The task of classification systems is then to divide the pattern space into a number of regions that correspond to pre-specified classes. For example, in Figure 2, the two-dimensional pattern space for a three-class problem is divided into three regions with each region corresponding to one of the three classes. A representation that divides the pattern space is called a hypothesis. Thus, classification systems generate a hypothesis that is consistent with a given set of training patterns. The hypothesis is also called classification rule.
Figure 2. Division of pattern space (two-dimensional case)
TopCategorisation Of Classification Systems
There are several types of classification systems. Three major types are described below:
- 1.
Distance-based type: An input pattern is classified as the class with the nearest training pattern in terms of a pre-defined distance measure.
- 2.
Statistical type: An input pattern is classified based on the statistical property that is derived from a given set of training patterns.
- 3.
Structure-based type: An input pattern is classified by using structural information of features. Structural information of features is usually defined by a human expert.