Fuzzification of Euclidean Space Approach in Machine Learning Techniques

1. Introduction

Fuzzification algorithms have been applied in most machine learning techniques to provide more human-like behavior and are successful in increasing the performance and accuracy of the classification results. Fuzzy logic introduces to machine learning techniques a framework for dealing quantitatively, mathematically, and logically with semantic and ambiguous concepts (Kyoomarsi et al. 2009). The membership of data points in a set or class label is not crisp but can be specified as a degree of membership. The machine learning techniques under investigation in this paper are C-Means clustering and support vector machine (SVM), where these techniques are fuzzified to generate FCM clustering (FCM) and fuzzy SVM, respectively. In instance-based techniques such as the C-Means clustering technique, fuzzy logic is used to determine the proximity of a given instance to the training set’s instances (Kotsiantis 2007). Fuzzy logic allows data instances to belong to two or more clusters where it is based on minimizing an objective or dissimilarity function (Gewenigera et al. 2010). With FCM, the centroid of a cluster is computed as the mean of all points, weighted by their degree of belonging to the cluster according to their proximity in the feature space. The degree of being in a certain cluster is related to the inverse of the distance to the cluster. SVM is a non-linear binary classification algorithm based on the theory of structural risk minimization. SVM solves complex classification tasks without suffering from the over-fitting problems that affect other classification algorithms. Computationally speaking, the SVM training problem is a convex quadratic programming problem, meaning local minima are not a problem (Cortes et al. 1995). In fuzzy SVM, the fuzzy membership values are calculated based on the distribution of training vectors where the outliers are given proportionally smaller membership values compared to the other training vectors (Lee et. al. 2006), (Shilton et. al. 2007).

The problem in such fuzzified techniques is that they apply the fuzzy logic concept on the level of objects and ignore the features composing the objects. Each object has a degree of membership in the class labels in the learning problem. For multivariate objects, the features of the objects have different degrees of relevance to the target class labels. When reducing the number of features by selecting the best features or extracting a lower number of features from the higher ones, the reduced features still have different relevance to the classification problem, while irrelevant features are ignored (Janecek et. al. 2008). Consequently, classifiers deal crisply with features without considering the degree of relevance: Either the full data set is used or only the selected features. The degree of relevance is calculated with feature evaluation techniques such as ChiMerge (Abdelwadood et. al. 2007). This technique successfully ranks continuous and discrete features. Using the degree of relevance in classification techniques could enhance their accuracy.