A Clustering Rule Based Approach for Classification Problems

A Clustering Rule Based Approach for Classification Problems

Philicity K. Williams (Auburn University, USA), Caio V. Soares (Auburn University and Robert Bosch LLC, USA) and Juan E. Gilbert (Clemson University, USA)
Copyright: © 2012 |Pages: 23
DOI: 10.4018/jdwm.2012010101
OnDemand PDF Download:
No Current Special Offers


Predictive models, such as rule based classifiers, often have difficulty with incomplete data (e.g., erroneous/missing values). So, this work presents a technique used to reduce the severity of the effects of missing data on the performance of rule base classifiers using divisive data clustering. The Clustering Rule based Approach (CRA) clusters the original training data and builds a separate rule based model on the cluster wise data. The individual models are combined into a larger model and evaluated against test data. The effects of the missing attribute information for ordered and unordered rule sets is evaluated and the collective model (CRA) is experimentally used to show that its performance is less affected than the traditional model when the test data has missing attribute values, thus making it more resilient and robust to missing data.
Article Preview


Rule based algorithms are widely used and accepted to perform classification tasks due to their ease of interpretability and understanding. These classifiers build a model or rule set from training data as a set of high-quality rules, which can be used to predict the class labels of unlabeled instances (Wang & Karypis, 2003). There have been many rule based classification systems studied and proposed in a variety of applications, such as RIPPER (Cohen, 1995), FOIL (Quinlan & Gaetz, 1993), and CPAR (Yin & Han, 2003). However, if the test data used to evaluate the model is not as complete as the training data used to build the model, the classification accuracy suffers, as the model tends to follow the training data too closely. This is problematic since real world data is often incomplete. Thus, it is advantageous to have a rule set or model which is robust and can make accurate predictions when the test data is incomplete. This research explores the notion of adding robustness to rule based classifiers via divisive clustering.

Rule based classifiers are widely used because of the ease of interpretability of the models or set of rules they generate. These classifiers perform exceptionally well on complete data sets, meaning, the data is clean, correct, and does not have missing attribute values. To generate the model or train the classifier, the training process uses attributes and values relative to each other to segregate the data and generate a rule relative to a particular class. This poses a problem with incomplete data as the models produced by traditional rule based classifiers are sensitive to missing attribute values in new/unseen data.

Consider, for example, the data set provided in Table 1 and the resulting ID3 Decision Tree provided in Figure 1. The rules derived from this tree are:

Table 1.
Hiring training data
IDHighest EdEntry LevelVisa ReqHired
1000DoctorateNoUS CitizenYes
1005BachelorYesUS CitizenNo
1006DoctorateYesUS CitizenYes
1007MasterNoUS CitizenNo
1008MasterYesUS CitizenNo
1009BachelorYesUS CitizenYes

Complete Article List

Search this Journal:
Open Access Articles
Volume 17: 4 Issues (2021): Forthcoming, Available for Pre-Order
Volume 16: 4 Issues (2020)
Volume 15: 4 Issues (2019)
Volume 14: 4 Issues (2018)
Volume 13: 4 Issues (2017)
Volume 12: 4 Issues (2016)
Volume 11: 4 Issues (2015)
Volume 10: 4 Issues (2014)
Volume 9: 4 Issues (2013)
Volume 8: 4 Issues (2012)
Volume 7: 4 Issues (2011)
Volume 6: 4 Issues (2010)
Volume 5: 4 Issues (2009)
Volume 4: 4 Issues (2008)
Volume 3: 4 Issues (2007)
Volume 2: 4 Issues (2006)
Volume 1: 4 Issues (2005)
View Complete Journal Contents Listing