Variations on Associative Classifiers and Classification Results Analyses

Introduction

The process of creating an associative classifier from a training data set has three main phases: (1) mining the training data for association rules and keeping only those that can classify instances, (2) pruning the mined rules to weed out irrelevant or noisy rules, and (3) selecting and combining the rules to classify unknown items. Within each of these steps, there is a great deal of potential for variation and improvement. The first three sections describe each of the three phases of the associative classification process in detail. In addition, three variations on this process are described, each within the context of the relevant classification phase. Each of these variations are outlined briefly in the following paragraphs and described using a running example of a department store sales dataset. To put the preceding paragraph into this context, we can imagine a store with data from previous months on the sales of various items in the store, and an assessment from a manager of whether the items were worth stocking. An associative classifier for this context would create a set of rules relating items’ sales figures to their overall profitability, and allow the manager to assess the current month’s stock based on data accumulated from previous months.

The first variation considers data sets with re-occurring items. Associative classifiers are typically concerned only with the presence of an attribute, which ignores potentially valuable information about the number of occurrences of that attribute. For example, in a text classification context, the number of occurrences of a word in a document and in a collection are crucial indicators of its importance. Or, to use the department store example, knowing how many shirts were sold might be more important than knowing whether or not any shirts were sold. A classification model by Rak et al (Rak, 2005) considers the number of occurrences of an attribute both in generating rules and in classifying items according to those rules. In the rule generation phase, the model increments a rule’s support by an amount proportional to the number of attribute occurrences. In the item classification phase, the model uses the Cosine Measure to measure the similarity between an item and rules which have re-occurring attributes.

The second variation presents a classifier which works with both positive and negative rules. Negative rules either use attribute/negated value pairs, or imply a negative classification, and can capture patterns and relationships that would be missed by positive only rule-based associative classifiers. In the department store context, knowing that a store did not sell any shirts of a certain brand could help a manager decide not to stock more shirts of that brand. Generating a complete set of negative association rules from a set of positive rules is a very difficult task, and can result in an exponential growth in the number of association rules. A method developed by Antonie and Zaïane (Antonie, 2004c) deals with this issue in two ways. First, the negative rules generated are restricted to those where either the entire antecedent or consequent is negated. Thus, a rule that identifies one brand of shirt that did not sell and another that did would not be generated using this method. Second, the correlation coefficient between a pattern and a frequent itemset is used to guide the generation of negative association rules. This method also incorporates both negative and positive rules into the item classification process.