Novel Efficient Classifiers Based on Data Cube

Abstract

Existing decision tree algorithms need to recursively partition dataset into subsets according to some splitting criteria. For large data sets, this requires multiple passes of original dataset and therefore is often infeasible in many applications. In this article we use statistics trees to compute the data cube and then build a decision tree on top of it. Mining on aggregated data will be much more efficient than directly mining on flat data files or relational databases. Since data cube server is usually a required component in an analytical system for answering OLAP queries, we essentially provide “free” classification. Our new algorithm generates trees of the same prediction accuracy as existing decision tree algorithms such as SPRINT and RainForest, but improves performance significantly. In this article we also give a system architecture that integrates DBMS, OLAP, and data mining seamlessly.