Cost-Sensitive Classification Using Decision Trees, Boosting and MetaCost

Cost-Sensitive Classification Using Decision Trees, Boosting and MetaCost

Kai Ming Ting (Monash University, Australia)
Copyright: © 2002 |Pages: 27
DOI: 10.4018/978-1-930708-26-6.ch003
OnDemand PDF Download:


This chapter reports results obtained from a series of studies on costsensitive classification using decision trees, boosting algorithms, and MetaCost which is a recently proposed procedure that converts an errorbased algorithm into a cost-sensitive algorithm. The studies give rise to new variants of algorithms designed for cost-sensitive classification, and provide insights into the strength and weaknesses of the algorithms. First, we describe a simple and effective heuristic of converting an error-based decision tree algorithm into a cost-sensitive one via instance weighting. The cost-sensitive version performs better than the error-based version that employs a minimum expected cost criterion during classification. Second, we report results from a study on four variants of cost-sensitive boosting algorithms. We find that boosting can be simplified for costsensitive classification. A new variant which excludes a factor used in ordinary boosting has an advantage of producing smaller trees and different trees for different scenarios; while it performs comparably to ordinary boosting in terms of cost. We find that the minimum expected cost criterion is the major contributor to the improvement of all cost-sensitive adaptations of ordinary boosting. Third, we reveal a limitation of MetaCost. We find that MetaCost retains only part of the performance of the internal classifier on which it relies. This occurs for both boosting and bagging as its internal classifier.

Complete Chapter List

Search this Book:
Table of Contents
Ruhul Sarker, Hussein A. Abbass, Charles S. Newton
Chapter 1
R. Sarker, H. Abbass, C. Newton
The terms Data Mining (DM) and Knowledge Discovery in Databases (KDD) have been used interchangeably in practice. Strictly speaking, KDD is the... Sample PDF
Introducing Data Mining and Knowledge Discovery
Chapter 2
A. M. Bagirov, A. M. Rubinov, J. Yearwood
The feature selection problem involves the selection of a subset of features that will be sufficient for the determination of structures or clusters... Sample PDF
A Heuristic Algorithm for Feature Selection Based on Optimization Techniques
Chapter 3
Kai Ming Ting
This chapter reports results obtained from a series of studies on costsensitive classification using decision trees, boosting algorithms, and... Sample PDF
Cost-Sensitive Classification Using Decision Trees, Boosting and MetaCost
Chapter 4
Agapito Ledezma, Ricardo Aler, Daniel Borrajo
Currently, the combination of several classifiers is one of the most active fields within inductive learning. Examples of such techniques are... Sample PDF
Heuristic Search-Based Stacking of Classifiers
Chapter 5
Craig M. Howard
The overall size of software packages has grown considerably over recent years. Modular programming, object-oriented design and the use of static... Sample PDF
Designing Component-Based Heuristic Search Engines for Knowledge Discovery
Chapter 6
Jose Ruiz-Shulcloper, Guillermo Sanchez-Diaz, Mongi A. Abidi
In this chapter, we expose the possibilities of the Logical Combinatorial Pattern Recognition (LCPR) tools for Clustering Large and Very Large Mixed... Sample PDF
Clustering Mixed Incomplete Data
Chapter 7
Bayesian Learning  (pages 108-121)
Paula Macrossan, Kerrie Mengersen
Learning from the Bayesian perspective can be described simply as the modification of opinion based on experience. This is in contrast to the... Sample PDF
Bayesian Learning
Chapter 8
Paul D. Scott
This chapter addresses the question of how to decide how large a sample is necessary in order to apply a particular data mining procedure to a given... Sample PDF
How Size Matters: The Role of Sampling in Data Mining
Chapter 9
The Gamma Test  (pages 142-167)
Antonia J. Jones, Dafydd Evans, Steve Margetts, Peter J. Durrant
The Gamma Test is a non-linear modelling analysis tool that allows us to quantify the extent to which a numerical input/output data set can be... Sample PDF
The Gamma Test
Chapter 10
Denny Meyer, Andrew Balemi, Chris Wearing
Neural networks are commonly used for prediction and classification when data sets are large. They have a big advantage over conventional... Sample PDF
Neural Networks - Their Use and Abuse for Small Data Sets
Chapter 11
Hyeyoung Park
Feed forward neural networks or multilayer perceptrons have been successfully applied to a number of difficult and diverse applications by using the... Sample PDF
How to Train Multilayer Perceptrons Efficiently With Large Data Sets
Chapter 12
Kevin E. Voges, Nigel K.L. Pope, Mark R. Brown
Cluster analysis is a common market segmentation technique, usually using k-means clustering. Techniques based on developments in computational... Sample PDF
Cluster Analysis of Marketing Data Examining On-line Shopping Orientation: A Comparison of K-Means and Rough Clustering Approaches
Chapter 13
Susan E. George
This chapter presents a survey of medical data mining focusing upon the use of heuristic techniques. We observe that medical mining has some unique... Sample PDF
Heuristics in Medical Data Mining
Chapter 14
A. de Carvalho, A. P. Braga, S. O. Rezende, E. Martineli, T. Ludermir
In the last few years, a large number of companies are starting to realize the value of their databases. These databases, which usually cover... Sample PDF
Understanding Credit Card User's Behaviour: A Data Mining Approach
Chapter 15
Alina Lazar
The goal of this research is to investigate and develop heuristic tools in order to extract meaningful knowledge from archeological large-scale data... Sample PDF
Heuristic Knowledge Discovery for Archaeological Data Using Genetic Algorithms and Rough Sets
About the Authors