Article Preview
Top1. Introduction
In binary classification, when the proportion of one class in the target variable is significantly smaller than the other class (imbalanced data), one faces the problem of classifying a rare event, or imbalanced classification. Traditional classification modeling techniques such as logistic regression and decision trees generally work quite well with a balanced data. However, with a rare event variable, when the data set is very large, and the event of interest is very small, most of these classification methods typically fail to detect the rare event. This can be particularly true when the percentage of the rare event is extremely small, as in the example presented here, where the rare event occurs in less than 1% of the observations. In this situation, all observations are predicted as being in the group consisting of the more dominant or common event.
For example, in the case of a binary target variable, if the rare event makes up only about 1% of the sample, these methods can be correct 99% of the time by predicting all items as falling in the event with the greater probability. In this way, these methods are correct 99% of the time, but may fail to predict the rare event which is most often the aim of the analysis. Unfortunately, many times these rare events, such as fraud, terrorism, etc., are important to predict.
To try to correct this problem, there are possible methods that have been suggested. These methods can be at either the data level, i.e. balancing the data before the modelling stage, or at the algorithmic level, i.e. tuning the existing algorithms to adapt the imbalance of the data.
Regarding the algorithmic approaches, Veropoulos et al. (1999) proposed using two different cost parameters for two classes in the objective functions of support vector machines (SVM) instead of having only one cost parameter, as in the traditional SVM. Imam et al. (2006) suggested modifying the decision boundary to remove the bias toward the majority class. Also modifying the SVM, Wu et al. (2003) used conformally transformed kernels to extend the class boundary region. Several other works on SVM can be found in Batuwita et al. (2013). Maalouf et al. (2016) proposed a method working on logistic regression. The authors used the truncated Newton method in prior correction logistic regression with an added regularization term. Many other modern predictive models have been involved in working with imbalanced data. Wang et al. (2016) proposed using a new loss function in training a deep neural network. Buda et al. (2017) gave an extensive study on the performances of convolutional neural network with imbalanced data.
Along with approaches at the algorithmic level, much research has been conducted at the data level. Methods at this level aim to balance the data before fitting it into conventional classification models. In general, these methods can be divided into two categories: undersampling, i.e. reducing the size of the majority class, and oversampling, i.e. increasing the size of the minority class. Chawla et al (2002) proposed artificially creating observations for the minority class with the idea of linear interpolation. Their method is called Synthetic Minority Over-Sampling Technique (SMOTE). Menardi et al. (2012) proposed ROSE (Random Over-Sampling Examples) where they generated observations for the minority class using its estimated kernel density. Both ROSE and SMOTE are of oversampling method. Although undersampling approaches provide a fast running time for computation, their main drawback is removing a large portion of the data, which may contain important information. The main disadvantages of over-sampling approaches are the slower running time for computation and, in many cases, poor accuracy due to possible noise these methods add to the data.