The common approach to unbalanced classification is to modify the weights, borrowing the idea from retrospective designs (see, e.g., Agrestri, 1990). This amounts to either decreasing the weight for the majority class by under-sampling or increasing the weight for the minority class by over-sampling. However, how to adjust the weights is quite an art. In the following, we shall present our procedure with justification and compare it with some alternative approaches.
To proceed, we first introduce some notations to set up the problem. Let
Let denote the distribution underlying the data. What is under modeling is the conditional probability that is equal to 1 conditioning on, i.e.,