Predicting Student Retention by Linear Programming Discriminant Analysis

2.1. Problem Description

Linear discriminant analysis is used in order to find a hyperplane that separates the two sets of students, in the best way achievable, thus we have:

In order to do that, we find the correct and the erroneous distance that each data point has to the hyperplane (see Figure 1). Correct distances are denoted by β-s and erroneous distances by α-s, which we add to each equation. The signs of β and α, will depend on whether the student will actually drop out, or graduate. Thus our objective is to simultaneously maximize the sum of β-s and to minimize the sum of α-s and we arrive at the following set of equations (note that for one student, only either α or β will be different from zero, depending on the accuracy of the prediction – or both will be zero, if the student lies on the discriminatory hyperplane):

Figure 1.

Separating hyperplane determining the correct and erroneous distances of data points. The color of the boundary line indicates whether the person has been successfully identified as either graduating or dropping out (black – successful identification; red – incorrect assessment) and the color inside the star indicates whether the person has really graduated or dropped out (blue – graduated; white – dropped out).

Note that some training set cases will indeed be placed on the hyperplane due to the nature of the linear programming methods used. The results still do only have a binary interpretation, though, as the validation and testing cases virtually never are laying on the hyperplane.