A Comprehensive Feature Selection Approach for Machine Learning

2 Literature Survey

Any typical machine-learning algorithm involves extracting the output from a given set of input variables. These variables are also called as features in the input space, the input space can be represented as a vector in The algorithms are developed assuming a functional relationship between the input vector and the output variable. . This assumption may not always be true; there may be some variables that may not have any influence on the output variables, to filter such variables numerous algorithms for ranking such variables are proposed. In a paper by Alain Rakotomamon new methods for variable selection are proposed for Support Vector Machines. Initial developments came from Guyon & Elisseeff (Guyon & Elisseeff, 2003). This paper contained an algorithm for selecting genes that are relevant for cancer classification problem. His goal was to find a subset of size r among d variables that maximizes the performance of predictor. The criterion is used where: ; ɸ is the kernel used in svm , w and b are the parameters for a particular model. The ranking criterion that has been used is his algorithm runs in linear time.

Similarly other methods for variable selection has been proposed for neural networks also, the starting development in this area was done by Garson(Beck, 2018) which was later modified by Goh (Goh, 1995) to rank the variable importance, there a simple equation that is based on connection weights were proposed, Qik determined the relative importance of i-th input on k-th output. However, the main disadvantage of the Garson’s algorithm was it used absolute values of weights that sometimes led to erroneous results. This disadvantage was removed by Olden(Olden et al., 2004). In a latest paper by Liu and Zaho (Liu & Zhao, 2017) a variable importance weighted random forest is used for classification and regression, the problem was the performance of random forest fall down when the number of features increased. In another paper by Kvalheim et al (Kvalheim et al., 2014) variable importance in latent variable regression model is proposed they presented some new graphical tools for improved interpretation of latent variable regression models that could assist in variable selection. Therefore, for different AI tools you have different algorithms. In this paper, present a novel algorithm that runs for any type of tool someone is using and provides the criterion to decide whether the results are meaningful.