This chapter introduces a committee of networks for estimating missing data. The first committee of networks consists of multi-layer perceptrons (MLPs), support vector machines (SVMs) and radial basis functions (RBFs). The committee was constructed from a weighted combination of these three networks. The second, third and fourth committees of networks were evolved using a genetic programming approach and used the MLPs, RBFs and SVMs, respectively. The committee of networks was collectively implemented with hybrid particle-swarm optimization and a genetic algorithm for missing data estimation. They were tested on an artificial taster as well as HIV datasets and then compared to the individual multi-layer perceptron, radial basis functions and support vector regression for missing data estimation. It was found that the committee of network approach provided improved results over the three methods acting individually. However, this improvement comes with a higher computational load than does using the individual approaches. Furthermore, it is found that evolving a committee method was a good way of constructing a committee.
TopIntroduction
Several techniques have been introduced for missing data estimation (Abdella, 2005; Abdella & Marwala, 2005, 2006; Allison, 2000). A number of of these methods make use of machine learning approaches to accomplish this mission. In this chapter, committees of machine learning algorithms are used for missing data estimation. The principal incentive for using the committees of networks technique is from the intuitive logic that many ‘heads’ are better than one and, therefore, using many networks is thus better than using one.
Du, Zhai, and Wan (2007) used a committee of probabilistic radial basis function neural networks to identify palm prints while Anthony (2007) studied the generalization error of fixed combinations of classifiers. Sheikh-Ahmad et al. (2007) used a committee of neural networks for force prediction models in a milling process; Marwala (2000) used a committee of multi-layered perceptrons for damage identification in structures, whereas Marwala (2001a) implemented a probabilistic fault identification process in structures using a committee of neural networks and vibration data. Furthermore, Marwala et al. (2001) used a committee of agents and genetic programming to evolve a stock market prediction system.
In this chapter, the fact that the committee of networks approach will yield superior performance to the stand-alone networks for regression problems is extended to the missing data estimation problem and then demonstrated mathematically. After that, it is tested on an artificial taster and HIV prediction problems that were described in earlier chapters. Two approaches are pursued in constructing committee methods. These are the traditional approach proposed by Perrone and Cooper (1993), and the second approach is evolving the committee method by using a type of evolutionary programming known as genetic programming.
TopMissing Data Approach
In this chapter, the missing data estimation approach that was adopted entailed the use of a committee consisting of an auto-associative Multi-Layer Perceptron (MLP), Radial Basis Function (RBF) and a Support Vector Machine (SVM). These auto-associative networks were trained to predict their own input vectors and therefore were called recall networks. The results obtained from these networks were averaged in a weighted manner. Thus far, the MLP, RBF and SVM, which are the members of the committee of networks presented in this chapter, have each been individually used for missing data estimation (Nelwamondo, 2008) as was shown in earlier chapters. Pelckmans et al. (2005) proposed a technique for handling missing values in support-vector machine classifiers while Junninen et al. (2004) developed missing values imputation methods for air quality data sets. On the other hand, Chandramouli et al. (2007) used artificial neural networks for estimating missing microbial concentrations in a riverine database whereas Zhong, Lingras, and Sharma (2004) estimated missing traffic counts using factor, genetic, neural, and regression techniques. In this chapter, just as in Zhong, Lingras, and Sharma (2004), regression methods are used for estimating the missing data. The missing data estimation procedure adopted in this chapter is composed of two components i.e., regression and optimization components. The missing-data estimation error equation was written in earlier chapters as follows:
