Nonlinear Adaptive Filtering with a Family of Kernel Affine Projection Algorithms

Introduction

Linear adaptive filters have been widely used in a variety of applications, e.g., mobile communications, video processing etc. In system identification applications, the main goal is to identify an unknown system using an adaptive filter. The well known normalized least-mean-square (NLMS) algorithm has been widely used, but converges very slowly (Sayed, 2003). The affine projection algorithm (APA) provides a much improved convergence speed (Sayed, 2003) and in some situations close to that of the more complex recursive least-squares (RLS) algorithm. Another related family of linear algorithms is the proportionate APA (PAPA) family. The update of each coefficient of the filter is made independently of the others, by adjusting the adaptation step-size in proportion to the magnitude of the estimated filter coefficient (Deng, 2006). The proportionate normalized least-mean-square (PNLMS) algorithm (Duttweiler, 2000) was one of the first proportionate-type algorithms. All of the APA/PAPA algorithms require the computation of the inverse of a matrix, which is a complex operation and can lead to algorithm instability if finite precision is used. One of the most efficient methods to avoid the matrix inversion is the use of the Dichotomous Coordinate Descent (DCD) methods (Zakharov & Tozer, 2004), (Zakharov, White & Liu, 2008) and (Albu, Bouchard & Zakharov, 2007).

As an extension of the linear counterparts, kernel adaptive filters have been proposed to identify non-linear systems (Liu, Principe & Haykin, 2010). Kernel adaptive filters are derived by applying the kernel methods to linear adaptive filters, and many such algorithms were proposed, (e.g., the kernel recursive least squares (KRLS) (Engel, Mannor & Meir, 2004), the kernel least mean square (KLMS) (Liu, Pokharel & Principe, 2008), the kernel normalized LMS (KNLMS) (Richard, Bermudez, & Honeine, 2009), kernel affine projection (KAPA) (Richard et al., 2009) and its efficient implementation (Albu, Coltuc, Rotaru & Nishikawa, 2013), etc. The application of a reproducing kernel Hilbert space (RKHS) methodology blended with many signal processing techniques is a powerful tool in solving numerous practical artificial intelligence or control systems problems (Müller, Adali, Fukumizu, Principe, & Theodoridis, 2013).

This chapter explains the similarities between the linear algorithms and their corresponding kernel versions. We start with a short description of the related work and of the linear NLMS, APA, and PAPA algorithms. Next, the kernel AP algorithm with a coherence criterion is presented. Following the resemblance of APA with KAPA, the proportionality criterion is used in order to derive a new algorithm called, kernel PAPA (KPAPA). The KNLMS and a new proportionate version of it are mentioned as particular cases of KAPA and KPAPA respectively. The recent research directions in the development of the reduced complexity algorithms based on DCD iterations are also discussed. Next, the chapter investigates the reduction of the numerical complexity of the kernel algorithms, the fine tuning of the parameters of the algorithms, etc. Their importance and significance for improving the overall performance is discussed. Several simulation results are shown in the Simulation section. Finally, the conclusions and future research direction conclude this work.