Semi Blind Source Separation for Application in Machine Learning

Introduction

Blind source separation (BSS), is the separation of a set of signals from a set of mixed signals, without the help of information (or with very little information) about the source signals or the mixing. BSS relies on the hypothesis that the source signals do not correlate with each other. For instance, the signals may be mutually statistically independent or decorrelated. BSS thus separates a set of signals into a set of other signals, such that the steadiness of each resulting signal is maximized, and the regularity between the signals is minimized (i.e. statistical independence is maximized). BSS is an unsupervised learning which is closely related to relate to the problem of density estimation in statistics. However unsupervised learning also encompasses many other techniques that seek to summarize and explain key features of the data. The basic method of unsupervised learning is clustering technique. Another method is blind source separation based on Independent Component Analysis (ICA) (Bell & Sejnowski, 1995; Hyvärinen, Karhunen, & Oja, 2001; Lee, 1998).

BSS can be seen as an extension to the classical methods of Principal Component Analysis and Factor Analysis. BSS is a much richer class of techniques, however, capable of finding the sources when the classical methods, implicitly or explicitly based on Gaussian models, fail completely. In many cases, the measurements are given as a set of parallel signals or time series. Typical examples are mixtures of simultaneous sounds or human voices that have been picked up by several microphones, muscle activities measurements from multiple Electromyography (EMG) sensors, several radio signals arriving at a portable phone, or multiple parallel time series obtained from some industrial process. Perhaps the best known single methodology in BSS is ICA, in which the latent variables are non-Gaussian and mutually independent. However, criteria other than independence can be used for finding the sources. One such simple criterion is the non-negativity of the sources. Sometimes more prior information about the sources is available or is induced into the model, such as the form of their probability densities, their spectral contents, etc. Then the term “blind” is often replaced by “semiblind.”

In BSS, we assume that a set of observations is generated via an instantaneous linear mixing of underlying source signals, following the standard model: x = As. There, x is a vector with the observations, s are the underlying source signals, and A expresses the mixing process. In order to solve the BSS problem, a set of general assumptions needs to be made, either on the sources or on the mixing. ICA, one of the most widely used tools to estimate the BSS solutions, assumes the sources to be statistically independent. An additional assumption is the non-Gaussian distribution of those sources. Algorithms performing ICA can be based on concepts such as negentropy, maximum likelihood, or mutual information. There is a considerable amount of algorithms capable of performing ICA (Bell & Sejnowski, 1995; Bofill, 2000; P. Bofill & Zibulevsky, 2001; J. Cardoso & Souloumiac, 1996; J. F. Cardoso, 1997; Hyvärinen, 1999; Lewicki & Sejnowski, 2000; Zibulevsky & Pearlmutter, 2001).

In this research we propose following semi blind BSS machine learning methods for audio and bio signal applications:

• o

  Hand gesture recognition using BSS of EMG

• o

  Evaluate the use of ICA for the separation of bioelectric signals when the number of active sources may not be known and

• o

  Determining the number of sources in given set of audio recordings using BSS

The authors have explored the various machine learning applications of BSS and in audio and biomedical signal processing. The robustness of the BSS machine learning techniques has been tested using different classification tools including twin support vector machine and neural networks.