Three easy ways for separating nonlinear mixtures?
Section snippets
Model and problem
Consider N samples of the m-dimension observed random vector , modeled bywhere is an unknown mixing mapping assumed invertible, is an unknown n-dimensional source vector containing the source signals s1,s2,…,sn, which are assumed to be statistically independent, and is an additive noise, independent of the sources.
Such a model is usual in multidimensional signal processing, where each sensor receives an unknown superimposition of unknown source signals at time instants t=1,…,N.
Existence and uniqueness of nonlinear ICA and BSS
Several authors [18], [24], [26], [39], [40] have recently addressed the important issues on the existence and uniqueness of solutions for the nonlinear ICA and BSS problems. Their main results, which are direct consequences of Darmois's results on factorial analysis [16], are reported in this section.
Smooth mappings
Recently, multi-layer perceptron (MLP) networks (see [19]) have been used in [2], [48] for estimating the generic nonlinear mappings . Especially, Almeida conjectured that smooth mappings providing by MLP networks leads to a regularization sufficient for ensuring that nonlinear ICA leads to nonlinear BSS, too. However, the following example [5] shows that smoothness alone is not sufficient for separation.
Without a loss of generality, consider two independent random variables which
Structural constraints
A natural way of regularizing the solution consists in looking for separating mappings belonging to a specific subspace . To characterize the indeterminacies for this specific model , one must solve the tricky independence preservation equation which can be writtenwhere is the set of all the measurable compacts in (in other words, is a σ-algebra on ), and Fsi denotes the distribution function of the random variable si .
Let denote the set2
Prior information on the sources
In this section we show that prior information on the sources can simplify or relax the indeterminacies. The first example takes into account that sources are bounded. The second example exploits the temporal correlation of the sources.
Concluding remarks
In this paper, we have considered ICA and BSS problems for nonlinear mixture models. It appears clearly BSS and ICA are difficult and ill-posed problems, and regularization is necessary for actually achieving ICA solutions which coincide to BSS.
In this purpose, two main ways can be used. First, solving the nonlinear BSS problem appropriately using only the independence assumption is possible only if mixtures as well as separation structure are structurally constrained: for example
Acknowledgements
This work has been supported by the European Commission project BLISS (IST-1999-14190). The authors thank Dr. A. Taleb for contributing many of the results presented in this paper, and the three anonymous reviewers for their very relevant and detailed comments which contributed to improve this paper.
References (49)
Independent component analysis, a new concept?
Signal Processing
(1994)- et al.
Multi-tag radio-frequency identification systems based on new blind source separation neural networks
Neurocomputing
(2002) - et al.
Nonlinear independent component analysis: existence and uniqueness results
Neural Networks
(1999) - et al.
Blind separation of sources, Part I: an adaptive algorithm based on a neuromimetic architecture
Signal Processing
(1991) - et al.
A neural net for blind separation of nonstationary signals
Neural Networks
(1995) - et al.
Information-theoretic approach to blind separation of sources in non-linear mixture
Signal Processing
(1998) Lectures on Functional Equations and Their Applications
(1966)- L. Almeida, Linear and nonlinear ICA based on mutual information, in: Proceedings of IEEE 2000 Adaptive Systems for...
- L. Almeida, C. Jutten, H. Valpola, Realistic models of nonlinear mixtures, BLISS (IST1999-14190) Project Report D5,...
- S. Amari, A. Cichocki, H. Yang, A new learning algorithm for blind signal separation, in: Advances in Neural...
An information-maximization approach to blind separation and blind deconvolution
Neural Computation
A blind source separation technique based on second order statistics
IEEE Trans. Signal Processing
Blind signal separation: statistical principles
Proc. IEEE
Equivariant adaptive source separation
IEEE Trans. Signal Processing
Blind beamforming for non gaussian signals
IEE Proc.-F
Adaptive Blind Signal and Image Processing—Learning Algorithms and Applications
Robust learning algorithm for blind separation of signals
Electron. Lett.
Neural Networks—A Comprehensive Foundation
On the separability of nonlinear mixtures of temporally correlated sources
IEEE Signal Processing Lett.
Cited by (68)
Recurrent networks for separating extractable-target nonlinear mixtures. Part II. Blind configurations
2013, Signal ProcessingCitation Excerpt :A natural idea for reducing these indeterminacies is to constrain the structure of mixing and separating models to belong to a certain set of transformations. This supplementary constraint can be viewed as a regularization of the initially ill-posed problem [9,10]. Thus, several authors have studied specific classes of nonlinear mixing models like post-nonlinear mixtures [8,11,12], linear-quadratic mixtures [13–19], linearizable mappings [20] or nonlinear mixtures encountered on gas or chemical sensors [21,22] (see also [9] for a more general framework).
Unearthing InSights into Mars: Unsupervised Source Separation with Limited Data
2023, Proceedings of Machine Learning ResearchBLIND SEPARATION OF LINEAR-QUADRATIC MIXTURES OF MUTUALLY INDEPENDENT AND AUTOCORRELATED SOURCES
2022, ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings