Abstract
Independent subspace analysis (ISA) that deals with multi-dimensional independent sources, is a generalization of independent component analysis (ICA). However, all known ISA algorithms may become ineffective when the sources possess temporal structure. The innovation process instead of the original mixtures has been proposed to solve ICA problems with temporal dependencies. Here we show that this strategy can be applied to ISA as well. We demonstrate the idea on a mixture of 3D processes and also on a mixture of facial pictures used as two-dimensional deterministic sources. ISA on innovations was able to find the original subspaces, while plain ISA was not.
Chapter PDF
Similar content being viewed by others
Keywords
- Independent Component Analysis
- Innovation Process
- Independent Component Analysis
- Blind Source Separation
- Permutation Matrix
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
Jutten, C., Herault, J.: Blind separation of sources: An adaptive algorithm based on neuromimetic architecture. Signal Proc. 24, 1–10 (1991)
Comon, P.: Independent component analysis, a new concept? Signal Proc. 36, 287–314 (1994)
Bell, A.J., Sejnowski, T.J.: An information maximisation approach to blind separation and blind deconvolution. Neural Comp. 7, 1129–1159 (1995)
Bell, A.J., Sejnowski, T.J.: The independent components of natural scenes are edge filters. Vision Research 37, 3327–3338 (1997)
Hyvärinen, A.: Sparse code shrinkage: Denoising of nongaussian data by maximum likelihood estimation. Neural Comp. 11, 1739–1768 (1999)
Kiviluoto, K., Oja, E.: Independent component analysis for parallel financial time series. In: Proc. of ICONIP 1998, vol. 2, pp. 895–898 (1998)
Makeig, S., Bell, A.J., Jung, T.P., Sejnowski, T.J.: Independent component analysis of electroencephalographic data. In: Proc. of NIPS, vol. 8, pp. 145–151 (1996)
Vigário, R., Jousmaki, V., Hamalainen, M., Hari, R., Oja, E.: Independent component analysis for identification of artifacts in magnetoencephalographic recordings. In: Proc. of NIPS, vol. 10, pp. 229–235 (1997)
Hyvärinen, A., Karhunen, J., Oja, E.: Independent Component Analysis. John Wiley, New York (2001)
Cardoso, J.: Multidimensional independent component analysis. In: Proc. of ICASSP 1998, Seattle, WA, p. 1941 (1998)
Hyvärinen, A., Hoyer, P.: Emergence of phase and shift invariant features by decomposition of natural images into independent feature subspaces. Neural Comp. 12, 1705–1720 (2000)
Akaho, S., Kiuchi, Y., Umeyama, S.: MICA: Multimodal independent component analysis. In: Proc. of IJCNN, pp. 927–932 (1999)
Vollgraf, R., Obermayer, K.: Multi-dimensional ICA to separate correlated sources. In: Proc. of NIPS, vol. 14, pp. 993–1000 (2001)
Bach, F.R., Jordan, M.I.: Finding clusters in independent component analysis. In: Proc. of ICA 2003, pp. 891–896 (2003)
Póczos, B., Lőrincz, A.: Independent subspace analysis using k-nearest neighborhood distances. In: Duch, W., Kacprzyk, J., Oja, E., Zadrożny, S. (eds.) ICANN 2005. LNCS, vol. 3697, pp. 163–168. Springer, Heidelberg (2005)
Póczos, B., Lőrincz, A.: Independent subspace analysis using geodesic spanning trees. In: Proc. of ICML, Bonn, Germany (2005) (accepted)
Hyvärinen, A.: Independent component analysis for time-dependent stochastic processes. In: Proc. of ICANN, Skövde, Sweden, pp. 541–546 (1998)
Choi, S.: Acoustic source separation: Fundamental issues. In: Proc. of ICSP, Seoul, Korea, pp. 505–510 (1999)
Cheung, Y., Xu, L.: Dual multivariate auto-regressive modeling in state space for temporal signal separation. IEEE Trans. on Sys. Man. and Cyb. 33, 386–398
Yukich, J.E.: Probability Theory of Classical Euclidean Optimization Problems. In: Estublier, J. (ed.) SCM 1999. LNCS, vol. 1675. Springer, Heidelberg (1999)
Costa, J.A., Hero, A.O.: Manifold learning using k-nearest neighbor graphs. In: Proc. of ICASSP, Montreal, Canada (2004)
Amari, S., Cichocki, A., Yang, H.: A new learning algorithm for blind source separation. In: Proc. of NIPS, vol. 8, pp. 757–763 (1996)
Schneider, T., Neumaier, A.: Algorithm 808: ARfit - A Matlab package for the estimation of parameters and eigenmodes of multivariate autoregressive models. ACM Trans. Math. Softw. 27, 58–65 (2001)
Penny, W.D., Everson, R., Roberts, S.J.: Hidden Markov independent component analysis. In: Giroliami, M. (ed.) Advances in Independent Component Analysis, pp. 3–22. Springer, Heidelberg (2000)
Pearlmutter, B.A., Parra, L.C.: A context-sensitive generalization of ICA. In: Proc. of ICONIP 1996, Hong Kong, pp. 151–157 (1996)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2005 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Póczos, B., Takács, B., Lőrincz, A. (2005). Independent Subspace Analysis on Innovations. In: Gama, J., Camacho, R., Brazdil, P.B., Jorge, A.M., Torgo, L. (eds) Machine Learning: ECML 2005. ECML 2005. Lecture Notes in Computer Science(), vol 3720. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11564096_71
Download citation
DOI: https://doi.org/10.1007/11564096_71
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-29243-2
Online ISBN: 978-3-540-31692-3
eBook Packages: Computer ScienceComputer Science (R0)