Skip to main content
SpringerLink
Log in

On the Effect of the Form of the Posterior Approximation in Variational Learning of ICA Models

  • Published:
Neural Processing Letters Aims and scope Submit manuscript

Abstract

We show that the choice of posterior approximation affects the solution found in Bayesian variational learning of linear independent component analysis models. Assuming the sources to be independent a posteriori favours a solution which has orthogonal mixing vectors. Linear mixing models with either temporally correlated sources or non-Gaussian source models are considered but the analysis extends to nonlinear mixtures as well.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Abbreviations

ICA:

Independent component analysis

MoG:

Mixture of Gaussians

PCA:

Principal component analysis

References

  1. H. Attias (1999) ArticleTitleIndependent factor analysis Neural Computation 11 IssueID4 803–851 Occurrence Handle10.1162/089976699300016458 Occurrence Handle10226184

    Article  PubMed  Google Scholar 

  2. Lappalainen, H.: Ensemble learning for independent component analysis, In: Proceedings of Int. Workshop on Independent Component Analysis and Signal Separation (ICA99), pp. 7–12, Aussois: France, 1999.

  3. Miskin, J. and MacKay, D. J. C.: Ensemble learning for blind source separation, In: S. Roberts and R. Everson (eds.),Independent Component Analysis: Principles and Practice, pp. 209–233, Cambridge University Press, 2001.

  4. Choudrey, R., Penny, W. and Roberts, S.: An ensemble learning approach to independent component analysis, In: Proceeding of the IEEE Workshop on Neural Networks for Signal Processing, pp. 435–444, IEEE Press: Sydney, Australia, 2000.

  5. Valpola, H.: Nonlinear independent component analysis using ensemble learning: theory, In: Proceeding of International Workshop on Independent Component Analysis and Blind Signal Separation (ICA2000), pp. 251–256, Helsinki, Finland, 2000.

  6. K. Chan T. Lee T. J. Sejnowski (2002) ArticleTitleVariational learning of clusters of undercomplete nonsymmetric independent components Journal of Machine Learning Research 3 99–114 Occurrence Handle10.1162/153244303768966120 Occurrence HandleMR1966055

    Article  MathSciNet  Google Scholar 

  7. K. Chan T. Lee T. J. Sejnowski (2003) ArticleTitleVariational bayesian learning of ICA with missing data Neural Computation 15 IssueID8 1991–2011 Occurrence Handle10.1162/08997660360675116

    Article  Google Scholar 

  8. H. Valpola J. Karhunen (2002) ArticleTitleAn unsupervised ensemble learning method for nonlinear dynamic state-space models Neural computation 14 IssueID11 2647–2692 Occurrence Handle10.1162/089976602760408017

    Article  Google Scholar 

  9. Ilin, A. and Valpola, H.: On the effect of the form of the posterior approximation in variational learning of ICA models, In: Proceeding of the 4th International Conference on Independent Component Analysis and Blind Signal Separation (ICA2003), pp. 915–920, Nara: Japan, 2003.

  10. Barber, D. and Bishop, C. Ensemble learning for multi-layer networks, In: M. Jordan, M. Kearns and S. Solla (eds.), Advances in Neural Information Processing Systems 10, pp. 395–401, The MIT Press: Cambridge, MA, USA.

  11. Z. Ghahramani G. E. Hinton (2000) ArticleTitleVariational learning for switching state-space models Neural computation 12 IssueID4 963–996 Occurrence Handle10.1162/089976600300015619

    Article  Google Scholar 

  12. Hyvärinen, A., Karhunen, J. and Oja, E.: Independent Component Analysis, Wiley, 2001.

  13. A. Belouchrani K. A. Meraim J.-F. Cardoso E. Moulines (1997) ArticleTitleA blind source separation technique based on second order statistics IEEE Trans. on Signal Processing 45 IssueID2 434–444 Occurrence Handle10.1109/78.554307

    Article  Google Scholar 

  14. Ziehe, A., Müller, K.-R., Nolte, G., Mackert, B.-M. and Cuiro, G.: Artifact reduction in manetoneurography based on time-delayed second order correlations, Technical Report 31, GMD–Forschungszentrum Informationstechnik GmbH, 1998.

  15. Tong, L., Soo, V., Liu, R. and Huang, Y.: Amuse: a new blind identification algorithm, In: Proceedings ISCAS, New Orleans, USA, 1990.

  16. Z. Ghahramani M. Beal (2001) Propagation algorithms for variational Bayesian learning T. Leen T. Dietterich V. Tresp (Eds) Advances in Neural Information Processing Systems 13 The MIT Press Cambridge, MA, USA 507–513

    Google Scholar 

  17. Valpola, H., Honkela, A., Harva, M., Ilin, A., Raiko, T. and Östman, T.: Bayes blocks software library, http://www.cis.hut.fi/projects/bayes/sofware/, 2003.

  18. Valpola, H., Raiko, T. and Karhunen, J.: Building blocks for hierarchical latent variable models, In: Proceedings of the 3rd International Conference on Independent Component Analysis and Signal Separation (ICA2001), pp. 710–715, San Diego, USA, 2001.

  19. Ilin A.: Matlab code for variational bayesian learning of linear ICA models, http://www.cis.hut.fi/projects/bayes/sofware/np12004/, 2004.

  20. H. Valpola M. Harva J. Karhunen (2004) ArticleTitleHierarchical models of variance sources Signal Processing 84 IssueID2 267–282 Occurrence Handle10.1016/j.sigpro.2003.10.014

    Article  Google Scholar 

  21. M. Harva (2004) Hierarchical variance models of image sequences Helsinki University of Technology Espoo

    Google Scholar 

  22. P. A. d. F. R. Højen-Sørensen O. Winter L. K. Hansen (2002) ArticleTitleMean-field approaches to independent component analysis Neural Computation 14 IssueID4 889–918 Occurrence Handle10.1162/089976602317319009 Occurrence Handle11936966

    Article  PubMed  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Neural Networks Research Centre, Helsinki University of Technology, P.O. Box 5400, 02015, HUT, Finland

    Alexander Ilin

  2. Laboratory of Computational Engineering, Helsinki University of Technology, P.O. Box 9203, 02015, HUT, Finland

    Harri Valpola

Authors
  1. Alexander Ilin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Harri Valpola
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Alexander Ilin.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Ilin, A., Valpola, H. On the Effect of the Form of the Posterior Approximation in Variational Learning of ICA Models. Neural Process Lett 22, 183–204 (2005). https://doi.org/10.1007/s11063-005-5265-0

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11063-005-5265-0

Keywords

Search

Navigation