Skip to main content
SpringerLink
Log in

Learning Incoherent Subspaces: Classification via Incoherent Dictionary Learning

  • Published:
Journal of Signal Processing Systems Aims and scope Submit manuscript

Abstract

In this article we present the supervised iterative projections and rotations (s-ipr) algorithm, a method for learning discriminative incoherent subspaces from data. We derive s-ipr as a supervised extension of our previously proposed iterative projections and rotations (ipr) algorithm for incoherent dictionary learning, and we employ it to learn incoherent sub-spaces that model signals belonging to different classes. We test our method as a feature transform for supervised classification, first by visualising transformed features from a synthetic dataset and from the ‘iris’ dataset, then by using the resulting features in a classification experiment.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Figure 1
Figure 2
Figure 3
Figure 4
Figure 5
Figure 6

Similar content being viewed by others

Notes

  1. Here e is a vector of ones.

  2. Note that the term cluster implies that a this stage the algorithm needs to make an unsupervised decision, since there is no any a-priori reason to assign a given atom to any particular class.

  3. The Matlab code used to generate the results in this Section is available from https://github.com/danieleb/2014-SJSPS

References

  1. Aronszajn, N. (1950). Theory of reproducing kernels. Transactions of the American Mathematical Society, 68(3), 337–404.

    Article  MATH  MathSciNet  Google Scholar 

  2. Bair, E., Paul, D., Tibshirani, R. (2006). Prediction by supervised principal components. Journal of the American Statistical Association, 101, 119–137.

    Article  MATH  MathSciNet  Google Scholar 

  3. Barchiesi, D., & Plumbley, M.D. (2013). Learning incoherent dictionaries for sparse approximation using iterative projections and rotations. IEEE Transactions on Signal Processing, 61(8), 2055–2065.

    Article  Google Scholar 

  4. Barshan, E., Ghodsi, A., Azimifar, Z., Jahromi, M.Z. (2011). Supervised principal component analysis: Visualization, classification and regression on subspaces and submanifolds. Pattern Recognition, 44(7), 1357–1371.

    Article  MATH  Google Scholar 

  5. Duda, R., & Hart, P.E. (1973). Pattern classification and scene analysis. New York: Wiley.

  6. Elad, M. (2010). Sparse and redundant representations. New York: Springer.

  7. Elhamifar, E., & Vidal, R. (2013). Sparse subspace clustering: algorithm, theory, and applications. To appear in IEEE transactions on pattern analysis and machine intelligence.

  8. Gretton, A., Bousquet, O., Smola, A., Schölkopf, B. (2005). Measuring statistical dependence with hilbert-schmidt norms. Algorithmic learning theory (pp. 63–77). New York: Springer.

  9. Li, K.-C. (1991). Sliced inverse regression for dimension reduction. Journal of the American Statistical Association, 86(414), 316–327.

    Article  MATH  MathSciNet  Google Scholar 

  10. Pearson, K. (1901). On lines and planes of closest fit to systems of points in space. The London, Edinburgh and Dublin Philosophical Magazine and Journal of Science, Sixth Series, 2, 559–572.

    Article  Google Scholar 

  11. Rubinstein, R., Bruckstein, A., Elad, M. (2010). Dictionaries for sparse representation modeling. In Proceedings of the IEEE, 98(6), 1045–1057.

    Article  Google Scholar 

  12. Schnass, K., & Vandergheynst, P. (2010). A union of incoherent spaces model for classification. In Proceedings of the IEEE international conference on acoustics, speech and signal processing (ICASSP) (pp. 5490–5493).

  13. Van Der Maaten, L., Postma, E., Van Den Herik, J. (2009). Dimensionality reduction: A comparative review. Tech. Rep., TiCC, Tilburg University.

  14. Xing, E.P., Jordan, M.I., Russell, S., Ng, A. (2002). Distance metric learning with application to clustering with side-information. In Advances in neural information processing systems (pp. 505–512).

Download references

Author information

Authors and Affiliations

  1. Centre for Digital Music, Queen Mary University of London, E1 4NS, London, UK

    Daniele Barchiesi & Mark D. Plumbley

Authors
  1. Daniele Barchiesi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mark D. Plumbley
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Daniele Barchiesi.

Additional information

This work has been supported by the Platform Grant EP/K009559/1 and the Leadership Fellowship EP/G007144/1, both from the UK Engineering and Physical Sciences Research Council (EPSRC).

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Barchiesi, D., Plumbley, M.D. Learning Incoherent Subspaces: Classification via Incoherent Dictionary Learning. J Sign Process Syst 79, 189–199 (2015). https://doi.org/10.1007/s11265-014-0937-5

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11265-014-0937-5

Keywords

Search

Navigation