Skip to main content
Log in

Fast Nonnegative Tensor Factorizations with Tensor Train Model

Lobachevskii Journal of Mathematics Aims and scope Submit manuscript

Abstract

Tensor train model is a low-rank approximation for multidimensional data. In this article we demonstrate how it can be succesfully used for fast computation of nonnegative tensor train, nonnegative canonical and nonnegative Tucker factorizations. The proposed approaches can be incorporated in wide range of methods to solve big data problems.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

REFERENCES

  1. F. L. Hitchcock, ‘‘Multiple invariants and generalized rank of a p-way matrix or tensor,’’ J. Math. Phys. 7, 39–79 (1927).

    Article  Google Scholar 

  2. R. A. Harshman, ‘‘Determination and proof of minimum uniqueness conditions for PARAFAC1,’’ in UCLA Working Papers in Phonetics (1972), Vol. 22.

  3. J. D. Carroll and J. J. Chang, ‘‘Analysis of individual differences in multidimensional scaling via an n-way generalization of Eckart–Young decomposition,’’ Psychometrika 35, 283–319 (1970).

    Article  Google Scholar 

  4. J. B. Kruskal, ‘‘Three-way arrays: Rank and uniqueness of trilinear decompositions, with application to arithmetic complexity and statistics,’’ Linear Algebra Appl. 18, 95–138 (1977).

    Article  MathSciNet  Google Scholar 

  5. J. D. Carroll, G. de Soete, and S. Pruzansky, ‘‘Fitting of the latent class model via iteratively reweighted least squares CANDECOMP with nonnegativity constraints,’’ in Multiway Data Analysis (Elsevier, Amsterdam, The Netherlands, 1989), pp. 463–472.

    Google Scholar 

  6. A.-H. Phan, A. Cichocki, I. Oseledets, G. G. Calvi, S. Ahmadi-Asl, and D. P. Mandic, ‘‘Tensor networks for latent variable analysis: Higher order canonical polyadic decomposition,’’ IEEE Trans. Neural Networks Learn. Syst. 31, 2174–2188 (2020). https://doi.org/10.1109/TNNLS.2019.2929063

    Article  MathSciNet  Google Scholar 

  7. O. Lebedeva, ‘‘Tensor conjugate-gradient-type method for Rayleigh quotient minimization in block qtt-format,’’ Russ. J. Numer. Anal. Math. Model. 26, 465–489 (2011).

    Article  MathSciNet  Google Scholar 

  8. I. Oseledets, ‘‘Tensor-train decomposition,’’ SIAM J. Sci. Comput. 33, 2295–2317 (2011).

    Article  MathSciNet  Google Scholar 

  9. S. Kuksova, E. W. Skau, and B. S. Alexandrov, ‘‘Error analysis of nonnegative tensor train utilized for nonnegative canonical polyadic decomposition,’’ in Proceedings of the 37th International Conference on Machine Learning ICML, 2020.

  10. A. Cichocki and Phan Anh-Huy, ‘‘Fast local algorithms for large scale nonnegative matrix and tensor factorizations,’’ in IEICE Transactions on the Fundamentals of Electronics, Communication and Computer Science, 2009.

  11. N. Lee, A. Phan, F. Cong, and A. Cichocki, ‘‘Nonnegative tensor train decompositions for multi-domain feature extraction and clustering,’’ Lect. Notes Comput. Sci. 9949, 87–95 (2016).

    Article  Google Scholar 

  12. I. Oseledets et al., TT-Toolbox (TT=Tensor Train), Version 2.2.2 (Inst. Numer. Math., Moscow, Russia, 2009–2013). https://github.com/oseledets/TT-Toolbox.

  13. I. Oseledets and E. Tyrtyshnikov, ‘‘TT-cross approximation for multidimensional arrays,’’ Linear Algebra Appl. 432, 70–88 (2010).

    Article  MathSciNet  Google Scholar 

  14. Anh-Huy Phan, P. Tichavsky, and A. Cichocki, TENSORBOX: A MATLAB Package for Tensor Decomposition. https://github.com/phananhhuy/TensorBox. Accessed 2019.

  15. Y. Xu, ‘‘Alternating proximal gradient method for sparse nonnegative Tucker decomposition,’’ Math. Program. Comput. 7, 39–70 (2015).

    Article  MathSciNet  Google Scholar 

  16. E. Shcherbakova, ‘‘Nonnegative tensor train factorization with DMRG technique,’’ Lobachevskii J. Math. 40, 1863–1872 (2019).

    Article  MathSciNet  Google Scholar 

  17. E. Tyrtyshnikov and E. Shcherbakova, ‘‘Nonnegative matrix factorization methods based on low-rank cross approximations,’’ Comput. Math. Math. Phys. 59 (8) (2019).

  18. E. Shcherbakova and E. Tyrtyshnikov, ‘‘Nonnegative tensor train factorizations and some applications,’’ in Large-Scale Scientific Computing (Springer Int., Cham, 2020), pp. 156–164.

    Google Scholar 

  19. G. Zhou, A. Cichocki, Q. Zhao, and S. Xie, ‘‘Efficient nonnegative Tucker decompositions: Algorithms and uniqueness,’’ IEEE Trans. Image Process. 24, 4990–5003 (2015).

    Article  MathSciNet  Google Scholar 

  20. F. Cong, Q.-H. Lin, L.-D. Kuang, X.-F. Gong, P. Astikainen, and T. Ristaniemi, ‘‘Tensor decomposition of EEG signals: A brief review,’’ J. Neurosci. Methods 248, 59–69 (2015).

    Article  Google Scholar 

  21. F. Cong, A. Phan, Q. Zhao, T. Huttunen-Scott, J. Kaartinen, T. Ristaniemi, H. Lyytinen, and A. Cichocki, ‘‘Benefits of multi-domain feature of mismatch negativity extracted by nonnegative tensor factorization from low-density array EEG,’’ Int. J. Neural Syst. 22 (6) (2012).

  22. L. R. Tucker, ‘‘The extension of factor analysis to three-dimensional matrices,’’ in Contributions to Mathematical Psychology, Ed. by H. Gulliksen and N. Frederiksen (Holt, Rinehart and Winston, New York, 1964), pp. 110–127.

    Google Scholar 

  23. Y. D. Kim and S. Choi, ‘‘Nonnegative Tucker Decomposition,’’ in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR07) (Minneapolis, 2007), pp. 1–8.

  24. A. Marmoret, J. E. Cohen, N. Bertin, and F. Bimbot, ‘‘Uncovering audio patterns in music with Nonnegative Tucker Decomposition for structural segmentation,’’ arXiv (2021), arXiv:2104.08580.

  25. Y. Qiu, G. Zhou, Y. Wang, Y. Zhang, and S. Xie, ‘‘A Generalized Graph Regularized Non-Negative Tucker Decomposition Framework for Tensor Data Representation,’’ IEEE Transactions on Cybernetics 52 (1), 594–607 (2022).

    Article  Google Scholar 

  26. Brett W. Bader, Tamara G. Kolda, et al., Tensor Toolbox for MATLAB, Version 3.2.1 (2021). https://tensortoolbox.org.

Download references

Funding

The work was supported by the Moscow Center of Fundamental and Applied Mathematics (agreement 075-15-2019-1624 with Ministery of education and science of Russian Federation).

Author information

Authors and Affiliations

Authors

Corresponding authors

Correspondence to E. M. Shcherbakova or E. E. Tyrtyshnikov.

Additional information

(Submitted by V. V. Voevodin)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Shcherbakova, E.M., Tyrtyshnikov, E.E. Fast Nonnegative Tensor Factorizations with Tensor Train Model. Lobachevskii J Math 43, 882–894 (2022). https://doi.org/10.1134/S1995080222070228

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1134/S1995080222070228

Keywords:

Navigation