Abstract
Tensor train model is a low-rank approximation for multidimensional data. In this article we demonstrate how it can be succesfully used for fast computation of nonnegative tensor train, nonnegative canonical and nonnegative Tucker factorizations. The proposed approaches can be incorporated in wide range of methods to solve big data problems.
Similar content being viewed by others
REFERENCES
F. L. Hitchcock, ‘‘Multiple invariants and generalized rank of a p-way matrix or tensor,’’ J. Math. Phys. 7, 39–79 (1927).
R. A. Harshman, ‘‘Determination and proof of minimum uniqueness conditions for PARAFAC1,’’ in UCLA Working Papers in Phonetics (1972), Vol. 22.
J. D. Carroll and J. J. Chang, ‘‘Analysis of individual differences in multidimensional scaling via an n-way generalization of Eckart–Young decomposition,’’ Psychometrika 35, 283–319 (1970).
J. B. Kruskal, ‘‘Three-way arrays: Rank and uniqueness of trilinear decompositions, with application to arithmetic complexity and statistics,’’ Linear Algebra Appl. 18, 95–138 (1977).
J. D. Carroll, G. de Soete, and S. Pruzansky, ‘‘Fitting of the latent class model via iteratively reweighted least squares CANDECOMP with nonnegativity constraints,’’ in Multiway Data Analysis (Elsevier, Amsterdam, The Netherlands, 1989), pp. 463–472.
A.-H. Phan, A. Cichocki, I. Oseledets, G. G. Calvi, S. Ahmadi-Asl, and D. P. Mandic, ‘‘Tensor networks for latent variable analysis: Higher order canonical polyadic decomposition,’’ IEEE Trans. Neural Networks Learn. Syst. 31, 2174–2188 (2020). https://doi.org/10.1109/TNNLS.2019.2929063
O. Lebedeva, ‘‘Tensor conjugate-gradient-type method for Rayleigh quotient minimization in block qtt-format,’’ Russ. J. Numer. Anal. Math. Model. 26, 465–489 (2011).
I. Oseledets, ‘‘Tensor-train decomposition,’’ SIAM J. Sci. Comput. 33, 2295–2317 (2011).
S. Kuksova, E. W. Skau, and B. S. Alexandrov, ‘‘Error analysis of nonnegative tensor train utilized for nonnegative canonical polyadic decomposition,’’ in Proceedings of the 37th International Conference on Machine Learning ICML, 2020.
A. Cichocki and Phan Anh-Huy, ‘‘Fast local algorithms for large scale nonnegative matrix and tensor factorizations,’’ in IEICE Transactions on the Fundamentals of Electronics, Communication and Computer Science, 2009.
N. Lee, A. Phan, F. Cong, and A. Cichocki, ‘‘Nonnegative tensor train decompositions for multi-domain feature extraction and clustering,’’ Lect. Notes Comput. Sci. 9949, 87–95 (2016).
I. Oseledets et al., TT-Toolbox (TT=Tensor Train), Version 2.2.2 (Inst. Numer. Math., Moscow, Russia, 2009–2013). https://github.com/oseledets/TT-Toolbox.
I. Oseledets and E. Tyrtyshnikov, ‘‘TT-cross approximation for multidimensional arrays,’’ Linear Algebra Appl. 432, 70–88 (2010).
Anh-Huy Phan, P. Tichavsky, and A. Cichocki, TENSORBOX: A MATLAB Package for Tensor Decomposition. https://github.com/phananhhuy/TensorBox. Accessed 2019.
Y. Xu, ‘‘Alternating proximal gradient method for sparse nonnegative Tucker decomposition,’’ Math. Program. Comput. 7, 39–70 (2015).
E. Shcherbakova, ‘‘Nonnegative tensor train factorization with DMRG technique,’’ Lobachevskii J. Math. 40, 1863–1872 (2019).
E. Tyrtyshnikov and E. Shcherbakova, ‘‘Nonnegative matrix factorization methods based on low-rank cross approximations,’’ Comput. Math. Math. Phys. 59 (8) (2019).
E. Shcherbakova and E. Tyrtyshnikov, ‘‘Nonnegative tensor train factorizations and some applications,’’ in Large-Scale Scientific Computing (Springer Int., Cham, 2020), pp. 156–164.
G. Zhou, A. Cichocki, Q. Zhao, and S. Xie, ‘‘Efficient nonnegative Tucker decompositions: Algorithms and uniqueness,’’ IEEE Trans. Image Process. 24, 4990–5003 (2015).
F. Cong, Q.-H. Lin, L.-D. Kuang, X.-F. Gong, P. Astikainen, and T. Ristaniemi, ‘‘Tensor decomposition of EEG signals: A brief review,’’ J. Neurosci. Methods 248, 59–69 (2015).
F. Cong, A. Phan, Q. Zhao, T. Huttunen-Scott, J. Kaartinen, T. Ristaniemi, H. Lyytinen, and A. Cichocki, ‘‘Benefits of multi-domain feature of mismatch negativity extracted by nonnegative tensor factorization from low-density array EEG,’’ Int. J. Neural Syst. 22 (6) (2012).
L. R. Tucker, ‘‘The extension of factor analysis to three-dimensional matrices,’’ in Contributions to Mathematical Psychology, Ed. by H. Gulliksen and N. Frederiksen (Holt, Rinehart and Winston, New York, 1964), pp. 110–127.
Y. D. Kim and S. Choi, ‘‘Nonnegative Tucker Decomposition,’’ in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR07) (Minneapolis, 2007), pp. 1–8.
A. Marmoret, J. E. Cohen, N. Bertin, and F. Bimbot, ‘‘Uncovering audio patterns in music with Nonnegative Tucker Decomposition for structural segmentation,’’ arXiv (2021), arXiv:2104.08580.
Y. Qiu, G. Zhou, Y. Wang, Y. Zhang, and S. Xie, ‘‘A Generalized Graph Regularized Non-Negative Tucker Decomposition Framework for Tensor Data Representation,’’ IEEE Transactions on Cybernetics 52 (1), 594–607 (2022).
Brett W. Bader, Tamara G. Kolda, et al., Tensor Toolbox for MATLAB, Version 3.2.1 (2021). https://tensortoolbox.org.
Funding
The work was supported by the Moscow Center of Fundamental and Applied Mathematics (agreement 075-15-2019-1624 with Ministery of education and science of Russian Federation).
Author information
Authors and Affiliations
Corresponding authors
Additional information
(Submitted by V. V. Voevodin)
Rights and permissions
About this article
Cite this article
Shcherbakova, E.M., Tyrtyshnikov, E.E. Fast Nonnegative Tensor Factorizations with Tensor Train Model. Lobachevskii J Math 43, 882–894 (2022). https://doi.org/10.1134/S1995080222070228
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1134/S1995080222070228