Abstract
As a demonstrated and foremost approach of extracting the key features from corrupted observations, tensor robust principal component analysis has been considered in various fields of data processing related to tensors. It is usually modeled as a low-rank and sparse tensor decomposition problem and can be solved by minimizing a simple convex program. However, convex optimization methods often fail to deeply explore the rank and sparsity of tensors, which leads to suboptimality. Based on tensor singular value decomposition, in this work, we introduce generalized nonconvex regularizers accommodating most popular nonconvex (and possibly nonsmooth) surrogate functions to be used as effective approximations of the tensor rank function and \(\ell _{0}\)-norm. The established unified frame equips universality for a large group of nonconvex surrogate functions. Moreover, we consider tube-wise sparse noise in addition to entry-wise sparse noise, which provides a better way of handling structured corrupted observations arising from practical issues. We further develop an efficient algorithm with convergence guarantees to implement generalized nonconvex optimization based upon the alternating direction method of multipliers. The satisfactory performance results of the proposed method are verified by simulations and visual inpainting applications.
Graphic abstract
Similar content being viewed by others
Availability of data and materials
The datasets generated and/or analyzed during the current study are available from the corresponding author on reasonable request.
Notes
Originated from http://www.eecs.qmul.ac.uk/~phao/IP/Images/
RelErrorL is usually much higher than that of the sparse component; thus, we only use the relative error of low-rank component recovery to measure the performance of algorithms from now on.
References
Wang Y, Lin L, Zhao Q, Yue T, Meng D, Leung Y (2017) Compressive sensing of hyperspectral images via joint tensor tucker decomposition and weighted total variation regularization. IEEE Geosci Remote Sens Lett 14(12):2457–2461
Zheng Y, Huang T, Zhao X, Jiang T, Ma T, Ji T (2020) Mixed noise removal in hyperspectral image via low-fibered-rank regularization. IEEE Trans Geosci Remote Sens 58(1):734–749
Tao D, Guo Y, Li Y, Gao X (2018) Tensor rank preserving discriminant analysis for facial recognition. IEEE Trans Image Process 27(1):325–334
Frolov E, Oseledets I (2017) Tensor methods and recommender systems. Wiley Interdiscip Rev Data Min Knowl Disc 7(3):e1201
Zhang Y, Bi X, Tang N, Qu A (2021) Dynamic tensor recommender systems. J Mach Learn Res 22(65):1–35
Lu C, Feng J, Chen Y, Liu W, Lin Z, Yan S (2020) Tensor robust principal component analysis with a new tensor nuclear norm. IEEE Trans Pattern Anal Mach Intell 42(4):925–938
Huang H, Liu Y, Long Z, Zhu C (2020) Robust low-rank tensor ring completion. IEEE Trans Comput Imaging 6:1117–1126
Li B, Zhao X, Wang J, Chen Y, Jiang T, Liu J (2021) Tensor completion via collaborative sparse and low-rank transforms. IEEE Trans Comput Imaging 7:1289–1303
Kilmer M, Horesh L, Avron H, Newman E (2021) Tensor-tensor algebra for optimal representation and compression of multiway data. Proc Natl Acad Sci 118(28)
Zhou P, Feng J (2017) Outlier-robust tensor pca. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp 2263–2271
Chen Y, Wang S, Zhou Y (2018) Tensor nuclear norm-based low-rank approximation with total variation regularization. IEEE J Sel Top Signal Process 12(6):1364–1377
Jiang T, Huang T, Zhao X, Deng L (2020) Multi-dimensional imaging data recovery via minimizing the partial sum of tubal nuclear norm. J Comput Appl Math 372:112680
Zhang F, Wang J, Wang W, Xu C (2021) Low-tubal-rank plus sparse tensor recovery with prior subspace information. IEEE Trans Pattern Anal Mach Intell 43(10):3492–3507
Zhang X, Wang D, Zhou Z, Ma Y (2021) Robust low-rank tensor recovery with rectification and alignment. IEEE Trans Pattern Anal Mach Intell 43(1):238–255
Xie Q, Zhao Q, Meng D, Xu Z (2018) Kronecker-basis-representation based tensor sparsity and its applications to tensor recovery. IEEE Trans Pattern Anal Machine Intell 40(8):1888–1902
Chen Y, Xiao X, Peng C, Lu G, Zhou Y (2022) Low-rank tensor graph learning for multi-view subspace clustering. IEEE Trans Circ Syst Video Technol 32(1):92–104
Li Z, Wang Y, Zhao Q, Zhang S, Meng D (2022) A tensor-based online rpca model for compressive background subtraction. IEEE Trans Neural Netw Learn Syst
Kiers H (2000) Towards a standardized notation and terminology in multiway analysis. J Chemom 14(3):105–122
Tucker L (1966) Some mathematical notes on three-mode factor analysis. Psychometrika 31(3):279–311
Oseledets I (2011) Tensor-train decomposition. SIAM. J Sci Comput 33(5):2295–2317
Kilmer M, Martin C (2011) Factorization strategies for third-order tensors. Linear Algebra Appl 435(3):641–658
Liu J, Musialski P, Wonka P, Ye J (2013) Tensor completion for estimating missing values in visual data. IEEE Trans Pattern Anal Mach Intell 35(1):208–220
Zhang Z, Ely G, Aeron S, Hao N, Kilmer M (2014) Novel methods for multilinear data completion and de-noising based on tensor-svd. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp 3842–3849
Lu C, Feng J, Chen Y, Liu W, Lin Z, Yan S (2016) Tensor robust principal component analysis: Exact recovery of corrupted low-rank tensors via convex optimization. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp 5249–5257
Bengua J, Phien H, Tuan H, Do M (2017) Efficient tensor completion for color image and video recovery: Low-rank tensor train. IEEE Trans Image Process 26(5):2466–2479
Friedland S, Lim LH (2018) Nuclear norm of higher-order tensors. Math Comput 87(311):1255–1281
Yang J, Zhao X, Ji T, Ma T, Huang T (2020) Low-rank tensor train for tensor robust principal component analysis. Appl Math Comput 367:124783
Candès E, Wakin M, Boyd S (2008) Enhancing sparsity by reweighted \(\ell _{1}\) minimization. J Fourier Anal Appl 14(5):877–905
Gao C, Wang N, Yu Q, Zhang Z (2011) A feasible nonconvex relaxation approach to feature selection. In: Proceedings of the AAAI Conference on Artificial Intelligence. pp 356–361
Geman D, Yang C (1995) Nonlinear image recovery with half-quadratic regularization. IEEE Trans Image Process 4(7):932–946
Trzasko J, Manduca A (2008) Highly undersampled magnetic resonance image reconstruction via homotopic \(\ell _{0}\)-minimization. IEEE Trans Med Imaging 28(1):106–121
Friedman J (2012) Fast sparse regression and classification. Int J Forecast 28(3):722–738
Frank L, Friedman J (1993) A statistical view of some chemometrics regression tools. Technometrics 35(2):109–135
Zhang C (2010) Nearly unbiased variable selection under minimax concave penalty. Ann Stat 38(2):894–942
Zhang T (2010) Analysis of multi-stage convex relaxation for sparse regularization. J Mach Learn Res 11(3):1081–1107
Fan J, Li R (2001) Variable selection via nonconcave penalized likelihood and its oracle properties. J Am Stat Assoc 96(456):1348–1360
Xue J, Zhao Y, Liao W, Chan J (2019) Nonconvex tensor rank minimization and its applications to tensor recovery. Inf Sci 503:109–128
Cai S, Luo Q, Yang M, Li W, Xiao M (2019) Tensor robust principal component analysis via non-convex low rank approximation. Appl Sci 9(7):1411
Chen Y, Wang S, Peng C, Hua Z, Zhou Y (2021) Generalized nonconvex low-rank tensor approximation for multi-view subspace clustering. IEEE Trans Image Process 30:4022–4035
Qiu D, Bai M, Ng M, Zhang X (2021) Nonlocal robust tensor recovery with nonconvex regularization. Inverse Probl 37(3):035001
Yang M, Luo Q, Li W, Xiao M (2022) Nonconvex 3d array image data recovery and pattern recognition under tensor framework. Pattern Recogn 122:108311
Lu C, Tang J, Yan S, Lin Z (2015) Nonconvex nonsmooth low rank minimization via iteratively reweighted nuclear norm. IEEE Trans Image Process 25(2):829–839
Boyd S, Parikh N, Chu E, Peleato B, Eckstein J (2011) Distributed optimization and statistical learning via the alternating direction method of multipliers. Found Trends Mach Learn 3(1):1–122
Wang J, Hou J, Eldar Y (2022) Tensor robust principal component analysis from multilevel quantized observations. IEEE Trans Inf Theory 69(1):383–406
Wang A, Jin Z, Tang G (2020) Robust tensor decomposition via t-svd: Near-optimal statistical guarantee and scalable algorithms. Signal Process 167:107319
Lu C, Tang J, Yan S, Lin Z (2014) Generalized nonconvex nonsmooth low-rank minimization. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp 4130–4137
Lu C, Zhu C, Xu C, Yan S, Lin Z (2015) Generalized singular value thresholding. In: Proceedings of the AAAI Conference on Artificial Intelligence. pp 1805–1811
Zhang X (2018) A nonconvex relaxation approach to low-rank tensor completion. IEEE Trans Neural Netw Learn Syst 30(6):1659–1671
Wen F, Ying R, Liu P, Truong T (2019) Nonconvex regularized robust pca using the proximal block coordinate descent algorithm. IEEE Trans Signal Process 67(20):5402–5416
Yang Z, Fan L, Yang Y, Yang Z, Gui G (2020) Generalized nuclear norm and laplacian scale mixture based low-rank and sparse decomposition for video foreground-background separation. Signal Process 172:107527
Liu X, Lu J, Shen L, Xu C, Xu Y (2020) Multiplicative noise removal: nonlocal low-rank model and its proximal alternating reweighted minimization algorithm. SIAM J Imaging Sci 13(3):1595–1629
Wang H, Zhang F, Wang J, Huang T, Huang J, Liu X (2021) Generalized nonconvex approach for low-tubal-rank tensor recovery. IEEE Trans Neural Netw Learn Syst 33(8):3305–3319
Zhang X, Zheng J, Zhao L, Zhou Z, Lin Z (2022) Tensor recovery with weighted tensor average rank. IEEE Trans Neural Netw Learn Syst
Kilmer M, Braman K, Hao N, Hoover R (2013) Third-order tensors as operators on matrices: A theoretical and computational framework with applications in imaging. SIAM J Matrix Anal Appl 34(1):148–172
Donoho D (1995) De-noising by soft-thresholding. IEEE Trans Inf Theory 41(3):613–627
Eckstein J, Bertsekas D (1992) On the douglas-rachford splitting method and the proximal point algorithm for maximal monotone operators. Math Program 55(3):293–318
Huang B, Mu C, Goldfarb D, Wright J (2015) Provable models for robust low-rank tensor completion. Pac J Optim 11(2):339–364
Martin D, Fowlkes C, Tal D, Malik J (2001) A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In: Proceedings Eighth IEEE International Conference on Computer Vision. pp 416–423
Georghiades A, Belhumeur P, Kriegman D (2001) From few to many: Illumination cone models for face recognition under variable lighting and pose. IEEE Trans Pattern Anal Mach Intell 23(6):643–660
Lee K, Ho J, Kriegman D (2005) Acquiring linear subspaces for face recognition under variable lighting. IEEE Trans Pattern Anal Mach Intell 27(5):684–698
Wielgosz M, Pietroń M (2017) Using spatial pooler of hierarchical temporal memory to classify noisy videos with predefined complexity. Neurocomputing 240:84–97
Mousavi S, Ellsworth W, Zhu W, Chuang L, Beroza G (2020) Earthquake transformer-an attentive deep-learning model for simultaneous earthquake detection and phase picking. Nat Commun 11(1):3952
Karimi D, Dou H, Warfield S, Gholipour A (2020) Deep learning with noisy labels: Exploring techniques and remedies in medical image analysis. Med Image Anal 65:101759
Acknowledgements
This work was supported by the National Natural Science Foundation of China (Grant No. 12101512), the China Postdoctoral Science Foundation (Grant No. 2021M692681), the Natural Science Foundation of Chongqing, China (Grant No. cstc2021jcyj-bshX0155) and the Fundamental Research Funds for the Central Universities (Grant No. SWU120078).
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Zhang, F., Wang, H., Qin, W. et al. Generalized nonconvex regularization for tensor RPCA and its applications in visual inpainting. Appl Intell 53, 23124–23146 (2023). https://doi.org/10.1007/s10489-023-04744-9
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10489-023-04744-9