Brief papersMean-square analysis of the gradient projection sparse recovery algorithm based on non-uniform norm
Introduction
Obtaining the sparse recovery solution under the framework of compressed sensing (CS) has gained more and more attention in recent years. The -norm reconstruction algorithms, such as the interior-point method for large-scale -regularized least squares (l1-ls) method [2] and gradient projection for sparse reconstruction [3], are usually adopted by searching for a solution with minimum -norm. In comparison, because direct search of minimum -norm will generally lead to NP hard problem [4], [5], [6], [7], various approximation methods are also investigated to solve the difficulties caused by -norm [4], [5], [6] such as the smoothing –norm (SL0) [8] and –norm zero-point attracting projection (–ZAP) [9]. Another type of popular approaches is greedy method such as matching pursuit (MP) and orthogonal matching pursuit (OMP) [4], [5], by which the approximation is generated via an iteratively process to search the column vectors that most closely resemble the required. Based on block based least square and minimum mean-square-error cost function, iteratively reweighted least squares (IRLS) minimization technique is used for iterative sparse recovery [6]. Moreover, in [10] the Frobenius norm and the -norm of the Euclidean norm are used to design an iterative optimization algorithm for the structured sparse coding model.
In [11], [12], a new non-uniform norm called -norm, which consisted of a sequence of or norm elements according to relative magnitude, is proposed to exploit the sparseness while providing adaptability to different sparsity of sources. It is pointed out in [11] that imposing -norm constraint on the Least Mean Square (LMS) iteration yields enhanced convergence rate as well as better tolerance upon different sparsity in system identification. The sparsity exploitation performance of the -norm LMS algorithm has been compared with that of other existing sparsity-aware algorithms in [14], [15]. In [16], the concept of non-uniform norm is further combined with variable step-size to derive the p norm variable step-size LMS algorithm, which claims to outperform the classic -norm LMS.
The concept of -norm can be also introduced into the sparse signal reconstruction at the presence of source signals associated with different sparsity. Similar to previous gradient projection type approaches [3], the iterative optimization solution of the proposed -norm sparse reconstruction can be derived by directly minimizing the -norm cost function via the steepest descent method, and then affine projecting the solution to the feasible set. However, there is a lack of analytical steady state performance in terms of -norm sparsity recovery. In this letter, the steady state mean-square performance of the -norm gradient projection sparse recovery algorithm is theoretically performed. Finally numerical simulation results are provided to verify the analysis.
Section snippets
Derivation of the non-uniform norm CS Algorithm
The problem to obtain -norm or -norm sparse solution can be respectively expressed as:where and .
In [11], [12], the concept of p-norm like is introduced and defined as:
Furthermore, a non-uniform norm, denoted as in this study, is defined in [12]. It is noticeable that utilizes different value of p for each entry of , i.e. . Moreover, by classifying
Steady state performance analysis of the norm CS algorithm
Without loss of generality, in this letter the entries of mixing matrix are independently sampled from a normal distribution with mean zero and variance of , which ensure each column vector of are normalized.
We define the misalignment vector as , where is the optimum gradient descent solution, and the actual possible sampling result related to the optimum gradient descent solution can be rewritten as:where can be regarded as deviation noise signal between the
Numerical simulation
In this section, numerical simulations are performed to verify the theoretical analysis of mean-square performance. The MSD of the sparse reconstruction is defined as:where is ideal sparse signal size of . In order to formulate the CS problem, the matrix and vectors are generated according to (10), and the nonzero value of is generated according to normal distribution process as following:where and are the variance of the
Conclusion
Extending from the application of the non-uniform norm for sparsity exploitation in the framework of LMS algorithm, a novel -norm sparse recovery algorithm is derived in this letter, by projecting the gradient descent norm solution to the reconstruction feasible set. Meanwhile, the steady state MSD performance of the proposed norm compressed sensing signal recovery algorithm is theoretically investigated in terms of different sparsity as well as different additive noise. The performance
Acknowledgment
The authors are grateful for the funding by Grants from the National Natural Science Foundation of China (Project no. 11274259 and Project no. 11574258) in support of the present research. The authors also would like to thank the anonymous reviewers for improving the quality of this letter.
Feiyun Wu received his B.S., M.S and Ph.D degrees from Nanchang University, Nanchang, Sun Yat-Sen University, Guangzhou and Xiamen University, Xiamen, China, 2006, 2010, 2016, respectively. From 2006-2008, he worked as a college lecturer in Electronic Engineering Department of Nanchang University Gongqing College. During 2013-2015, he was a visiting Ph.D student in University of Delaware, DE, US, Now he is an assistant professor in Northwestern Polytechnical University, Xi'an, China. His
References (16)
- et al.
performance analysis of minimum l1-norm solutions for underdetermined source separation
IEEE Trans. Signal Process.
(2004) - et al.
An interior point method for large-scale l1-regularized least squares
IEEE J. Sel. Top. Signal Process.
(2007) - et al.
Gradient projection for sparse reconstruction: application to compressed sensing and other inverse problems
IEEE J. Sel. Top. Signal Process.
(2007) Greed is good: algorithmic results for sparse approximation
IEEE Trans. Inform. Theory
(2004)- et al.
Signal recovery from random measurements via orthogonal matching pursuit
IEEE Trans. Inform. Theory
(2007) - et al.
Iteratively reweighted least squares minimization for sparse recovery
Commun. Pure Appl. Maths.
(2010) - et al.
l0 norm constraint LMS algorithm for sparse system identification
IEEE Signal Process. Lett.
(2009) - et al.
A fast approach for overcomplete sparse decomposition based on smoothed l0 Norm
IEEE Trans. Signal Process.
(2009)
Cited by (15)
A fast threshold OMP based on self-learning dictionary for propeller signal reconstruction
2023, Ocean EngineeringA neighborhood-based multiple orthogonal least square method for sparse signal recovery
2023, Signal ProcessingAn effective framework for underwater acoustic data acquisition
2021, Applied AcousticsOptimized compression and recovery of electrocardiographic signal for IoT platform
2020, Applied Soft Computing JournalAn Automatic Threshold OMP Algorithm Based on QR Decomposition for Magnetic Resonance Image Reconstruction
2024, Circuits, Systems, and Signal Processing
Feiyun Wu received his B.S., M.S and Ph.D degrees from Nanchang University, Nanchang, Sun Yat-Sen University, Guangzhou and Xiamen University, Xiamen, China, 2006, 2010, 2016, respectively. From 2006-2008, he worked as a college lecturer in Electronic Engineering Department of Nanchang University Gongqing College. During 2013-2015, he was a visiting Ph.D student in University of Delaware, DE, US, Now he is an assistant professor in Northwestern Polytechnical University, Xi'an, China. His research interests include adaptive signal processing for underwater acoustics communications, compressed sensing, ICA and wavelet algorithms and their applications.
Feng Tong received his PhD in underwater acoustics at Xiamen University, China in 2000. From 2000–2002, he worked as a post-doctoral fellow in the Department of Radio Engineering, Southeast University, China. Since 2003, he has been a research associate at the Department of Manufacturing Engineering and Engineering Management, City University of Hong Kong for one and a half year. From Dec 2009 to Dec 2010 He was a visiting scholar in Department of Computer Science and Engineering, University of California San Diego, USA. Currently he is a professor with the Department of Applied Marine Physics and Engineering, Xiamen University, China. His research interests focus on underwater acoustic communication and acoustic signal processing. He is a member of IEEE, ASC (Acoustical Society of China) and CSIS (China Ship Instrument Society). He serves on the editorial board for the Journal of Marine Science and Application.