A Gradient Projection Algorithm with a New Stepsize for Nonnegative Sparsity-Constrained Optimization

where f: R⟶ R is a continuously differential function with a lower bound. S: � x ∈ R: ‖x‖0 ≤ s 􏼈 􏼉 is a sparse set, where s< n is a given integer regulating the sparsity level in x and Rn+ is the nonnegative orthant in R. ‖x‖0 is the l0 norm of x, counting the number of nonzero elements in x. Many application problems can be translated into problem (1), such as the widely studied linear compressing sensing problem of f(x) � (1/2)‖Ax − b‖2 with A ∈ R being a sensing matrix, b ∈ R is the observation vector, and ‖ · ‖ is the Euclidean norm in R [1]. Problem (1) has also used to the regularized logistic regression cost function [2]. Recently, a great deal of work has been devoted to algorithms for sparsity-constrained optimization problem. Beck and Eldar [3] established the IHT algorithm which converges to L-stationary under the Lipchitz continuity of the gradient of objective function. Beck and Hallak [4] generalized these results to sparse symmetric sets. Lu [5] designed a nonmonotone algorithm for symmetric set constraint problems. Pan, Xiu, and Zhou [6, 7] established the B-stationary, C-stationary, and α-stationary based on the Bouligand tangent cone and Clarke tangent. Recently, Pan, Zhou, and Xiu [8] established the improved IHT algorithm (IIHT) for problem (1) by using Armijo line search. )ey proved that any accumulation point converged to an α-stationary point under the restricted strong smoothness of objective function which is weaker than the Lipchitz continuity of the gradient. Inspired by the above literature studies, in this paper, we establish a gradient projection algorithmwith a new stepsize. )e new algorithm removes the condition of the restricted strong smoothness of objective function which makes it more applicable. Meanwhile, we prove the convergence of the algorithm. )e rest of this paper is organized as follows. In Section 2, we present some notations, definitions, and lemmas. In Section 3, we give the algorithm of (1) and prove the convergence properties.


Introduction
In this paper, we are mainly concerned with the nonnegative sparsity-constrained optimization problem (NN-SCO): where f: R n ⟶ R is a continuously differential function with a lower bound. S: � x ∈ R n : ‖x‖ 0 ≤ s is a sparse set, where s < n is a given integer regulating the sparsity level in x and R n + is the nonnegative orthant in R n . ‖x‖ 0 is the l 0 norm of x, counting the number of nonzero elements in x. Many application problems can be translated into problem (1), such as the widely studied linear compressing sensing problem of f(x) � (1/2)‖Ax − b‖ 2 with A ∈ R m×n being a sensing matrix, b ∈ R m is the observation vector, and ‖ · ‖ is the Euclidean norm in R n [1]. Problem (1) has also used to the regularized logistic regression cost function [2].
Recently, a great deal of work has been devoted to algorithms for sparsity-constrained optimization problem. Beck and Eldar [3] established the IHT algorithm which converges to L-stationary under the Lipchitz continuity of the gradient of objective function. Beck and Hallak [4] generalized these results to sparse symmetric sets. Lu [5] designed a nonmonotone algorithm for symmetric set constraint problems. Pan, Xiu, and Zhou [6,7] established the B-stationary, C-stationary, and α-stationary based on the Bouligand tangent cone and Clarke tangent. Recently, Pan, Zhou, and Xiu [8] established the improved IHT algorithm (IIHT) for problem (1) by using Armijo line search. ey proved that any accumulation point converged to an α-stationary point under the restricted strong smoothness of objective function which is weaker than the Lipchitz continuity of the gradient. Inspired by the above literature studies, in this paper, we establish a gradient projection algorithm with a new stepsize. e new algorithm removes the condition of the restricted strong smoothness of objective function which makes it more applicable. Meanwhile, we prove the convergence of the algorithm. e rest of this paper is organized as follows. In Section 2, we present some notations, definitions, and lemmas. In Section 3, we give the algorithm of (1) and prove the convergence properties.

Definitions
Definition 1 (see [8]). Let x * ∈ S ∩ R n + be a given feasible point of (1). We say that x * is an α-stationary point, if there exists α > 0 such that Definition 2 (see [9]). A function f is called 2s-restricted strongly smooth (2s-RSS) with parameter L 2s > 0, and if for any x, y ∈ R n satisfying |I 1 (xy)| < 2s, it holds that Definition 3 (see [9]). A function f is called 2s-restricted strongly convex (2s-RSC) with parameter l 2s > 0, and if for any x, y ∈ R n satisfying |I 1 (xy)| < 2s, it holds that If and only if for any x, y ∈ R n and |I 1 (xy)| ≤ 2s, we have In particular, in (5), if l 2s � 0, the function f is called 2srestricted convex (2s-RC).
Lemma 3 (see [8]). For any where e i ∈ R n is a vector whose i th component is one and others are zeros.

Main Results
In this section, we establish a new algorithm which improves the IIHT algorithm for (1) and then we analyze its convergence properties. At first, let us develop the gradient projection algorithm with a new stepsize rule.
Next, let us list the following assumptions for convenience: Let the sequence x k be generated by Algorithm 1, and set l k � 3L k . en, we have where Proof. Let en, 2 Mathematical Problems in Engineering en, (11) is tenable. □ Lemma 5. We suppose l ≥ l k . For x k ∈ S ∩ R n + and x k+1 � P S∩R n where σ � (l − l k /2).
Proof. Since x k+1 ∈ P S∩R n + (x k − (1/l)∇f(x k )), by the definition of projection, we get Moreover,
(2) We can easily get that f(x k ) is an increasing sequence by (15). Moreover, by the assumptions (H 2 ), we can get that f(x k ) converges. (3) Let μ � ((1/α k ) − l k /2) in (1). We can get Summing over both sides of this inequality, we get (25) Since f is bounded below, we get (4) It easily can be got by (2). □ Lemma 7. Let the sequence x k be generated by Algorithm 1. Suppose that the function f is 2s-RC. We have Proof. Because the sequence x k be generated by Algorithm 1, we get |I 1 (xy)| < 2s. By Lemma 4 and Lemma 5 in reference [8], we can get □ Theorem 1. Let the sequence x k be generated by Algorithm 1. en, the following results hold: (1) Any accumulation of sequence x k is an α-stationary point.

Proof
(1) Suppose that x * is an accumulation point of sequence x k . en, there exists a subsequence x k n converges to x * . Because we get Moreover, x k n +1 � P S∩R n + R n x k − α k n ∇f x k n , � P S P R n + x k − α k n ∇f x k n . (32) We consider the next two cases: Mathematical Problems in Engineering Case 1. For i ∈ I 1 (x * ), there must exist a sufficiently large index N and a constant c 0 > 0 such that By P S∩R n + � P S (P R n + ) and (33), we can get x Since lim n⟶+∞ inf α k n > 0, without loss of generality, we can suppose lim k⟶+∞ α k n > c. Let n ⟶ + ∞. We get i.e., Case 2. For i ∈ I 0 (x * ), we consider two subcases.
Due to the property of the projections P S and P R n us, Taking limits on both sides, we obtain For all sufficiently large n, we have Since ‖x * ‖ 0 < s, for all sufficiently large n, we have which contradicts with i ∈ I 0 (x * ). us, ∇ i f(x * ) ≥ 0. Summarizing the two cases, we obtain us, x * is an α-stationary point of (1).
By Definition 4, we have Moreover, the maximum value is taken at ‖v‖ � 1. For any ε > 0, there exists v k ∈ R n Γ k and ‖v k ‖ � 1 satisfies Because x k+1 � P S∩R n + (x k − α k ∇f(x k )) and x Γ k+1 � y Γ k+1 , x ∈ P S∩R n + (y), we get i.e., us, for any ϖ k+1 ∈ R n Γ k+1 , we get Taking ϖ k+1 � x k+1 + v k+1 , we get By the Cauchy-Schwartz inequality, we get i.e., By Lemma 7, we get Mathematical Problems in Engineering Taking limits on both sides and using Lemma 6, we have By (32), we get Theorem 2. Let the sequence x k be generated by Algorithm 1. x * is an accumulation point of the sequence x k . Suppose f(x) is 2s − RC, then the following results hold: (1) If ‖x * ‖ 0 < s, then x * is a global minimizer of (1) (2) If ‖x * ‖ 0 � s, then x * is a local minimizer of (1)