A Modified Self-Adaptive Conjugate Gradient Method for Solving Convex Constrained Monotone Nonlinear Equations for Signal Recovery Problems

In this article, we propose a modified self-adaptive conjugate gradient algorithm for handling nonlinear monotone equations with the constraints being convex. Under some nice conditions, the global convergence of the method was established. Numerical examples reported show that the method is promising and efficient for solving monotone nonlinear equations. In addition, we applied the proposed algorithm to solve sparse signal reconstruction problems.


Introduction
Suppose Ω is a nonempty, closed and convex subset of R n , F a continuous function from R n to R n . A constrained nonlinear monotone equation involves finding a point x ∈ Ω, such that Many algorithms have been proposed in the literature to solve nonlinear constrained equations, some of which are the trust region and the Levenberg-Marquardt method [1]. However, the need for these methods to compute and store matrix in every iteration, make them unsuitable for solving large-scale nonlinear equations.
All through this article, we assume the following (A 1 ) The solution set of (1), denoted by Ω , is nonempty. (A 2 ) The mapping F is monotone, that is, (A 3 ) The mapping F(.) is Lipschitz continuous, that is there exists a positive constant L such that F(x) − F(y) ≤ L x − y , ∀x, y ∈ R n .
Step 1. If F(x k ) ≤ Tol, stop, otherwise go to Step 2.
Step 2. Compute where Step 3. Compute the step length α k = βρ m k and m k is the smallest non-negative integer m such that − F(x k + βρ m d k ), d k ≥ σβρ m d k 2 .
Step 4. Set z k = x k + α k d k and compute Step 5. Let k = k + 1 and go to Step 1.
It can be observed that the modification made is by replacing β HS k , θ k in [20] with β k , θ k respectively in the proposed algorithm.

Remark 1.
Using Cauchy-Schwartz inequality, we get Remark 2. From the definition of w k−1 , t k−1 and (8), we have The above inequality shows that the denominator of β k and θ k cannot be zero except at the solution. However, there is no any guarantee that the denominator of β HS k and θ k defined in Reference [20] cannot be zero. In addition, the conditions imposed in step 2 of Algorithm 2.1 of [20] were not considered in our case.

Convergence Analysis
To prove the global convergence of Algorithm 1, the following Lemmas are needed. The following Lemma shows that Algorithm 1 is well-defined.
Proof. Suppose there exists k 0 ≥ 0 such that (6) does not hold for any non-negative integer i, that is, Using assumption (A 3 ) and allowing i → ∞, we get Also from (7), we have − F(x k 0 ), d k 0 = F(x k 0 ) 2 > 0, which contradicts (10). The proof is complete.
Lemma 3. Suppose that (A 3 ) hold and the sequences {x k } and {z k } be generated by Algorithm 1. Then we have Proof. Suppose α k = β, then α k ρ does not satisfy Equation (6), that is This combined with (7) and the fact that F is Lipschitz continuous yields The above equation implies which completes the proof.
and lim Proof. We will start by showing that the sequences {x k } and {z k } are bounded. Supposex ∈ Ω , then by monotonicity of F, we get Also by definition of z k and the line search (6), we have So, we have Thus the sequence { x k −x } is non increasing and convergent and hence {x k } is bounded. Furthermore, from Equation (16), we have and we can deduce recursively that Then from Assumption (A 3 ), we obtain By the definition of z k , Equation (15), monotonicity of F and the Cauchy-Schwatz inequality, we get The boundedness of the sequence {x k } together with Equations (18) and (19), implies that the sequence {z k } is bounded. Since {z k } is bounded, then for anyx ∈ Ω , the sequence {z k −x} is also bounded, that is, there exists a positive constant ν > 0 such that This together with Assumption (A 3 ) yields Therefore, using Equation (16), we have Equation (20) implies lim However, using statement 2 of Lemma 1, the definition of ζ k and the Cauchy-Schwatz inequality, we have which yields lim k→∞ x k+1 − x k = 0. (12) and definition of z k , we have

Numerical Examples
This section reports some numerical results to show the efficiency of Algorithm 1. For convenience sake, we denote Algorithm 1 by the modified self-adaptive method MSCG. We also divide this section into two. First we compare the MSCG method with the projected conjugate gradient PCG and the self-adaptive three-term conjugate gradient SATCGM methods in References [19,20] respectively, by solving some monotone nonlinear equations with convex constraints using different initial points and several dimensions. Secondly, the MSCG method is applied to solve signal recovery problems. All codes were written in MATLAB R2017a and run on a PC with intel COREi5 processor with 4 GB of RAM and CPU 2.3 GHZ.

Numerical Examples on Some Convex Constrained Nonlinear Monotone Equations
Same line search implementation was used for both MSCG, PCG and SATCGM. The specific parameters used for each method are as follows: All parameters are chosen as in [19]. SATCGM method: All parameters are chosen as in [20]. All runs were stopped whenever F(x k ) < 10 −6 .

Problem 6
Linear monotone problem and Ω = R n + .
The numerical results indicate that the MSCG method is more effective than the PCG and SATCGM methods for the given problems as it solves and win 6 out of 9 of the problems tested both in terms of number of iterations, number of function evaluations and CPU time (see Tables 1-9). In particular, the PCG method fails to solve problems 4 completely while MSCG was able to solve all the problems except for the initial points x 6 and x 7 (see Table 4). In addition, the SATCGM method fails to solve problem 4 except in the case of dimension 1000 and initial point x 1 for the remaining dimensions considered. Therefore, we can conclude that the MSCG method is a very efficient tool for solving nonlinear monotone equations with convex constraints, especially for large-scale dimensions. 10,000 ,000

Experiments on Solving Some Signal Recovery Problems in Compressive Sensing
There are many problems in signal processing and statistical inference involving finding sparse solutions to ill-conditioned linear systems of equations. Among popular approaches is minimizing an objective function which contains quadratic ( 2 ) error term and a sparse 1 −regularization term, that is, where x ∈ R n , y ∈ R k is an observation, A ∈ R k×n (k << n) is a linear operator, τ is a nonnegative parameter, x 2 denotes the Euclidean norm of x and It is easy to see that problem (30) is a convex unconstrained minimization problem. Due to the fact that if the original signal is sparse or approximately sparse in some orthogonal basis, problem (30) frequently appears in compressive sensing, and hence an exact restoration can be produced by solving (30).
Iterative methods for solving (30) have been presented in the literature, (see References [24][25][26][27][28][29]). The most popular method among these methods is the gradient based method and the earliest gradient projection method for sparse reconstruction (GPRS) was proposed by Figueiredo et al. [27]. The first step of the GPRS method is to express (30) as a quadratic problem using the following process. Let x ∈ R n and splitting it into its positive and negative parts. Then x can be formulated as .., n, and (.) + = max{0, .}. By definition of 1 -norm, we have x 1 = e T n u + e T n v, where e n = (1, 1, ..., 1) T ∈ R n . Now (30) can be written as which is a bound-constrained quadratic program. However, from Reference [27], Equation (31) can be written in standard form as Clearly, D is a positive semi-definite matrix, which implies that Equation (32) is a convex quadratic problem. Xiao et al. [17] translated (32) into a linear variable inequality problem which is equivalent to a linear complementarity problem. Furthermore, they pointed out that z is a solution of the linear complementarity problem if and only if it is a solution of the nonlinear equation: It was proved in Reference [30,31] that F(z) is continuous and monotone. Therefore problem (30) can be translated into problem (1) and thus MSCG method can be applied to solve (30).
In this experiment, we consider a simple compressive sensing possible situation, where our goal is to reconstruct a sparse signal of length n from k observations. The quality of restoration is assessed by mean of squared error (MSE) to the original signalx, where x * is the recovered or restored signal. The signal size is chosen as n = 2 12 , k = 2 10 and the original signal contains 2 7 randomly nonzero elements. A is the Gaussian matrix generated by the command rand(m, n) in MATLAB. In addition, the measurement y is distributed with noise, that is, y = Ax + η, where η is the Gaussian noise distributed normally with mean 0 and variance 10 −4 (N(0, 10 −4 )).
To show the performance of the MSCG method in compressive sensing, we compare it with the PCG method. The parameters in both MSCG and PCG methods are chosen as β = 1, σ = 10 −4 , ρ = 0.8, and r = 0.1 and the merit function used is f (x) = 1 2 y − Ax 2 2 + τ x 1 . To achieve fairness in comparison, each code was run from same initial point, same continuation technique on the parameter τ, and observed only the behaviour of the convergence of each method to have a similar accurate solution. The experiment is initialized by x 0 = A T y and terminates when where f k is the function evaluation at x k . In Figure 1, MSCG and PCG methods recovered the disturbed signal almost exactly. In order to show the performance of both methods visually, four figures were plotted to demonstrate their convergence behaviour based on MSE, objective function values, number of iterations and CPU time (see . Furthermore, the experiment was repeated for 25 different noise samples (see Table 10). From the Table, it can be observed that the MSCG is more efficient in terms of iterations and CPU time than the PCG method in most cases.

Conclusions
In this paper, a modified three-term conjugate gradient method for solving monotone nonlinear equations with convex constraints was presented. The proposed algorithm is suitable for solving non-smooth equations because it requires no Jacobian information of the nonlinear equations. Under some assumptions, global convergence properties of the proposed method were proved. The numerical experiments presented clearly show how effective the MSCG algorithm is compared to the PCG and SATCGM methods of References [19,20] for the given constrained problems. In addition, the MSCG algorithm was shown to be effective in signal recovery problems.