A MODIFIED FLETCHER-REEVES-TYPE DERIVATIVE-FREE METHOD FOR SYMMETRIC NONLINEAR EQUATIONS

In this paper, we propose a descent derivative-free method for solving symmetric nonlinear equations. The method is an extension of the modified Fletcher-Reeves (MFR) method proposed by Zhang, Zhou and Li [25] to symmetric nonlinear equations. It can be applied to solve large-scale symmetric nonlinear equations due to lower storage requirement. An attractive property of the method is that the directions generated by the method are descent for the residual function. By the use of some backtracking line search technique, the generated sequence of function values is decreasing. Under appropriate conditions, we show that the proposed method is globally convergent. The preliminary numerical results show that the method is practically effective.

1. Introduction. We consider the nonlinear equation where : → is continuously differentiable. We suppose that for each ∈ , the Jacobian ′ ( ) of at is symmetric.
Among various methods for solving nonlinear equations (1), Newton's method is quite welcome due to its nice properties such as the rapid convergence rate, the decreasing of the function value sequence and the global convergence. However, at each iteration, Newton's method needs the computation of the derivative ′ as well as the solution of some system of linear equations.
On the other hand, the derivative-free methods including quasi-Newton methods [10,18,19,24,26,27], the so called spectral method [4,14,15] and some recently developed derivative-free methods for monotone nonlinear equations [20,23], can be used to solve nonlinear equations where the derivative is not available or very difficult to compute. The existing derivative-free methods are globally convergent if some nonmonotone line search strategy is used. We refer to a survey paper [17] for a summary in the derivative-free quasi-Newton methods for solving nonlinear equations. However, since the first-order information is not available, the existing derivative-free methods are generally not descent method in the sense that the directions generated by the method may not be descent directions for the residual function of (1).
Recently, Gu, Li, Qi and Zhou [10] got a way to construct norm descent quasi-Newton directions and proposed a norm descent Gauss-Newton-based BFGS method for solving symmetric nonlinear equations. It was proved that the method with Armijo-type line search is globally convergent. This technique was extended by Zhang and Li [24] to solving some kinds of nonsmooth equations.
In this paper, we further study the derivative-free methods for nonlinear equations. We focus our attention to the symmetric nonlinear equation (1). We will extend the recently developed modified Fletcher-Reeves (MFR) method [25] for solving unconstrained optimization to the symmetric nonlinear equations. The MFR method is a modification to the well-known Fletcher-Reeves (FR) [8] method for solving optimization problems. The FR method is the first nonlinear conjugate gradient method for solving optimization problems. It enjoys some nice properties such as the finite quadratic termination property and the global convergence property. The global convergence of the FR method for solving optimization problems has been well studied. The first global convergence result for the FR method was given by Zoutendijk [28] in 1970. It was proved in [28] that the FR method with exact line search is globally convergent. The first global convergence result of the FR method with an inexact line search was obtained by Al-Baali [1] in 1985. Other good results about the convergence property of the FR method and its improvement can be found in [5,7,9,11,13,16,21,22] etc.. We refer to a survey paper [12] for a good review in the convergence property of the FR method and related conjugate gradient methods for optimization.
The MFR method in [25] not only reserves good properties of the FR method but also possesses another nice property that it always generates descent directions for the objective function. This property is independent of the line search used. Under suitable conditions, the MFR method with Armijo line search is also globally convergent [25]. The purpose of this paper is to develop a norm descent MFR type derivative-free method for symmetric nonlinear equations. To this end, we first adopt the strategy in [10] to get an estimation to the gradient of the norm function of (1). We then use the idea of the MFR method in [25] to construct a descent direction for the norm function. By the use of some backtracking line search, we develop an MFR type derivative-free method for solving (1) that generates descent directions for the norm function of the equation (1). Under appropriate conditions, we establish the global convergence of the method. It should be pointed out that in contrast to the derivative-free quasi-Newton methods, the proposed method is not only derivative-free, but also completely matrix-free. Consequently, it can be applied to solve large scale symmetric nonlinear equations. We also do some preliminary numerical experiments to test the proposed method. The results show that the proposed method is practically efficient.
In the next section, we propose the method. Section 3 is devoted to the global convergence of the method. At last we present some numerical results in Section 4.

2.
Algorithm. In this section, we propose the MFR-type derivative-free method. First we simply recall the descent derivative-free quasi-Newton method in [10] for solving the symmetric nonlinear equation (1). Denote We call a direction ∈ a descent direction of at if it satisfies where ∇ ( ) = ′ ( ) ( ) denotes the gradient of at . Taking into account the symmetry of ′ ( ), we have The above relation shows that the vector ( ( + ( )) − ( )) −1 is a good estimate to ∇ ( ) if is small. Based on this observation, Gu, Li, Qi and Zhou [10] got an estimation to the steepest descent direction Let be the current iterate and ( ) be defined by (3) with = . Then we have for all sufficient small Let > 0 be sufficiently small such that −∇ ( ) ( ) < 0. Then the quasi-Newton direction in [10] is determined by the following system of linear equations: is an approximation to ′ ( ) ′ ( ) = ′ ( ) 2 . By the use of the approximate gradient function ( ), we are going to develop a descent MFR type method for solving symmetric nonlinear equation (1). The basic idea is to use − ( ) instead of the steepest descent direction in the MFR method for solving (2). Let us have a look at the MFR method proposed by Zhang, Zhou and Li [25].
Consider the unconstrained optimization problem: Let be the current iterate. The direction in the MFR method of [10] for solving the unconstrained optimization is determined by . This direction enjoys a nice property that This means that provides a descent direction of at . We use this idea to develop an MFR type method for solving (1). The key is to get the direction that takes the form of the MFR direction without computation of derivative and is a descent direction of defined by (2) at .
Suppose at the −th iteration, we already have the iterate . Let −1 be the direction generated by the previous iteration. Similar to the direction in the MFR method, we define ( ) with parameter as follows: where −1 is an estimation to ∇ ( ) to be determined, and ( ) is defined by (3). By direct computation, we can easily get It is not difficult to show by the definition of ( ) that the limit lim →0 ( ) exists. Moreover, it follows from the fact lim →0 ( ) = ∇ ( ) that If −1 is close to ∇ ( ), then (0) is close to the direction generated by the MFR method. Therefore we have This shows that when is sufficiently small, ( ) will be a descent direction of the norm function of defined by (2). In our method, we choose such that the following inequality is satisfied: where constants 1 , 2 and 3 satisfy 1 ∈ (0, 1) and 2 > 0, 3 > 0.
The following lemma shows that inequality (8) holds for all > 0 sufficiently small. It follows from (7) immediately.
Lemma 2.1. Let be continuously differentiable and ′ ( ) is symmetric for every . Then the inequality (8) holds for all > 0 sufficiently small.
The following procedure provides a way to determine and . It is very similar to the Procedure 1 in [10].
Let be generated by Procedure 1. Let constants , = 1, 2, 3 and be the same as those in Procedure 1. If = 0, we let = 1. Otherwise, we let be the largest positive integer ∈ {0, 1, 2, ⋅ ⋅ ⋅ , − 1} satisfying Now, we give the steps of the MFR type derivative-free method for solving the symmetric nonlinear equation (1) as follows.
Step 2. Use Procedure 2 to determine the steplength .
It is easy to see that the sequence of the function evaluations { ( )} generated by Algorithm 1 is decreasing. The following proposition shows that Algorithm 1 is well defined. 3. Global convergence. In this section, we prove the global convergence of Algorithm 1. We first make the following assumption.

Assumption 1.
(1) The level set Ω = { ∈ | ( ) ≤ ( 0 )} is bounded. (2) In some neighborhood of Ω, F is continuously differentiable and ′ ( ) is Lipschitz continuous, namely, there exists a constant > 0 such that Since { ( )} is decreasing, it is clear that the sequence { } generated by Algorithm 1 is contained in Ω and hence bounded. In addition, it is easy to see that there are positive constants 1 , 2 and 1 such that In the latter part of this section, without specification, we always suppose that the conditions in Assumption 1 hold and that ′ ( ) is symmetric for every ∈ .
The following lemma can be obtained by the steps of Algorithm 1 and the conditions of Assumption 1 directly.

DONG-HUI LI AND XIAO-LIN WANG
and The following lemma estimates a lower bound to the steplength.
Lemma 3.2. Let { } be generated by the MFR type method. Then there is a constant > 0 such that the steplength generated by Procedure 2 satisfies Proof. It follows from Procedure 2 that if ∕ = 1, then −1 dose not satisfy inequality (8). This implies By the use of the mean-value theorem and inequality (12), there is a ∈ (0, 1) such that Similarly, by the use of the mean-value theorem to the one dimensional function there is a¯ ∈ (0, 1) such that where the inequality follows from (11). We get from inequalities (16), (17) and (18) −1 Consequently, it holds that Since {∥ ( )∥} is bounded, the last inequality shows that there is a constant > 0 such that By the line search rule, we get (15).
The next lemma reveals a good property of , which is similar to a property of generated by the MFR method for unconstrained optimization. We let¯ = , where is determined by Procedure 1. Then we have = (¯ ) and = (¯ ). The equality (5) is written as For simplicity, we abbreviate (¯ ) and (¯ ) as and respectively, where ( ) and ( ) are defined by (5).
Proof. By the definition of , we can rewrite as where ∈ × denotes the identity matrix. We note that the two terms in the right hand side of (21) are orthogonal. Taking Euclidian norm in both sides of (21), we get for each ≥ 1 Dividing by ∥ ∥ 4 in both sides of the last equality and then taking summation from = 1 to , we get the desired inequality (20).
The following theorem establishes the global convergence of Algorithm 1.
For the sake of contradiction, we suppose that the (24) dose not hold. Then there exists a constant > 0 such that ∥ ∥ ≥ for all ≥ 0. By the boundedness 78 DONG-HUI LI AND XIAO-LIN WANG of {∥ ( )∥}, we get from (15) and (23) that there is a constant > 0 such that the following inequality holds for all sufficiently large where the equality follows from (19). This together with (14) implies However, we get from (19) that ∥ ∥ ≥ ∥ ∥ ≥ . The last inequality implies On the other hand, we get from (20) where¯ is an upper bound of {∥ ∥ 4 }. So, we get from (25) It is a contradiction. Consequently, (24) holds and the proof is complete. 4. Numerical experiments. In this section, we report some preliminary numerical results with the proposed method. We test the performance of Algorithm 1 on the following two problems coming from [2], [18] and [27] with various dimensions and different initial points. with Engval function : → defined by The related symmetric nonlinear equation is . Problem 2. The function is given by and is given by [2].
We compared the performance of Algorithm 1, which we call MFR t, with that of the descent BFGS method in [10], which we call DBFGS, and the DF SANE method in [15] on the two problems. The algorithms were coded in MATLAB 6.5 and run on a personal computer with a 1.66GHZ CPU processor.
The parameters are specified as follows. For Algorithm 1, we set 1 = 10 −4 , 2 = 10 −4 , 3 = 10 −4 , = 0.4. For the DBFGS method in [10], we set = 10 −5 , = 10 −5 , = 2000 and 0 = . The parameters of the DF SANE method were set to be the same as those used in [15]. We stopped the iteration if the inequality ≤ 10 −5 is satisfied or the number of iterations exceeds 10 4 . We first tested the algorithms on small and medium size problems and compared them in the total number of iterations and the CPU time used. The results are listed in Tables 1 and 2. The meaning of each column in Tables 1 and 2 is stated as follows.
"Init": the initial point 1 = (1, 1/2, . . . , 1/ ) , ' ' means 0 = ( , , . . . , ) ; "Dim": the dimension of the problem; "iter": the total number of iterations; "time": the CPU time in seconds; "f": the final value of the norm function of the equation. In each table, we also counted the average number of iterations or the average CPU time, which corresponds to the "average" row. Table 3 lists the average CPU time and the average iteration numbers for each algorithm on the two problems with different dimensions.
We see from Tables 4.1-4.3 that the proposed method can solve the symmetric nonlinear equations efficiently. In all cases, the method terminated at a solution of the problem. In most cases, the MFR t method performed much better than the DBFGS method did. This feature becomes more evident as the dimension is increased to 200. The results in Table 3 also show that the MFR t method performed better than the DF SANE method did, especially for Problem 1 (see Table 1). However, The MFR t method performed not as well as the DF SANE method for Problem 2 (see Table 2).   We then tested the MFR t method and the DF SANE method on the two problems with larger dimensions. The results are listed in Tables 4 and 5. We see from Table 4 that the MFR t method also performed better than the DF SANE method did on Problem 1. The results in Table 5 show that both methods can solve Problem 2 efficiently. However, the performance of the MFR t method is not as good as that of the DF SANE method.