A Modified Hybrid Conjugate Gradient Method for Unconstrained Optimization

The nonlinear conjugate gradient algorithms are a very eﬀective way in solving large-scale unconstrained optimization problems. Based on some famous previous conjugate gradient methods, a modiﬁed hybrid conjugate gradient method was proposed. The proposed method can generate decent directions at every iteration independent of any line search. Under the Wolfe line search, the proposed method possesses global convergence. Numerical results show that the modiﬁed method is eﬃcient and robust.


Introduction
Consider the following unconstrained optimization problem: where x ∈ R n is a real vector with n ≥ 1 component and f: R n ⟶ R is a smooth function and its gradient g(x) ≜ ∇f(x) is available. Unconstrained problem is an important problem with a broad range of scientific and operational applications. During the last decade, the conjugate gradient methods constitute an active choice for efficiently solving the above optimization problem, especially when the dimension n is large, characterized by the simplicity of their iteration, their low memory requirement, and their excellent numerical performance. e general procedure of the iterative computational scheme is as follows.
When applied to solve problem (1), starting from an initial guess x 1 ∈ R n , the conjugate gradient method usually generates a sequence x k as where x k is the current iterate, α k > 0 is called a step size determined by some suitable line search, and d k is the search direction defined by where g k � ∇f(x k ) and β k is an important scalar parameter. Generally inexact line search is used in order to get the global convergence of conjugate gradient method, such as the Wolfe line search or the strong Wolfe line search. at is, the step length α k is usually computed by the Wolfe line search: or the strong Wolfe line search: where 0 < δ < σ < 1 are some fixed parameters. Different conjugate gradient methods correspond to different values of the scalar parameter β k . e well-known conjugate gradient methods include the Fletcher-Reeves (FR) method [1], the Polak-Ribière-Polyak (PRP) method [2,3], the Hestenes-Stiefel (HS) method [4], the Dai-Yuan (DY) method [5], the conjugate-descent method (CD) [6], and the Liu-Storey method (LS) [7]. e parameters β k in these conjugate gradient methods are specified as follows: where ‖ · ‖ stands for the Euclidean norm. ese methods are identical when f(x) is a strongly convex quadratic function and the line search is exact, since the gradients are mutually orthogonal, and the parameters β k in these methods are equal. When applied to general nonlinear functions with inexact line searches, the behavior of these methods is marked different. It is well known that FR, DY, and CD methods have strong convergent properties, but they may not perform well in practice due to jamming. Moreover, although PRP, HS, and LS methods may not converge in general, they often perform better. Naturally, people try to devise some new methods, which have the advantages of these two kinds of methods. So far, various hybrid methods have been proposed (see [8][9][10][11][12][13][14][15][16][17][18][19][20]).
In [14], Wei et al. gave a variant of the PRP method, WYL method for short, where the parameter β k is yielded by e above WYL method can be considered as the modification of the PRP method. It inherits the good properties of the PRP method, such as excellent numerical effect. Furthermore, Huang et al. [15] proved that the WYL method satisfies the sufficient descent condition and converges globally under the strong Wolfe line search (5) if the parameter satisfies σ < (1/4).
Yao et al. [16] gave a modification of the HS conjugate gradient method as follows: Under the strong Wolfe line search (5) with the parameter σ < (1/3), it has been shown that the MHS method can generate sufficient descent directions and converges globally for general objective functions.
Jiang et al. [17] proposed a hybrid conjugate gradient method with Under the Wolfe line search (4), the method possesses global convergence and efficient numerical performance.
In [18], Jiang et al. proposed a hybrid conjugate gradient method with Under the parameter μ > 1, it has been shown that the MDY method can generate sufficient descent directions and converges globally for general objective functions.
In [19], Jiang et al. proposed a hybrid conjugate gradient method, denote it by JHS with Under the parameter μ > 2, it has been shown that the JHS method can generate sufficient descent directions and converges globally for general objective functions.
In this paper, we introduce a new hybrid choice for parameter β k . is motivation mainly comes from [18,19]. For convenience, we call the iteration method a FW method as follows: where μ > 1.
It is easy to know that β FW . e proposed method has attractive property of satisfying the sufficient descent condition independent of any line search and attains global convergence if the step length is yielded by the Wolfe line search (4).
is paper is organized as follows. In Section 2, we give the details of our algorithm and discuss its sufficient descent property. In Section 3, we prove the global convergence of the proposed method with Wolfe line search (4). A number of numerical experiments comparing the proposed method with other conjugate gradient methods are given in Section 4. Finally, conclusion is given in Section 5.

Algorithm and Its Property
In this section, first, based on the discussed above, we describe our algorithm framework (Algorithm 1) without fixed line search as follows.
e following lemma states that the search direction in Algorithm 1 is always sufficient descent depending on no line search.
Step 2. Determine a step length α k by a suitable line search.
erefore, g T k d k < 0 holds for all k ≥ 1.
□ Lemma 2. Let x k be generated by Algorithm 1. en, for any k ≥ 1, we can obtain the following relations: Proof. From formula (12), it is easy to see that Now we are ready to prove β FW k ≤ (g T k d k /g T k− 1 d k− 1 ) by considering the following four cases.
In view of Lemma 1 and (14), we have erefore, one gets Dividing both sides of (24) by g T k− 1 d k− 1 , it follows that If g T k d k− 1 < 0, similarly, we can get that Dividing both sides of (26) by g T k− 1 d k− 1 , one has From the above formula, we have Combining with (17), we get (iv) If g T k g k− 1 > 0 and μ|g Noticing that g T k d k− 1 < μ|g T k d k− 1 | + g T k− 1 d k− 1 , we have from the above formula that Dividing both sides of (31) by g T k− 1 d k− 1 , we have e proof is complete.

Global Convergence of Algorithm
is section is devoted to the global convergence of algorithm framework under the Wolfe line search condition, i.e., the step length α k is yielded by condition (4). For this goal, we make the following basic assumptions in subsequent discussions.
is continuous differentiable, and its gradient g(x) is Lipschitz continuous, that is to say, for all x, y ∈ Ω 1 , there exists a constant L > 0 such that

Lemma 3. Suppose that assumptions (H 3.1) and (H 3.2)
hold. Let x k be generated by Algorithm 1, where the step length α k satisfies the Wolfe line search (4). en, Proof. In view of (4) and (33), we have erefore, we get Combining (4) and (20), we can get that Let us sum up the inequalities (36) for k � 1, . . .. We obtain is inequality along with the assumption (H3.1) results in Proof. We prove this theorem by contradiction. If lim k⟶∞ inf‖g k ‖ ≠ 0, in view of ‖g k ‖ > 0, there exists a constant c > 0 such that ‖g k ‖ ≥ c, ∀k ≥ 1. Again, from (3), it follows that d k + g k � β FW k d k− 1 . Squaring both sides of above equality, we have Dividing both sides of (39) by (g T k d k ) 2 and in view of (20), it follows that Combining with (40), by a recurrence of relation ‖g k ‖ ≥ c and d 1 � − g 1 , we have

Journal of Mathematics
computing the corresponding test problem (unit: seconds). To visualize the whole behaviour of the algorithms, we use the performance profiles proposed by Dolan and Morè [22] to compare the performance based on the CPU time, the number of function evaluation, the number of gradient evaluation, and the number of iteration, respectively. For each method, we plot the fraction P/τ of the problems for which the method is within a factor τ of the best time. e left side of the figure gives the percentage of the test problems for which a method is the fastest. Based on the theory of the performance profile above, four performance figures, i.e., Figures 1-4, can be generated according to Table 1. From the four figures, we can see that our methods perform effectively on the testing problems.

Conclusion
In this paper, we proposed a new hybrid conjugate gradient method for solving unconstrained optimization problems. e proposed method satisfied sufficient descent condition    Journal of Mathematics irrespective of the line searches condition. Moreover, the global convergence of the proposed method has been established under the Wolfe line search (4). Numerical experiments show the efficiency and robustness of the new algorithm in solving a collection of unconstrained optimization problems from [21].

Data Availability
Data will be available upon request.

Conflicts of Interest
e authors declare that they have no conflicts of interest.