A New Hybrid Algorithm for Convex Nonlinear Unconstrained Optimization

In this study, we tend to propose a replacement hybrid algorithmic rule which mixes the search directions like Steepest Descent (SD) and Quasi-Newton (QN). First, we tend to develop a replacement search direction for combined conjugate gradient (CG) and QN strategies. Second, we tend to depict a replacement positive CG methodology that possesses the adequate descent property with sturdy Wolfe line search. We tend to conjointly prove a replacement theorem to make sure global convergence property is underneath some given conditions. Our numerical results show that the new algorithmic rule is powerful as compared to different standard high scale CG strategies.


Introduction
The nonlinear CG technique could be a helpful procedure to search out the minimum value of any nonlinear function through exploitation unconstrained nonlinear optimization strategies.
Let us contemplate the subsequent unconstrained minimization problem: where  :   →  is a real-valued smooth function.The repetitious formula is given as where  k is associate optimum step-size computed by any line search procedure [1].The search direction   is defined as and g k = g(x k ) denotes ∇f(x k ), while  k is a positive scalar.
In addition, Ibrahim et al. [11] propose another search direction that is outlined as The positive scalar  and   are the Hestenes-Stiefel parameters.

A New Proposed Search Direction
In this section, we advise a replacement search direction as deduced from Ibrahim et al. [9][10][11].The new search direction is outlined as whereas H k+1 (the approximation matrix of BFGS updating matrix) denotes approximations of Hessian matrix G, and  k could be a positive constant.In order to drive the value of  k , we have a tendency to multiply either side of ( 11) by y T k to induce Since     +1 = s T k and     +1 = −t    +1 (Perry condition [12]), then In order to see value of   , we tend to additionally multiply either side of (11) to induce As a result of processes of multiplication shown in ( 13) and ( 16), we tend to reach the following search directions in (17a), (17b), and (17c).The subsequent new search directions are our new projected algorithmic program: In the following step, we have a tendency to assume that each search direction got to satisfy the subsequent descent condition (     < 0, for all k).Also, there should exist a constant c>0 in order to get For all  ≥ 0, the new direction that is outlined in (18) ought to satisfy the sufficient descent condition.The enough descent conditions are going to be used later to prove our new theorem (see Section 2.2).So to prove our new theorem, we have a tendency to necessarily use the subsequent given assumptions (see Section 2.1).[9,11,13] (A1) f : R n → R is twice continuously differentiable.

Assumptions in
(A2) f is uniformly convex; that is, m and M are positive constants, such that for all x, z ∈ R n , and G is the Hessian matrix of f.
(A3) The matrix G is Lipschitz continuous at the point x * ; that is, there exists the positive constant L satisfying for all x in a neighborhood of x * .

A New Theorem for Proving Sufficient Descent Property.
To prove that our new projected algorithm defined in (17a), (17b), and (17c) satisfies sufficiently descent condition, we tend to suppose that assumptions in (Section 2.1) square measure are true.Additionally, the sequence   is bounded.Then, sufficient descent condition (18) is true for all k ≥ 0.
Proof.When taking (17a), (17b), and (17c) and achieving the descent condition, we can see the following: We get value  = −(  −   ), which is bounded away from zero.Therefore ( 18) is true.[10,14].Suppose that assumptions in Section 2.1 are true.Then, the step-size   which is determined by (2) satisfies

Lemma in
when  3 is a positive constant.

New Theorem for
Hence, from our new theorem in Section 2.2, we can define that ‖d k ‖ ≤ −c‖g k ‖, and we can therefore simplify (23) as Thus, the proof is established.

A New Form for the Parameter 𝛽 𝑘 in Conjugate Gradient Method
To obtain an updated version of the conjugate gradient method associated with a new parameter   , we compare the standard CG-method specified in (3) with the proposed new algorithm specified in (17a), (17b), and (17c): Multiplying both sides of (24) by y T k we get Or and   =  +1 −  ,   =  +1 −  .It should be noted that when using exact line searches assuming that  k = 1 and the matrix H k+1 is the identity matrix, the new standard will be reduced to HS.This condition must also be met, with s T k H k+1 s k > 0 (positive constant), and since after these conditions we know the new parameter  hh k is In conjunction with both parameters and the search direction is defined as
Step 2. Compute  k by strong Wolfe line search conditions in (4) and in (6).
Step 4. 15] is satisfied, then go to Step 1; if not, then continue.
(i) The level set Proof.By using the mathematical induction we demonstrate this new theorem; for initial direction (k=1) we have which satisfies (18).We suppose that and since 0 <  < 1 → (1 − ) > 0 The above equations ensure that condition (18) is satisfied.Hence, the proof is complete.[1,16].By assuming that assumptions in Section 2.1 are true, by supposing that any CG-method with search direction  +1 could be a descent direction provided that the step-size  k is obtained by the strong Wolfe line search conditions, and if

A New Theorem for Proving the Global Convergence
Property of the New Algorithm in (29a), ( 29b), ( 29c), (29d), and (29e).If we suppose that assumptions (i) and (ii) in (30) and in (31), respectively, are true and if we have a tendency to assume that (A3) in Section 2.1 is additionally true, then the search directions  +1 outlined in (29e) are descent provided that the step-size   is computed using ( 4) and ( 6

Numerical Results and Comparisons
In this work, we have a tendency to compare our new proposed CG-method with some normal classical CG strategies like Hestenes-Stiefel [HS] and Dai-Yuan [DY] by exploitation of fifty unconstrained nonlinear cases; take a look at functions obtained from Andrei [17,18].As for the computer program, it was stopped when ‖  ‖ ≤ 10 −6 .In addition, the term     +1   , which is defined in (29a), can be computed as     +1   = 2(  −  +1 ) + 2    +1 .Numerical results for new algorithm in (29a), (29b), (29c), (29d), and (29e) with  = 1 and  = 0.1 are for the total of 50 test problems from the CUTE library.The Sigma plotting software was used to graph the data.We adopt the performance profiles given by Dolan and Moré [19].Thus, new, HS, and DY strategies are compared in terms of NOI, CPU, and NOF in Figures 1-3.For each method, we plotted the fraction of problems that were solved correctly within a factor of the best time.In the figures, the uppermost curve is the method that solves the most problems within a factor t of the best time.In Figures 1-3, the new method outperforms the HS algorithm and DY method in terms of NOI, CPU, and NOF.If the solution had not converged after 800 seconds, the program was terminated.Generally, convergence was achieved within this time limit; functions for which the time limit was exceeded are denoted by "F" for fail-in.

Figure 2 :Figure 3 :
Figure 2: Performance profiles based on CPU time.