α-PSB update method

In this work, we propose modification for PSB update with a new extended Quasi – Newton condition for unconstrained optimization problem, so it’s called (α – PSB) method. PSB update kind of rank two update, which solve the unconstrained optimization problem, but this update can’t guarantee the positive definite property of Hessian matrix. In this work, the guarantee of positive definite property for the Hessian matrix be confirmed by updating the vector sk, which represent the difference between the next gradient and the current gradient of the objective function which assume to be continuous twice and differentiable. Then we proved the existentialism of this update. Numerical results are reported where the comparison between the proposed method and the PSB update method under standard problems.


Introduction
Today's Quasi -Newton methods are considered one of the most effective methods to solve nonlinear unconstrained or bounded constrained optimization problem.
These methods are mostly used when the second derivative matrix of the target function is either un available or too expensive to compute. It is very similar to Newton's method, but avoid the need of computing Hessian matrices by repeating a symmetric matrix that, from repetition to repetition can be considered as an approximation for Hessian. Thus, it allow the curvature of the problem to be exploited in the numerical algorithm, despite the fact that only first derivative (gradients) and the values of the functions are required [2]. The Quasi-Newton methods are very useful and effective methods for solving the unconstrained minimization problem   (2) where is satisfy the following formula of Quasi-Newton condition where the is the step length and is a search direction we got it by solving the equation in which is the gradient of at and is an approximation to the Hessian matrix The updating matrix is wanted to satisfy the Quasi-Newton equation (3) with (4) . So that is reasonable approximation to .
PSB update is important in both theoretical and practical research computing. But, the disadvantage is that the PSB update can't keep the Positive definiteness of updates is detrimental to their computing performance. Luckily, The disadvantage can be avoided if we use the trust region range with PSB updated [7].

2.
method PSB update it is officially known as the Powell-symmetric-Broyden update is important in theoretical research and practical computing. But, the disadvantages that PSB update can't retain the positive definiteness of updates hurts its performance in computing [7].

PSB update it can written as
The PSB method tries to update the Hessian matrix by using the formula (2.1) which represent the solution of the Quasi-Newton condition (3) but the update is not preserve the positive definite property that means there is no guarantee to minimize the function at each iteration, so if the current Hessian matrix approximation is positive definite then, the next Hessian matrix approximation may be In this modification we guarantee the positive definite property for the Hessian matrix by updating the vector by multiplication with , so will become .

Algorithm of update
Step Given an initial point and an initial symmetric and positive definite matrix , , let k = 0 .
Step Compute , set ; Step : Carry out a line search along getting ; Step : Set ; Step : Determine by equation (2.8) ; Step : Determine by (2.9), where and are defined by Q-N equation (3).
Step : If then stop .
Step k = k + 1 , go to step .

Numerical results:
In this part, we devoted to numerical experiments. We compare the performance of the modified PSB algorithm with the performance of the standard PSB algorithm, using the same starting points and convergence criteria and limits. We wrote the computer programs by MATLAP version (5.3). The reason for choosing them is that the problems appear to have been used in standard problems in the most of the literature, and these functions are a result of application in the branch of technology and industry. The performance of method has been tested and compared with standard PSB method using the following test functions. For the sake of uniformity in comparison. The results are presented in table . The test function are chosen as follows 1-Extended Himmelbla function [1] ∑ ( ) ( )

7-Extended Rosen brock function, [7]
∑[ ] 8-Wood function, [7] 9-Least square equation for two dimensions [1] 10-Cube function, [7] The table  gives the comparison between the usual PSB update method and the modified for convex optimization, for our selected test functions. At the first, we give the number of iterations and function evaluation of each function from deferent starting point. Now we discuss the results between α update method and PSB update. Note that the function number (1) at the starting point [-1; 5; -1; 5; -1; 5] from the table (4-1) that was resolved by α method it is better than the same function that was resolved by PSB update, because Feval for α update method is (1.4767e-013) but the Feval for PSB update is (118.2017). And we notice the preference in converging to the rest of the functions number (2),(3),(4),(5),(7),(8),(9) and (10) Also in the function number (2) with starting point [-1;0;-1;0;-1;0] and the function number (6) with starting point [-1;0] we notice, the function solved by method but can't be solved by PSB method. PSB can't terminate at the minimum point, because the Hessian matrix is not positive definite at every iteration, that means there is some iterations do not finding the minimizer at this iteration, so the objective function can't have a descent direction.

Conclusion
In this paper, we proposed an update formula to modify PSB method with an extended Quasi-Newton condition (3) for unconstrained optimization problem.
We used a new equivalent method (2.9), to update the Hessian matrix. We develop a new formula of PSB update method which makes the system convergence to a minimum by generating a positive definite Hessian matrix in each iteration. This new formula called modified PSB update . This formula of update method meets the modifying of Quasi-Newton condition over condition as the derivation of this formula relied on this vector and then is given suitable algorithm, in addition to resolving. Then we proved the symmetric property for in theorem (2.1) and we proved the existentialism of this update in lemma (2.2). Then we achieved numerically convergence in Table ( Table (4.1) have a better convergence for the method than the PSB method.