A new modified Leverrier's VM Update for solving

In this paper, a new VM-updating formula for solving unconstrained non-linear optimization problems is proposed. The new method is derived by using Faddeev's modification in order to modify Leverrier's VM-update.The new algorithm is tested by (8) nonlinear test functions, with different dimensions and compared with the standard BFGS algorithm. Introduction Quasi-Newton (QN) methods are subclasses of the more general variable metric methods, but we content ourselves here with the quasi-Newton approach ( Davidon,1959) . The basic form of these algorithms is given as:     k g x f x  ...(1) Where f is twice continually differentiable function xR n Here the notation gk is used, as before, for the gradient vector of the objective function at xk, and we also retain the notations: k k k 1 k k d λ x x v     ...(2) k 1 k k g g y    ...(3) Methods vary according to how the metric matrix Hk is updated from iteration to iteration. Once such an updating formula has been defined, according to QN-Condition, the algorithm is completed. The matrices are called metric and they are assumed to be positive definite and defined on R n . The search direction is defined in the above algorithm as the steepest descent direction relative to this metric. The idea is to update Hk, in such a way that the search directions used for the first few iterations are close to the steepest descent direction until, the minimum value is approached. Thus, the matrix Hk can be viewed as an approximation to the inverse Hessian matrix of the objective function f at its minimum x * . It is usual to take Ho=I so that the initial search direction is the scaled steepest descent direction.The development of these algorithms started with Davidon (1959), whose Argoune National Laboratory report was improved upon by Journal of Kirkuk University –Scientific Studies , vol.3, No.2,2008 Fletcher and Powell (1963) to produce the famous Davidon-FlethcherPowell ( DFP), method. We begin our study for the quasi-Newton update formulas with this DFP algorithm. For the development of this theory, we will again restrict our attention to the positive definite quadratic objective function.   2 Gx x x b c x f T T    ...(4) In this paper we are going to develop a much wider applicability algorithm than the DFP method. We recall that for any step k 1 k k x x v    We have k k v y G  ...(5) And so, if Hk+1 is to be viewed as an approximation to G -1 , it is natural to require that; k k 1 k v y H   , ...(6) which is called the quasi-Newton condition. Since G is positive definite, it is natural to demand that the Hk’s should be also. This requirement can also be viewed as the requirement that the search directions are downhill, since the positive definiteness of Hk implies that: 0 g H g g d k k T k k T k    ...(7) Provided that gk0, that is, provided the minimum has not already been found. The derivation of the particular updating formula is also based on the fact that generally conjugate search directions will guarantee the finite termination property; in ths case the final matrix generated would be Hk. Ideally, we would like this to be G -1 . All the requirements are fulfilled by the DFP updating formula:  


Introduction
Quasi-Newton (QN) methods are subclasses of the more general variable metric methods, but we content ourselves here with the quasi-Newton approach ( Davidon,1959) . The basic form of these algorithms is given as: (1) Where f is twice continually differentiable function xR n Here the notation g k is used, as before, for the gradient vector of the objective function at x k , and we also retain the notations: Methods vary according to how the metric matrix H k is updated from iteration to iteration. Once such an updating formula has been defined, according to QN-Condition, the algorithm is completed. The matrices are called metric and they are assumed to be positive definite and defined on R n . The search direction is defined in the above algorithm as the steepest descent direction relative to this metric. The idea is to update H k , in such a way that the search directions used for the first few iterations are close to the steepest descent direction until, the minimum value is approached. Thus, the matrix H k can be viewed as an approximation to the inverse Hessian matrix of the objective function f at its minimum x * . It is usual to take H o =I so that the initial search direction is the scaled steepest descent direction.The development of these algorithms started with Davidon (1959), whose Argoune National Laboratory report was improved upon by Fletcher and Powell (1963) to produce the famous Davidon-Flethcher-Powell ( DFP), method. We begin our study for the quasi-Newton update formulas with this DFP algorithm. For the development of this theory, we will again restrict our attention to the positive definite quadratic objective function.

 
In this paper we are going to develop a much wider applicability algorithm than the DFP method. We recall that for any step ...(5) And so, if H k+1 is to be viewed as an approximation to G -1 , it is natural to require that; ) which is called the quasi-Newton condition. Since G is positive definite, it is natural to demand that the H k 's should be also. This requirement can also be viewed as the requirement that the search directions are downhill, since the positive definiteness of H k implies that: Provided that g k 0, that is, provided the minimum has not already been found. The derivation of the particular updating formula is also based on the fact that generally conjugate search directions will guarantee the finite termination property; in ths case the final matrix generated would be H k . Ideally, we would like this to be G -1 . All the requirements are fulfilled by the DFP updating formula: The general formula, in a slight modification of Fletcher's (1970) parameterization, is given by Where   0 is the free parameter and Much effort, both analytically and computationally, has been devoted to identifying the best quasi-Newton formula or even the best from the much wider class of variable metric methods introduced by Huang (1970). The choice with the widest support is the BFGS algorithm which was derived independently in 1970 in four different ways by Broyden, Fletcher, Goldfarb, and Shanno. In the parameterization of (9), this BFGS update formula corresponds to choosing.

Faddeev's Modification of Leverrier's Method
Consider that we have the system of equations: Gv= v …(12) where the scalar  is an eigenvalue (or a characteristic value) of matrix G nxn and the vector v is the eigenvector (or a characteristic value) corresponding to . The problem often requires the solution of the homogeneous set of equation: To determine the values of  and v which satisfy this set. However, before we proceed with developing method of solution. Let's examine equation (13)  It can be stated that for a nonsingular matrix G of order n there are n characteristic directions in which the multiplication by G does not change the direction of the vector, but only changes its length. More simple stated, matrix G has n eigenvectors and n eigenvalues. ... (16) The determinate can be expanded by minors to yield a polynomial of n th degree: This polynomial, which is called the characteristic equation of matrix G, has n roots which are the eigenvalues of G. A nonsingular real symmetric matrix of order n has n real distinct eigenvalues and n linearly independent eigenvectors. The eigenvectors of a real symmetric matrix are orthogonal to each other. The coefficients  i of the characteristic polynomial are functions of the matrix elements a ij , and must be determined before the polynomial can be used.
The well-known Cayley-Hamilton theorem states that a square matrix satisfies its own characteristic equation, i.e.
The problem of evaluating the eigenvalues and eigenvector of matrices is a complex multipstip procedur. Several methods have been developed for this purpose some of these apply to symmetric matrices,others to tridiagonal matrices, and a few can be used for general matrices. The method in this section work the original matric G and its characteristic polynomial (17) to evaluate the coefficients  i of the polynomial(Alkis ,1987). One such method is the Faddeev-Leverrier procedure. Once the coefficients of the polynomial are known, the methods use root-finding techniques, such as the Newton-Raphson or Graeffe's method. To determine the eigenvalues. Finally, the algorithms employ a reduction method, such a Gauss elimination, to calculate the eigenvectors (Fletcher,1970).
The Faddeev-Leverrier method calculates the coefficients  i to  n of the characteristic polynomial (17) by generating a series of matrices G k whose traces are equal to the coefficients of the polynomial. Let us start in matrix G and first coefficient are: ...(20)

Journal of Kirkuk University -Scientific Studies , vol.3, No.2,2008
In addition to this, the Faddeev-Leverrier method yields the inverse of the matrix G by: For more details, see Alkis (1987)

New VM-Updating Formula for Leverrier Method
In this section we present a new updating formula to minimize the function f(x), x R n where f(x) is assumed to be at least twice continuously differentiable and where the function and the first derivatives can be evaluated at any point x. Now from equation (22) and (23) : , and let G k satisfy the equation Now multiplying and dividing equation (24)

Outlines of New proposed Algorithm
Step (1): set x 1 , , Step (2): for k=1 to n set d 1 = -H 1 g 1 Step (3): set H tr k   Step (4): compute x k+1 = x k + k d k , where  k is obtained from line search procedure.

Numerical Results
In order to investigate the performance of the new proposed algorithm eight test functions were tested with different dimensions (4  n  500) and all programs are written in FORTRAN 90 language; and for all cases the stopping criterion is taken to be ||g k+1 || < 110 -5 . The numerical results are given in the table (1) is specifically quoted the number of functions NOF and the number of iterations NOI.

List of symbols
Meaning Symbol n is the dimensions of the problems K is the K-th step of iterations F is the twice differentiable real value function g is the n ×1 gradient vector of f(x) d is the n ×1 search direction vector G is the n ×n Hessian matrix H is the n ×n approximation to G -1 matrix v is the n ×1 difference vector between two successive points λ is the positive scalar which minimizes f(