New Extended CG Algorithm For Non-Linear Optimization

This paper presents the development and implementation of a new algorithm based on non-quadratic rational function model.The derivation of the new algorithm is based on aquadratic function with exact line searches and evaluated numerically against the standard CG-algorithm by using(25) non-linear test functions with different dimensions. The numerical results indicate that the new algorithm is found to be superior to the standard CG algorithm.


Introduction:
Conjugate gradient methods are iterative methods which generate a sequence of approximations to minimize a function f(x).The methods are based on an important concept of conjugating two vectors x are orthogonal when G is n×n identity matrix I). Several algorithms have been proposed in literature for generating conjugate directions of quadratic forms.The first conjugate gradient method was published by Hestense and Stiefel in (Hajitharwat,H. and AI-Bayati,A.Y.,2005), for solving a system of linear algebraic equations. Fletcher and Reeves (Bunday,1984) were the first among others scholars, to use this technique to minimize a non linear function of several variables.

Definition:
If q(x) is quadratic function, then a function f is defined as a non-linear scaling of q(x) if the following condition holds: where  x is the minimizer of q(x) with respect to x (Taqi,& AI-Assady,2000). The following properties are immediately derived from the above condition : x is a minimizer of q(x) ,then it is a minimizer of f. iii) That  x is a global minimum of q(x) does not necessarily mean that it is a global minimum of f.
Many authors have published related work in this area: i) A CG methods which minimize the following polynomial model ...(2) In at most n steps has been proposed in (Fried, 1971). ii) Two CG methods which minimize the following polynomial model.
...(6) have been implemented by Al-Bayati (Al-Bayati, 1995). v) And Taqi A. and Al-Assady (Tassopoulos & Storey,1984b) described their ECG algorithm which based on the natural log function for the rational q(x) function: where 1  and 2  are scalars. vi) Also Al-Mashhadany H (Al-Mashhadany, 2002) has been developed a new rational models which is defined as following: vii) Finally, another specific rational model was considered by Haji Tharwat H. (Hajitharwat & AI-Bayati,2005) which is defined as following: In this paper a new extended CG method (ECG) is investigated and tested on a set of some standard test functions. The new model is given as: where 1  and 2  are scalars.
It is assumed that: holds However, we first observe that q(x) and f(x) given in the above new model have identical contours through with different function values, and they have the same unique minimum point  x .

The derivation of new ECG -method:
The key element of the modified algorithms is the determination of the expression It is first assumed that neither 1  and 2  is zero in (10), solving (10) for q(x), then: the quantity which has be determined explicitly is 2 1   during every iteration 2 1   must be evaluated as a function of known available quantities.
From the relation: where G is Hessian matrix and  x is the minimum point from the above system we have: ... (17) from (12) and (15), we get where i q is the quadratic function defined by:   (14) and (21), it follows that: Equation (23) can be rewritten as: modified P/r in (Polak& Ribier,1969) (vii) Check for convergence If   i g , then stop, else go to step (viii) (viii) Check for restarting criterion If i = n, set i = 0 and x 0 = x n then go to step (i) Else set i = i +1 and go to step (ii)

Numerical Computation:
To test the effectiveness of ECG-method, a number of standard test functions were solved in order to compare the new algorithm with the standard CG method the identical linear search was used, namely a cubic fitting procedure described in Bunday (Bunday, 1984).Finally the convergence criterion used in each case is that 5 10 5  i g for this ECGmethod with E/S all computations in double precision arithmetic are performed by using personal computer (Pentium iv), all programs are written in Fortran language. All the results given in the tables specifically count the number of function calls (NOF) and the number of the iterations call (NOI). Results in table 1 and 2 give the comparison of new ECG with standard CG method.