A new logarithmic penalty function approach for nonlinear constrained optimization problem

. 2018 by the authors; licensee Growing Science, Canada ©


Introduction
In this paper, we consider the following nonlinear constrained optimization problem: where : → and : → , ∈ , are continuously differentiable functions on a nonempty set ⊂ .For the sake of simplicity, let : 0,⩝ 1,2, … , be the set of all feasible solutions for the constrained optimization problem (P).
The problem (P) has many practical applications in engineering, decision theory, economics, etc.The area has received much concern and it is growing significantly in different directions.Many researchers are working tirelessly to explore various methods that might be advantageous in contrast to existing ones in the literature.In recent years, an important approach called penalty function method has been used for solving constrained optimization.The idea is implemented by replacing the constrained optimization with a simpler unconstrained one, in such a way that the constraints are incorporated into an objective function by adding a penalty term and the penalty term ensures that the feasible solutions would not violate the constraints.Zangwill (1967) was the first to introduce an exact penalty function and presented an algorithm which appears most useful in the concave case, a new class of dual problems has also be shown.Morrison (1968) proposed another penalty function methods which confirmed that a least squares approach can be used to get a good approximation to the solution of the constrained minimization problem.Nevertheless, the result obtained in this method happens to be not the same as the result of the original constrained optimization problem.Mangasarian (1985) introduced sufficiency of exact penalty minimization and specified that this approach would not require prior assumptions concerning solvability of the convex program, although it is restricted to inequality constraints only.Antczak (2009Antczak ( ,2010Antczak ( ,2011) ) studied an exact penalty function and its exponential form, by paying more attention to the classes of functions especially in an optimization problem involving convex and nonconvex functions.The classes of penalty function have been studied by several researchers, (e.g.Ernst & Volle, 2013;Lin et al., 2014;Chen & Dai, 2016).Just a while ago, Jayswal and Choudhury (2014) extended the application of exponential penalty function method for solving multi-objective programming problem which was originally introduced by Liu and Feng (2010) to solve the multi-objective fractional programming problem.Furthermore, the convergence of this method was examined.
Other researchers (see for instance (Echebest et al., 2016;Dolgopolik, 2018) further investigated exponential penalty function in connection with augmented Lagrangian functions.Nevertheless, most of the existing penalty functions are mainly applicable to inequality constraints only.The work of Utsch De Freitas Pinto and Martins Ferreira (2014) proposed an exact penalty function based on matrix projection concept, one of the major advantage of this method is the ability to identify the spurious local minimum, but it still has some setback, especially in matrix inversion to compute projection matrix.The method was restricted to an optimization problem with equality constraints only.Venkata Rao (2016) proposed a simple and powerful algorithm for solving constrained and unconstrained optimization problems, which needs only the common control parameters and it is specifically designed based on the concept that should avoid the worst solution and, at the same time, moves towards the optimal solution.The area will continue to attract researcher's interest due to its applicability to meta heuristics approaches.
Motivated by the work of Utsch De Freitas Pinto and Martins Ferreira (2014), Liu & Feng (2010) and Jayswal and Choudhury (2014), we propose a new logarithmic penalty function (LPF) which is designed specifically for nonlinear constrained optimization problem with equality constraints.Moreover, the main advantage of the proposed LPF is associated with the differentiability of the penalty function.At the same time, LPF method is able to handle some problems with irregular features due to its differentiability.
The presentation of this paper is organized as follows: in section 2, notation and preliminary definitions and some lemmas that are essential to prove some result are presented.Section 3 provides the convergence theorems of the proposed LPF.In section 4, the first order necessary optimality condition to derive the KKT multiplier is presented.Section 5 is devoted to numerical test results using the benchmars adopted from Hock and Schittkowski (1981) and finally, in section 6, the conclusions are given in the last to summarize the contribution of the paper.

Preliminary Definitions and Notations
In this section, some useful notations and definitions are presented.Consider the problem (P), where is an objective function with as a decision variable and 0 is the equality constraints with indexes ∈ 1,2, … … , .
Conventionally, a penalty function method substitutes the constrained problem by an unconstrained problem of the form (Bazaraa et al., 2006): where is a positive penalty parameter and is a penalty function satisfying:

and only if 0
The penalty function reflects the feasible points by ensuring that the constraints are not violated.For example, the proposed absolute value penalty function introduced by Zangwill (1967) for equality constraints is as follows, , where Eq. ( 2) is clearly non-differentiable.
Definition 2.1.A function : → is called a penalty function for the problem (P), if satisfies the following: Now, the proposed penalty functions for the problem (P) can be constructed as follows Let , denote the penalized optimization problem, the proposed penalized optimization for the problem (P) can be written in the following form: Definition 2.2.A feasible solution ̅ ∈ is said to be an optimal solution to penalized optimization problem ̅ , if there exist no ∈ such that , ̅ , .
In the following lemma, the feasibility of a solution to the original mathematical programming problems is demonstrated and we determine the limit point of the logarithmic penalty function with respect to the penalty parameter .
Lemma 2.1 where is the set of feasible solutions to the penalized optimization problem, then the following hold for the penalty function.
Suppose that 0 or 0 for some we have ln 1 0.
By sequential unconstrained minimization technique (SUMT) 0 with ( is monotonically increasing). Therefore, In the following lemma, we derive the necessary condition for a point to be a feasible solution of the penalized nonlinear optimization problem, by using the previous lemma.Then by the definition of optimal solutions to the penalized optimization problem (2.4), ∄ ̅ , * , that is Contrary to the result in Eq. ( 5), suppose that * ∉ then by (ii) in lemma 2.1 we can have For any point ̅ ∈ , by (i) in lemma 2.1 it follows that In this way, for a sufficiently large say , we can deduce that which is in contradiction with inequality given in Eq. ( 5).This complete the proof.■

Convergence of The Proposed Logarithmic Penalty Function Method
In this section, the sequence set of feasible solutions of the logarithmic penalized optimization problem convergence to the optimal solution of the original constrained optimization problem shall be proved.
does not hold for .
By using lemma 2.2 and taking the limit as → ∞ in the inequality (7), it follows that ̅ does not hold which is a clear contradiction to inequality (6).
Upon assumption that ∉ .Then by (ii) in lemma 2.1, we have the following Therefore, in inequality ( 14) 0 which is a contradiction to 0. This complete the proof.■

LaGrange Multiplier for The Proposed Logarithmic Penalty Function
In nonlinear optimization problems, the first order necessary conditions for a nonlinear optimization problem to be an optimal is Karush-Kuhn-Tucker (KKT) conditions, or Kuhn-Tucker conditions if and only if any of the constraints qualifications are satisfied.Moreover, for equality constraints only, the multiplier is known as LaGrange multiplier.
Let : → , and assume that is continuously differentiable at , then ln 1 2 1 Theorem 4.1.Let * solves the penalized optimization problem and it satisfies the first-order necessary optimality conditions of the constrained problem (P).Then * is a solution of the penalized problem.
Proof.If * is a feasible point which satisfies the first-order necessary optimality conditions of the problem, then * * 0, that is From Eq. ( 15), we may define * as * 2 * * 1 Then Eq. ( 15) can be rewrite as * ∑ * * 0 where * is a vector of KKT multiplier and * 0 for all ∈ .Note that the penalty parameter is an increasing sequence of constants (i.e. ), as → ∞, → ∞.
The well-known method for solving penalized optimization is sequential unconstraint minimization techniques (SUMT).Some other methods for solving unconstrained optimization problem are also applicable, even though those algorithms available are not specifically designed for this type of the problems.

Numerical tests
In this section, some numerical examples are presented to validate the proposed LPF, the experiments have been implemented to investigate the performance of the proposed method and to gain a perception of the efficiency of the proposed LPF.Hock and Schittkowski's (1981) collection set of continuous problems with equality constraints have been solved using the fminuc function with a quasi-newton algorithm in MATLAB R2018a.The results obtained have been compared with the original constrained optimization problem and that of the penalty function method based on matrix projection.Based on the result shown in Table1, the proposed LPF has been able to deal with those problems with irregular features as observed and reported by Utsch De Freitas Pinto and Martins Ferreira (2014).But for the case of problem HS068, the proposed LPF failed to overcome its irregularity because the solver stopped prematurely at 57 iterations.However, all the test problems have converged to their local minimum.
Comparatively with the result obtained by original constrained optimization and penalized optimization based on the projection matrix, the proposed LPF reduced the number of iterations for the problems that required higher number of iterations in PM approach while it increased the number of iterations for the problems that required lower number of iterations in C and PM approach with an improved objective value.

Conclusion
In this paper, we have proposed a new penalty function method which transformed non-linear constrained optimization with equality constraints into an unconstrained optimization problem.The proposed LPF, which could handle the features of the classical penalty functions and an exact penalty function, which conclusively depends on the penalty parameter.It has been shown that the convergence theorem proved in this paper were validated through some numerical tests.
All the problems tested from Hock-Schittkowski (Hock & Schittkowski, 1981) collections converged to their local minimum including those with irregular features apart from HS068.In the future research, the issues that need to be addressed are; (i) To develop a new algorithm specifically for this type of formulation, (ii) To ensure that the number of iterations required for the problem to converge to a local minimum is reduced, (iii) To solve a multi-objective constrained optimization problem and (iv) To solve the practical problem from any of the following areas;  Engineering  Decision theory  Economics, etc.

Table 1
Comparative results of number of iteration and objective value in respect to C, PM and P

Table 2
Description of the notations used in Table1