A globally convergent QP-free algorithm for nonlinear semidefinite programming

In this paper, we present a QP-free algorithm for nonlinear semidefinite programming. At each iteration, the search direction is yielded by solving two systems of linear equations with the same coefficient matrix; \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$l_{1}$\end{document}l1 penalty function is used as merit function for line search, the step size is determined by Armijo type inexact line search. The global convergence of the proposed algorithm is shown under suitable conditions. Preliminary numerical results are reported.


Introduction
Consider the following nonlinear semidefinite programming (NLSDP for short): where f : R n → R, h j (j ∈ E) : R n → R l and A : R n → S m are continuously differentiable functions, not necessarily convex. S m is a space whose elements are real symmetric matrices of size m × m. denotes the negative semidefinite order, that is, A B if and only if A -B is a negative semidefinite matrix. NLSDP (.) has a broad range of applications such as eigenvalue problems, control problems, optimal structural design, truss design problems (see [-]). So it is desired to develop numerical methods for solving NLSDP (.).
In recent years, NLSDPs have been attracting a great deal of research attention [, -]. As is well known, NLSDP (.) is an extension of nonlinear programming, some efficient numerical methods for the latter are generalized to solve NLSDP. For example, Correa and Ramirez [] proposed an algorithm which used the sequential linear SDP method. Fares et al. [] applied the sequential linear SDP method to robust control problems. Freund et al. [] also studied a sequential SDP method. Kanzow et al. [] presented a successive linearization method with a trust region-type globalization strategy.
In addition, Kovara and Stingl [] developed a computer code PENNON for solving NLSDP (.), where the augmented Lagrangian function method was used. Sun et al. [] and Luo et al. [, ] proposed an augmented Lagrangian method for NLSDP (.), respectively. Sun et al. [] analyzed the rate of local convergence of the augmented Lagrangian method for NLSDPs. Yamashita et al. recently proposed a primal-dual interior point method for NLSDP (.) (see []). The algorithm is globally convergent and locally superlinearly convergent under suitable conditions. Very recently Aroztegui [] proposed a feasible direction interior point algorithm for NLSDP (.) with only semidefinite matrix constraint.
As we know, QP-free (also called SSLE) method is a kind of efficient methods for standard nonlinear programs (see []- []). In this paper, motivated from QP-free method for standard nonlinear programs, based on techniques of perturbation and penalty function, we propose a globally convergent QP-free algorithm for NLSDP (.). The construction of systems of linear equations (SLE for short) is a key point. Based on KKT conditions of NLSDP (.) and techniques of perturbation, we construct two SLEs skillfully. At each iteration, the search direction is yielded by solving two SLEs with the same coefficient matrix; An exact penalty function is used as the merit function for line search and the step size is determined by suitable inexact line search. The global convergence of the proposed algorithm is shown under some mild conditions. The paper is organized as follows. In Section  we restate some definitions and results on NLSDP and matrix analysis. In Section  the algorithm is presented and its feasibility is discussed. The global convergence is analyzed in Section . Some preliminary numerical results are reported in Section  and some concluding remarks are given in the final section.

Preliminaries
For the sake of convenience, some results on matrix analysis and NLSDP are restated in this section, which will be employed in the following analysis of the proposed algorithm. More introduction for theory of matrices should be seen in [] and []. Denote by R m×n the space of m × n real matrices, denote by S m + and S m ++ the sets of m-order symmetric positive semidefinite and positive definite matrices, respectively. The sets S m and S m --are defined similarly.

the inner product of A and B is defined by
where Tr(P) means the trace of the matrix P.
For any matrix U ∈ S m , it is verified that the following equality is true: Note that the linear operator A ⊗ s B is defined implicitly in (.). In Appendix of [] a matrix representation of A ⊗ s B is given as follows: is the Kronecker product of A and B, Q is an orthogonal m × m  matrix (i.e. QQ T = I m ), with the following property: where vec(U) = (u  , u  , . . . , u m , u  , u  , . . . , u m , . . . , u mm ) T .
Remark . One choice for the matrix Q is given in the appendix of [].

Lemma . ([])
For any A, B ∈ S m , the following results are true: Proof Since A ∈ S m ++ , B ∈ S m --, and they commute, there exists an orthogonal matrix P ∈ R m×m such that where D A is a diagonal and positive definite matrix, and D B is a diagonal and negative definite matrix. It follows from Lemma .() that We know from Lemma .(), () that T is orthogonal, from Lemma .() that D A is a diagonal and positive definite matrix, and D B is a diagonal and negative definite matrix. Hence, In the rest of this section we state the first order optimality conditions for NLSDP (.). For the sake of convenience, we first introduce some notations. Given a matrix valued function A(·), we use the notation ∂A(x) ∂x n T for its differential operator evaluated at x, where ∂A(x) ∂x i denotes the partial derivative of A(x) with respect to x i with components ∂a pq (x) x i (p, q = , . . . , m). Then the derivative of A(·) in the direction d = (d  , . . . , d n ) T ∈ R n at x denoted by DA(x)d is defined by then by (.), the following equality is true: The Lagrangian function of NLSDP (.) L : R n × S m × R l → R is defined by where h(x) = (h  (x), h  (x), . . . , h l (x)) T . In view of (.), the above equality can be rewritten as follows: where λ := svec( ). The gradient of L(x, λ, μ) with respect to x is given as follows: where ∇h(x) = (∇h  (x), ∇h  (x), . . . , ∇h l (x)).
We are now in a position to restate the definition of the first order optimality conditions for NLSDP (.).

Definition . ([]
) For x ∈ R n , if there exist a matrix ∈ S m and a vector μ (∈ R l ) such that Remark . According to the Von Neumann-Theobald inequality, the complementarity condition A(x) =  has the following two useful equivalent forms:

The algorithm
In this section, we present our algorithm and show it is well defined. For the sake of simplicity, we introduce some notations: that is, is the feasible set of NLSDP (.).
In general, A(x) is not guaranteed to be symmetric, so we consider sym( A(x)) =  instead of A(x) = . Then the three equalities of KKT condition (.a)-(.c) can be rewritten in the following form: In order to solve (.) at each Newton iteration, we define a vector-value function ϕ : R n+m+l → R n+m+l as follows: It follows from (.) and Lemma . that thus, the Jacobian of ϕ is Instead of the Hessian ∇  xx L(x, λ, μ), we employ a positive definite matrix denoted by H which can be a quasi-Newton approximation or the identity matrix. A Newton-like iteration to solve (.) is given by the linear systems as follows: is the new estimates given by the Newton-like iteration, λ := svec( ) and λ  := svec(  ). Let d  = xx  , we obtain from (.) To obtain a better search direction, we modify (.b) by introducing an appropriate right hand side, so we obtain another linear equations as follows: In order to ensure that SLEs (.a)-(.c) and (.) have a unique solution, respectively, the following assumption is required.
A For any x ∈ F , the matrix is full of column rank. The following lemma gives a sufficient condition of the assumption A.
The proof is elementary and it is omitted here. In our algorithm the following exact penalty function is used as a merit function for line search: where σ >  is a penalty parameter. Further, we define a function P( Now the algorithm is described in detail. Step (.) If d k = , then stop, x k is a KKT point of NLSDP (.); else, go to Step .
Step . Let (d k , λ k , μ k ) be the solution of the SLE (.) in (d, λ, μ), i.e., Step . Compute the search direction d k and the approximate multiplier vector (λ k , μ k ): |}, otherwise. (.) Step . (Update the penalty parameter) Set The updating rule of σ k is as follows: otherwise. (.) Step . (Line search) Set the step size t k to be the first number of the sequence {, β, β  , . . .} satisfying the following two inequalities: Step . Set x k+ = x k + t k d k . Using the following methods to generate k+ commuting with A(x k+ ): Step .. If the search direction d k does not descend or is not feasible, set k+ = I m and go to Step .
Step . Set λ k+ = svec( k+ ), and update H k by some method to H k+ such that H k+ is symmetric positive definite. Let k := k + , return to Step .
By (.), the following lemma is obvious.
Proof First we show that the inequality holds. Premultiplying the first equation of (.) by (d k ) T , we obtain According to the second equation of (.), we get Substituting the above equality and the third equality of (.) into (.), we have In view of Lemma ., the matrix ( k ⊗ s I m ) - (A(x k ) ⊗ s I m ) is negative semidefinite, so it follows from the above equality that i.e., the inequality (.) holds. Next, we will prove the inequality (.) is true. The rest of the proof is divided into three cases.
From the definition (.) of the function P(x k ; d k ; σ k ) and (.c), we have the first inequality above is due to (.). Since x k is not a KKT point of NLSDP (.), it implies from Step  of Algorithm A that d k = , so (d k ) T H k d k > . On the other hand, it follows from the updating rule of σ k that σ k > (ξ ) max j∈E |μ k j |, therefore, (.) gives rise to that is, the inequality (.) holds.

Lemma . Suppose that the assumption A holds. If Algorithm
The second equality above is due to (.). From the convexity of P(x k ; d; σ k ) for d, we obtain which together with (.) and Lemma . gives for t small enough where α ∈ (, ). Hence, (.) holds for sufficiently small t > .
In what follows, we prove (.) holds for sufficiently small t > . Since A(x) is twice continuously differentiable function, it follows from Taylor expansion that (.) Note that the largest eigenvalue function λ max (A) = max v = v T Av, we deduce from (.) and A(x k ) ≺  that for  < t <  small enough, which implies (.) holds for  < t <  small enough. By summarizing the above discussions, we conclude that Algorithm A is well defined.

Global convergence
If Algorithm A terminates at x k after a finite number of iterations, we know from Lemma . that x k is a KKT point of NLSDP (.). In this section, without loss of generality, we assume that the sequence {x k } generated by Algorithm A is infinite. We will prove any accumulation point of {x k } is a stationary point or a KKT point of NLSDP (.), i.e., Algorithm A is globally convergent. We first generalize the definition of stationary point for nonlinear programming defined in [] to nonlinear semidefinite programming.
Definition . Let x ∈ R n , if there exist a matrix (∈ S m ) and a vector μ (∈ R l ) such that where W (x k , H k , k ) is defined by (.) and From the assumptions A-A, we obtain the following conclusions immediately.

Lemma . Suppose the assumptions A-A hold. Then there exists a constant M
Lemma . Suppose the assumptions A-A hold. Then The following result is an important property of the penalty parameter σ k , which is obtained by the updating rule (.).

Lemma . Suppose the assumptions A-A hold. Then the penalty parameter σ k is updated only in a finite number of steps.
Based on Lemma ., in the rest of the paper, we assume, without loss of generality, that σ k ≡σ for all k, wherẽ By using of Lemma ., we obtain the following result.

Lemma . Suppose the assumptions A-A hold. Then there exists a constant c  >  such that
For the sake of simplicity, in the rest of this section, let (d *  , μ *  , λ *  ) be the solution of the following SLE in (d, μ, λ): Let (d *  , μ *  , λ *  ) be the solution of the following SLE in (d, μ, λ): From the above equalities and Lemma ., we obtain the following conclusion.
Based on Lemma ., the following conclusion is obvious.

Lemma . Suppose the assumptions A-A hold. Let x k
KKT point or a stationary point of NLSDP (.).
Lemma . Suppose the assumptions A-A hold, Proof By contradiction, we assume that there exist a subset K ⊂ K and a constantd >  such that d k ≥d, ∀k (∈ K ) large enough. From the assumptions A-A, (.) and the updating rule of k , we assume without loss of generality that H k On the other hand, it follows from the updating rule of k and the assumption A that * is positive definite. According to Lemma .(iii), there exists d >  such that d k ≥ d for all k ∈ K .
Firstly, we show that there exists t >  independent of k such that (.) and (.) are satisfied for all t ≥ t. For any k ∈ K , it is clear from the assumptions A and A and Lemmas .-. and Lemmas .-. that P x k ; d k ;σ -P x k ; ;σ ≤ -ξ ad  .
(  .  ) Together with (.)-(.), there exists t f >  independent of k such that for all k ∈ K and t ∈ (, t f ], where α ∈ (, ). The above inequality shows the inequality (.) holds. We next prove the inequality (.) holds. It follows from (.) and Lemma .() and Lemma . that Combining with Lemmas .-. and (.), there exists a constant  <δ ≤  such that δ k ≥δ for k ∈ K . By the mean-value theorem and Lemmas .-., we obtain In view of (.), (.) and Lemma .(), we obtain Hence, (.), (.) and (.b) give rise to Based on the above equality, we have note the positive definiteness of k , hence, if then (.) holds for t ≤δ d M  . Since k and A(x k ) are symmetric and commuting, there exists an orthogonal matrix Q k such that for any t ∈ (, t A ) and k ∈ K . By Lemma . and k = smat(λ k ), we know { k } is bounded, furthermore, { k } is also bounded. Let * be an accumulation point of { k }. Without loss of generality, we assume that k K − → * . Let B k = k - * , obviously, B k K − → , thus there It follows from the assumption A that all eigenvalues of D k λ are between λ I and λ s for all k. According to Weyl's theorem (see []), there exists t  >  such that all eigenvalues of (D k λt * ) are positive for any t ∈ (, t  ]. We also know from A(x k ) ≺  and the second equality in (.) that D A is negative definite. Therefore, for any v with v =  and t ∈ (, t  ], it follows from Lemma . that (D k λt * )D k A is also negative definite. Combining with (.), for any v with v =  and any t ∈ (, t  ), we obtain together with (.) shows that (.) is satisfied, further, (.) and (.) hold. Let t A = min{t  , md M  }, thus (.) holds for any t ∈ (, t A ]. Hence, we see that A(x k + td k ) ≺  holds for t ∈ (, t A ] and any k ∈ K . Lett = min{t f , t A }, for any t ∈ (,t], (.) and (.) are satisfied for all t ≥ t. Combining with (.) and (.), we obtain for any k ∈ K P x k+ ;σ ≤ P x k ;σtαξ ad  .
Based on Lemmas .-., the following global convergence of Algorithm A is immediate.

Numerical experiments
Algorithm A has been implemented in Matlab b and the codes have been run on a . GHz Intel(R) Core(TM)i- machine with a Windows  system. We choose H  as n-order identical matrix and at each iteration, H k is updated by the damped BFGS formula in [] and  as m-order identical matrix. In the numerical experiments, we choose the parameters as follows: The stop criterion is d k ≤  - . The test problems are described as follows: I. The first test problem is Rosen-Suzuki problem [] combined with a negative semidefinite constraint and denoted by CM: II. We select some test problems from [] only with equality constraints and we add a negative semidefinite matrix constraint.
() We select the problems HS, HS, HS, HS combined with the following  ×  order symmetric matrix which comes from [] and rename them MHS, MHS, MHS and MHS, respectively: () Choose the problems HS, HS, HS and HS combined with the following  ×  order symmetric matrix and rename them MHS, MHS, MHS and MHS, respectively:  The numerical results are listed in Table  and Table . The meanings of the notations in Table  and Table 

Concluding remarks
We have presented a globally convergent QP-free algorithm for nonlinear SDP problems. Based on KKT conditions of nonlinear SDP problems and techniques of perturbation, we construct two SLEs skillfully. Under some linear independence condition, the SLEs have unique solution. At each iteration, the search direction is yielded by solving two SLEs with the same coefficient matrix; some penalty function is used as the merit function for line search and the penalty parameter is updated automatically in the algorithm. The preliminary numerical results show that the proposed algorithm is effective and comparable.