Analysis of Interior-Point Paths

Infeasible-interior-point paths are the main tools in interior-point methods for solving many kinds of optimization problems. These paths are usually parametrized by a penalty-parameter r ↓ 0 and further parameters describing their off-centrality and infeasiblilty. Starting with an early result of C. Witzgall et al. [12] in linear programming, this paper gives an overview on results concerning the existence of these paths, their analyticity and the limiting behavior of their derivatives as r ↓ 0, and this also for degenerate problems in the areas of linear programming, linear complementarity problems, and semi-definte programming.

Infeasible-interior-point paths are the main tools in interior-point methods for solving many kinds of optimization problems. These paths are usually parametrized by a penalty-parameter r ↓ 0 and further parameters describing their off-centrality and infeasiblilty. Starting with an early result of C. Witzgall et al. [12] in linear programming, this paper gives an overview on results concerning the existence of these paths, their analyticity and the limiting behavior of their derivatives as r ↓ 0, and this also for degenerate problems in the areas of linear programming, linear complementarity problems, and semidefinte programming.
Key words: infeasible interior-point paths; linear complementarity problems; linear programs; semidefinite linear complementarity problems; semidefinite linear programs. ∑ perturbed optimization problems that shrink to the unperturbed problem (1) as the path parameter r tends to zero. Again this poses the question of the existence of such paths and their limiting behavior, also for degenerate unperturbed problems. It is the purpose of this paper to give an overview over some results that are relevant in this context.
In a joint paper [12] with P. T. Boggs and P. D. Domich, C. Witzgall was one of the first to deal also with degenerate problems in the context of linear programs. Their pioneering results will be outlined in Sec. 2. Similar and even sharper results have been found for much more general classes of programs. We describe some of them that refer to linear complementarity problems in Sec. 3. Still more difficult are semidefinite linear programs and semidefinite linear complementarity problems. Results for them are described in the final Sec. 4.

Linear Programs
Consider the dual linear programs where A = (a 1 , ..., a n ) is a real m × n-matrix of rank m with columns a i . Let S P and S D be the optimal solution sets of (P) and (D).
Consider for (D) the Frisch logarithmic penalty function and its minimum u(r) for r > 0, The path {u(r)} r>0 , if it exists, is called the dual central path; then u = u(r) solves the equation (2) A central result of [12] adapted to (P) and (D) is is strictly feasible, (c) S D is nonempty and bounded. Then the path u(r) is well-defined for r > 0 and the following limits exist: The limit point u * is the analytic center of S D , and the function u(r) is analytic in r for r > 0. Let Then (x, y, u)(r), r > 0, the primal-dual central path, solves where e is the vector e : = (1, 1, ..., 1) T ∈ R n , and x°y is the componentwise product x°y : = (x 1 y 1 , ..., x n y n ) T .
By Eq. 2 the primal-dual path exists in the situation of Theorem 1. In general, the primal-dual path is welldefined for r > 0, provided (P) and (D) are strictly feasible; primal-dual interior point methods try to follow this path for r ↓ 0. But there are some difficulties with this concept: -(PD) r is a nonlinear equation for each r, so one can at best approximate points on it. -The path does not exist, if (P) and (D) are not strictly feasible, even when they are solvable.
The practical significance of these paths lies in their use in infeasible-interior-point methods (see e.g. Mizuno [10] for an early analysis of such methods in the context of linear programming). Typically, these methods start with an initial point on the initial trajectory at (r 0 , η 0 ) defined by the initialization. Then these methods construct further iterates where the r k > r k+1 > · · · > 0 converge to 0 and the weights η k ≈ e stay bounded and close to e. The construction of z k+1 from z k use, as a rule, the tangent of z(., η k ) at r k , that is the derivative z′ k : = ∂z(r, η k )/∂r| r=r k (usually called the affine scaling direction). The quality of these methods depends on how quickly the r k converge to 0. With most methods, the superlinear convergence of the r k relies on the existence (and boundedness) of the derivatives z′ k = z′ (r k , η k ) as r k ↓ 0.
Also with the paths z(r, η) the same questions arise as before: -their existence and uniqueness for 0 < r ≤ r 0 = 1, η > 0, -their analytical properties, -and their limiting properties for r ↓ 0.
We will answer these questions in the context of more general complementarity problems in Sec. 3. Here, we only formulate linear programs as particular complementarity problems, which is well-known.
By the duality theorem of linear programming, feasible solutions x, (y, u) of (P) and (D) satisfy and they are optimal solutions of (P) and (D), if they are feasible and x T y = 0.
By elimination of u, define the linear manifold M ⊂ R 2n , and the associated linear subspace Then (x, y) ∈ Φ ⇒ x T y = 0, and by the assumption rank (A) = m, The linear programs (P) and (D) are then equivalent to the associated complementarity problem of finding a solution (x, y) of Under the assumptions of Theorem 1, this particular complementarity problem satisfies

The set of solutions of (LCP) is denoted by
The LCPs associated with linear programs are monotone; by Eq. 3 they even satisfy x T y = 0 for all (x, y) ∈ Φ. For the presentation of the following results, we make the following rather weak standard assumption 1) rank [P,Q] = n, i.e. dimΦ = n, Note that (V) 3) does not require the unique solvability of (LCP). Degenerate problems with more than one solution are not excluded.
Still more general than monotone complementarity problems are sufficient linear complementarity problems. They were introduced by Cottle, Pang, and Venkateswaran in [2] and further characterized by Väliaho [11]. They are the most general class of linear complementarity problems of the form (LCP) with the following properties: 1. For all q, the solution set S = S(q) of (LCP) is a convex (perhaps empty) set.

Each stationary solution of the (nonconvex) quadratic optimization problem
inf{x T y | (x, y) is a feasible point of (LCP)} associated with (LCP) is a solution of (LCP).
Interior-point methods for solving (LCP) try to follow such paths for r ↓ 0 in much the same way as described for linear programs in Sec. 2.
As before, the same questions with respect to these paths (x, y)(r, η) arise, -their existence and uniqueness for 0 < r ≤ r 0 = 1, η > 0, -their analytical properties as functions of r, η, -and their limiting properties for r ↓ 0.

Remarks
1. By a result of Goldman and Tucker [6], any solvable complementarity problem associated with linear programs has also a strictly complementary solution. So, part b) of the theorem applies in this case, which generalizes Theorem 1: the limits of all derivatives exist, and are in fact the derivatives of an analytic function ẑ(r, η) at r = 0 extending z(r, η). 2. It is easy to find convex quadratic programs for which the corresponding complementarity problem is only solvable but has no strictly complementary solution, so case c) of the theorem can occur. 3. Theorem 2 remains true for sufficient complementarity problems (i.e. if "monotone" is replace by "sufficient" in Assumption (V) (see [8]). The proof of Theorem 2 is not easy in the case of degeneracies, when (LCP) has more than one solution or no strictly complementary solution. The mathematical reason is that the solutions (x, y, r, η) of (LCP) r,η with x ≥ 0, y ≥ 0, r ≥ 0 and η > 0 then form a semialgebraic variety with a singularity in r = 0, so that the implicit function theorem cannot be applied directly (though indirectly, see [9], [8] for details).

Semidefinite Linear Programs and Complementarity Problems
In this section we show that the results of Theorems 1 and 2 even generalize to semidefinite linear programs and to semidefinite linear complementarity problems. We use the following notation, which is standard: S n is provided with the scalar product 〈A,B〉 : = tr (B T A) = tr (BA).
A 0 (resp. A 0) means that A ∈ S n is positive semidefinite (resp. positive definite), and S n ++ is the set of all positive definite matrices in S n .
Then the standard dual pair of semidefinite linear programs has the form Here, The following results are known from the duality theory of semidefinite linear programs: If (P) and (D) have strictly feasible solutions then they also have optimal solutions, and X and (u, Y) are optimal for (P) and (D), respectively, iff By elimination of u one can formulate this as a particular semidefinite linear complementarity problem, which in general has the following form: Solve Here, q ∈ R¯n, n : = n(n + 1)/2 = dimS n , and P, Q: S n → R¯n are linear operators of the form As in the previous section we define:

The nullspace N ([P,Q]) of [P,Q] is denoted by
3. The set F of feasible points of (SDLCP) is defined by 4. The set of solutions of SDLCP is denoted by In the case of the dual semidefinite linear programs (P), (D), Φ is given by The following standard hypothesis, which corresponds to the standard assumption of the previous section, is very weak The definition of infeasible weighted interior-point paths differs slightly from the previous section: For 0 < r ≤ r 0 and weight matrices M 0, infeasible weighted interior point paths are now defined as solutions (X, Here, the condition XY = 0 of (SDLCP) is weakened to (XY + YX)/2 = rM, which corresponds to the "AHO" approach proposed by Alizadeh, Haeberly and Overton in [1].
The perturbation vectorq is chosen such that q ∈ M r 0 for some r 0 > 0, where A possible initialization is to choose r 0 : = 1, X 0 = Y 0 : = I and M 0 : = I and q : = P(X 0 ) + Q(Y 0 ) -q.
Again the same questions arise for these paths (X, Y)(r, M) as in the previous sections, namely their -existence and uniqueness for 0 < r ≤ r 0 , M 0, -analytical properties as functions of r and M, -and their limiting properties for r ↓ 0.
The following is the main result on (SDLCP). It corresponds to Theorems 1 and 2, and shows the fruitfulness of the early results of C. Witzgall: Theorem 3 Assume (V), r 0 > 0 and q ∈ M r 0 . Then: 1. The path function (X,Y)(r, M) is a well-defined analytic function for all (r, M) ∈ (0, r 0 ] × S n ++ , which converges to a solution of (SDLCP) as r ↓ 0, 2. If (SDLCP) even has a strictly complementary solution, then there is an ε > 0 so that the path function (X, Y)(r, M) can be extended to an analytic function ( X , Ŷ )(r, M) defined on the enlarged domain (r, M) ∈ (-ε, r 0 ] × S n ++ . "(X, Y)(r, M) is analytic in (r, M) even at r = 0." The proof of this theorem is even more involved than that of Theorem 2, it can be found in Preiss and Stoer [7].
It is an open problem, whether (and how) the path function (X, Y)(r, M) can be extended, if (SDLCP) satisfies only (V), but has no strictly complementary solutions.