IPRQP: a primal-dual interior-point relaxation algorithm for convex quadratic programming

Abstract

We propose IPRQP, an enhanced primal-dual interior-point relaxation method (IPRM), for solving convex quadratic programming. This method is based on a smoothing barrier augmented Lagrangian function for convex quadratic programming. IPRQP inherits the advantages of IPRM, including not requiring iterative points to be interior points, which makes IPRQP suitable for the warm-starting of combinatorial optimization problems. Compared to IPRM, the customized starting points allow the line search of IPRQP to contain only vector operations. In addition, IPRQP improves the updating scheme of the barrier parameter and provides a certificate of infeasibility. Some results on global convergence are presented. We implement the algorithm on convex quadratic programming problems from Maros-Mészaros and the benchmark problem sets NETLIB and Kennington, which contain feasible and infeasible linear programming problems. The numerical results show that our algorithm is reliable for feasible problems and efficient for detecting the infeasibility of infeasible problems.

Appendix: some proofs

Proof of Lemma 2.2

Since $(y_l)_i(z_l)_i=\dfrac{\mu }{\rho }$ and $(z_l)_i-(y_l)_i=x_i-l_i-\dfrac{(s_l)_i}{\rho }$, we can conclude that

$$\begin{aligned} (y_l)_i\nabla _{x} (z_l)_i+ (z_l)_i\nabla _{x} (y_l)_i=0 \text {\quad and \quad } \nabla _{x} (z_l)_i-\nabla _{x} (y_l)_i=e_i. \end{aligned}$$

Accordingly, the relations in the first line of (13) immediately follow. The remaining relations in (13) and those in (14) can be confirmed in the same manner.

Differentiating the equations $(y_l)_i(z_l)_i = \dfrac{\mu }{\rho }$ and $(z_l)_i-(y_l)_i=x_i-l_i-\dfrac{(s_l)_i}{\rho }$ on $\mu$ gives us

$$\begin{aligned} (y_l)_i\dfrac{\partial (z_l)_i}{\partial \mu }+(z_l)_i\dfrac{\partial (y_l)_i}{\partial \mu }=\dfrac{1}{\rho } \text { and } \dfrac{\partial (z_l)_i}{\partial \mu } - \dfrac{\partial (y_l)_i}{\partial \mu } = 0, \end{aligned}$$

thereby demonstrating that

$$\begin{aligned} \dfrac{\partial (z_l)_i}{\partial \mu }=\dfrac{\partial (y_l)_i}{\partial \mu }=\dfrac{1}{\rho ((z_l)_i+(y_l)_i)}. \end{aligned}$$

Similarly, we can deduce

$$\begin{aligned} \dfrac{\partial (z_u)_i}{\partial \mu }=\dfrac{\partial (y_u)_i}{\partial \mu }= \dfrac{1}{\rho ((z_u)_i+(y_u)_i)}. \end{aligned}$$

$\square$

Proof of Theorem 2.1

To simplify notation, we abbreviate $F(x,s_l,s_u;\mu ,\rho )$ as $F(x,s_l,s_u)$. It is differentiable with respect to x, and

$$\begin{aligned} \begin{array}{rl} &{} \!\!\!\!\!\!\!\!\!\!\!\!\! \nabla _x F(x,s_l,s_u) \\ =&{}c+Qx-\mu Z_l^{-1}\nabla _x z_l-\mu Z_u^{-1}\nabla _x z_u+s_l{\circ }(\nabla _x z_l-e)+s_u{\circ }(\nabla _x z_u+e)\\ {} &{}+\rho (z_l-x+l){\circ } (\nabla _x z_l-e)+\rho (z_u-u+x){\circ } (\nabla _x z_u+e)\\ =&{}c+Qx-\mu Z_l^{-1}\nabla _x z_l-\mu Z_u^{-1}\nabla _x z_u+\rho y_l {\circ } (\nabla _x z_l-e)+\rho y_u {\circ }(\nabla _x z_u+e)\\ =&{}c+Qx-\mu Z_l^{-1}\nabla _x z_l-\mu Z_u^{-1}\nabla _x z_u+\rho y_l {\circ } \nabla _x y_l+\rho y_u {\circ }\nabla _x y_u\\ =&{}c+Qx+(Z_l+Y_l)^{-1}(-\mu e-\rho y_l{\circ } y_l)+(Z_u+Y_u)^{-1}(\mu e+\rho y_u{\circ } y_u)\\ =&{}c+Qx-\rho y_l+\rho y_u, \end{array} \end{aligned}$$

where $Z_l=\text {diag}{(z_l)}$, $Y_l=\text {diag}{(y_l)}$, $Z_u=\text {diag}{(z_u)}$ and $Y_u=\text {diag}{(y_u)}$. The second equation follows from $\rho y_l = s_l+\rho (z_l-x+l)$ and $\rho y_u = s_u+\rho (z_u-u+x)$, whereas the last equation holds as a result of $\mu +\rho (y_l)_i^2=\rho (y_l)_i((z_l)_i+(y_l)_i)$ and $\mu +\rho (y_u)_i^2=\rho (y_u)_i((z_u)_i+(y_u)_i)$ for $i = 1,\dots ,n$.

Thus, we may conclude that

$$\begin{aligned} \nabla _x^2 F(x,s_l,s_u)=Q+\rho (Y_l+Z_l)^{-1}Y_l+\rho (Y_u+Z_u)^{-1}Y_u \succ 0, \end{aligned}$$

which implies that $F(x,s_l,s_u)$ is a strongly convex function with respect to the variable x.

$F(x,s_l,s_u)$ is also differentiable with respect to $s_l$ and $s_u$, and

$$\begin{aligned} \begin{array}{rl} \nabla _{s_l} F(x,s_l,s_u)=&{}-\mu (Z_l)^{-1}\nabla _{s_l} z_l+(z_l-x+l)+s_l{\circ }\nabla _{s_l} z_l+\rho (z_l-x+l){\circ }\nabla _{s_l} z_l\\ =&{}-\mu (Z_l)^{-1}\nabla _{s_l} z_l+(z_l-x+l)+\rho y_l{\circ }\nabla _{s_l} z_l\\ =&{}z_l-x+l,\\ \nabla _{s_u} F(x,s_l,s_u)=&{}z_u-u+x. \end{array} \end{aligned}$$

Thus, we have

$$\begin{aligned} \begin{array}{l} \nabla _{s_l}^2 F(x,s_l,s_u)=\nabla _{s_l} z_l = -\dfrac{1}{\rho }(Z_l+Y_l)^{-1}Z_l \prec 0,\\ \nabla _{s_u}^2 F(x,s_l,s_u)=\nabla _{s_u} z_u = -\dfrac{1}{\rho }(Z_u+Y_u)^{-1}Z_u \prec 0. \end{array} \end{aligned}$$

Therefore, $F(x,s_l,s_u)$ is a strongly concave function with respect to the variables $s_l$ and $s_u$. $\square$

Proof of Theorem 2.3

Since (17a) and (17b) are the primal and dual feasible conditions, respectively, we only need to consider the equivalence of the remaining formulas and the complementary slackness conditions.

If $(x^*,\lambda ^*,s_l^*,s_u^*)$ is a solution of (17), notice that for $i = 1,\dots ,n$,

$$\begin{aligned} (z_l)_i(x_i,(s_l)_i;0,\rho )&= \frac{1}{2\rho }(|(s_l)_i-\rho (x_i-l_i)|-((s_l)_i-\rho (x_i-l_i)))\\ {}&=\max \{0,x_i-l_i-\frac{(s_l)_i}{\rho }\}=x_i-l_i,\\ (z_u)_i(x_i,(s_u)_i;0,\rho )&= \frac{1}{2\rho }(|(s_u)_i-\rho (u_i-x_i)|-((s_u)_i-\rho (u_i-x_i)))\\ {}&=\max \{0,u_i-x_i-\frac{(s_u)_i}{\rho }\}=u_i-x_i. \end{aligned}$$

Then for any $i = 1,\dots ,n$, the equality $(z_l^*)_i=x_i^*-l_i$ implies that one has either $x_i^*=l_i,(s_l^*)_i=\rho (y_l^*)_i \ge 0$, or $x_i^*\ge l_i,(s_l^*)_i = \rho (x^*_i-l_i-(z_l^*)_i)=0$. The equality $(z_u^*)_i=u_i-x_i^*$ implies that one has either $x_i^*=u_i,(s_u^*)_i=\rho (y_u^*)_i\ge 0$, or $x_i^*\le u_i,(s_u^*)_i=\rho (u_i-x^*_i-(z_u^*)_i) = 0$. Thus, all $(x^*, \lambda ^*, s_l^*,s_u^*)$ satisfying (17) represent optimal solutions of the original problem (1).

If $(x^*,\lambda ^*,s_l^*,s_u^*)$ is an optimal solution of the original problem (1), then for each $i = 1,\dots ,n$, $x_i^* = l_i,(s_l^*)_i\ge 0$ or $x_i^*\ge l_i, (s_l^*)_i= 0$. It is easy to verify that $z_l(x^*,s_l^*;0,\rho )-x^*+l=0$ in both cases. In the same way, we have $z_u(x^*,s_u^*;0,\rho )-u+x^*=0$. Therefore, $(x^*,\lambda ^*,s_l^*,s_u^*)$ is also a solution to (17). $\square$

Proof of Lemma 3.2

The directional derivative of $\phi _{(\mu ^{(k)},\rho ^{(k)})}(w)$ along $(\Delta \mu ^{(k)},\Delta w^{(k)})$ at $(\mu ^{(k)},w^{(k)})$ is

$$\begin{aligned} \phi ^{'}_{(\mu ^{(k)},\rho ^{(k)})}(w^{(k)};\Delta \mu ^{(k)},\Delta w^{(k)}) =-2\phi _{(\mu ^{(k)},\rho ^{(k)})}(w^{(k)}). \end{aligned}$$

The Taylor’s expansion of ${\phi }_{(\mu ^{(k)}+\alpha \Delta \mu ^{(k)},\rho ^{(k)})}(w^{(k)}+\alpha \Delta w^{(k)})$ with respect to $\alpha$ at $\alpha = 0$ shows that

$$\begin{aligned}&\!\!\!\!\!\!\!\!{\phi }_{(\mu ^{(k)}+\alpha \Delta \mu ^{(k)},\rho ^{(k)})}(w^{(k)}+\alpha \Delta w^{(k)})-\phi _{(\mu ^{(k)},\rho ^{(k)})}(w^{(k)})\\&\quad = \alpha \phi ^{'}_{(\mu ^{(k)},\rho ^{(k)})}(w^{(k)};\Delta \mu ^{(k)},\Delta w^{(k)})+o(\alpha )\\&\quad = -2\tau \alpha \phi _{(\mu ^{(k)},\rho ^{(k)})}(w^{(k)})-2(1-\tau )\alpha \phi _{(\mu ^{(k)},\rho ^{(k)})}(w^{(k)})+o(\alpha ). \end{aligned}$$

Thus, (25) holds for all sufficiently small $\alpha > 0$, since $\tau < 1$ and $\phi _{(\mu ^{(k)},\rho ^{(k)})}(w^{(k)})>0$. $\square$

Proof of Lemma 3.3

By differentiating the equations $(y_l)_i(z_l)_i = \dfrac{\mu }{\rho }$ and $(z_l)_i-(y_l)_i=x_i-l_i-\dfrac{(s_l)_i}{\rho }$ on $\rho$, we obtain

$$\begin{aligned} \begin{array}{l} (y_l)_i\dfrac{\partial (z_l)_i}{\partial \rho }+(z_l)_i\dfrac{\partial (y_l)_i}{\partial \rho }=-\dfrac{\mu }{\rho ^2}=-\dfrac{(y_l)_i(z_l)_i}{\rho },\\ \dfrac{\partial (z_l)_i}{\partial \rho } - \dfrac{\partial (y_l)_i}{\partial \rho } = \dfrac{(s_l)_i}{\rho ^2}=\dfrac{x_i-l_i-((z_l)_i-(y_l)_i)}{\rho }. \end{array}\end{aligned}$$

One immediately has $\dfrac{\partial (z_l)_i}{\partial \rho }=\dfrac{(z_l)_i}{\rho }\dfrac{(x_i-l_i-(z_l)_i)}{(y_l)_i+(z_l)_i}$ and

$$\begin{aligned} \dfrac{\partial ((z_l)_i-x_i+l_i)^2}{\partial \rho }=2((z_l)_i-x_i+l_i)\dfrac{\partial (z_l)_i}{\partial \rho }= -\dfrac{2}{\rho }\dfrac{(z_l)_i}{(y_l)_i+(z_l)_i}((z_l)_i-x_i+l_i)^2. \end{aligned}$$

We can similarly prove $\dfrac{\partial (z_u)_i}{\partial \rho }=\dfrac{(z_u)_i}{\rho }\dfrac{(u_i-x_i-(z_u)_i)}{(y_u)_i+(z_u)_i}$ and

$$\begin{aligned} \dfrac{\partial ((z_u)_i-u_i+x_i)^2}{\partial \rho }=-\dfrac{2}{\rho }\dfrac{(z_u)_i}{(y_u)_i+(z_u)_i}((z_u)_i-u_i+x_i)^2. \end{aligned}$$

$\square$

Proof of Lemma 4.1

By Lemma 3.4, we know that $\phi _{(\mu ^{(k)},\rho ^{(k)})}(w^{(k)})\le \phi _{(\mu ^{(0)},\rho ^{(0)})}(w^{(0)})$ for all $k > 0$. According to the definition of the merit function, we have

$$\begin{aligned}\begin{array}{l} \frac{1}{2}\Vert z^{(k)}_l-x^{(k)}+l\Vert ^2\le \phi _{(\mu ^{(0)},\rho ^{(0)})}(w^{(0)}),\\ \frac{1}{2}\Vert z^{(k)}_u-u+x^{(k)}\Vert ^2\le \phi _{(\mu ^{(0)},\rho ^{(0)})}(w^{(0)}), \end{array} \end{aligned}$$

which together with Assumption 4.2 implies that $\{z^{(k)}_l\}$ and $\{z^{(k)}_u\}$ are bounded.

By the definition of $\{y^{(k)}_l\}$, we have

$$\begin{aligned} (y_l^{(k)})_i&= \dfrac{\sqrt{((s_l^{(k)})_i-\rho ^{(k)}(x_i^{(k)}-l_i))^2+4\rho ^{(k)}\mu ^{(k)}}+((s_l)_i^{(k)}-\rho (x_i^{(k)}-l_i))}{2\rho ^{(k)}}\\&\quad \le \dfrac{{|((s_l^{(k)})_i-\rho (x_i^{(k)}-l_i))|+\sqrt{\rho ^{(k)}\mu ^{(k)}}}}{\rho ^{(k)}}\\&\quad \le \dfrac{|(s_l^{(k)})_i|}{\rho ^{(k)}}+|x_i^{(k)}-l_i|+\sqrt{\dfrac{\mu ^{(0)}}{\rho ^{(0)}}}\\&\quad \le \max \{\Vert x^{(k)}\Vert ,1\}/\sigma +|x_i^{(k)}-l_i|+\sqrt{\dfrac{\mu ^{(0)}}{\rho ^{(0)}}}. \end{aligned}$$

Thus, the sequence $\{y_l^{(k)}\}$ is bounded under Assumption 4.2. The boundedness of $\{y_u^{(k)}\}$ can be proved in the same way.

The relations $\rho ^{(k)}(y_l^{(k)})_{i}(z_l^{(k)})_{i}=\mu ^{(k)}$ and $\rho ^{(k)}(y_u^{(k)})_{i}(z_u^{(k)})_{i}=\mu ^{(k)}$, together with the fact that all $\{y_l^{(k)}\}, \{z_l^{(k)}\}, \{y_u^{(k)}\}$, and $\{z_u^{(k)}\}$ are bounded, imply the desired inequalities. $\square$