Trust-region problems with linear inequality constraints: exact SDP relaxation, global optimality and robust optimization

Abstract

The trust-region problem, which minimizes a nonconvex quadratic function over a ball, is a key subproblem in trust-region methods for solving nonlinear optimization problems. It enjoys many attractive properties such as an exact semi-definite linear programming relaxation (SDP-relaxation) and strong duality. Unfortunately, such properties do not, in general, hold for an extended trust-region problem having extra linear constraints. This paper shows that two useful and powerful features of the classical trust-region problem continue to hold for an extended trust-region problem with linear inequality constraints under a new dimension condition. First, we establish that the class of extended trust-region problems has an exact SDP-relaxation, which holds without the Slater constraint qualification. This is achieved by proving that a system of quadratic and affine functions involved in the model satisfies a range-convexity whenever the dimension condition is fulfilled. Second, we show that the dimension condition together with the Slater condition ensures that a set of combined first and second-order Lagrange multiplier conditions is necessary and sufficient for global optimality of the extended trust-region problem and consequently for strong duality. Through simple examples we also provide an insightful account of our development from SDP-relaxation to strong duality. Finally, we show that the dimension condition is easily satisfied for the extended trust-region model that arises from the reformulation of a robust least squares problem (LSP) as well as a robust second order cone programming model problem (SOCP) as an equivalent semi-definite linear programming problem. This leads us to conclude that, under mild assumptions, solving a robust LSP or SOCP under matrix-norm uncertainty or polyhedral uncertainty is equivalent to solving a semi-definite linear programming problem and so, their solutions can be validated in polynomial time.

Appendix: Technical results

For the sake of self-containment, in this Section, we provide known technical results on hidden convexity of quadratic systems, S-lemma and tractable classes of robust optimization.

1.1 Hidden convexity of quadratic systems

The basic and probably the most useful result on the joint-range convexity of homogeneous quadratic functions, known as Dine’s Theorem [11], states as follows:

Lemma 7.1

(Dine’s Theorem) [11] Let $A_1, A_2\in S^n$. Then, the set $\{(x^TA_1x,x^TA_2x)\!:\!x \in \mathbb{R }^n\}$ is convex.

Dine’s theorem is known to fail for three homogeneous in general. Polyak [25] established the following joint-range convexity result for three homogeneous quadratic functions under a positive definite condition on the matrices involved.

Lemma 7.2

(Polyak’s Lemma [25, Theorem 2.1]) Let $n \ge 2$ and let $A_1, A_2,A_3 \in S^n$. Suppose that there exist $\gamma _1,\gamma _2,\gamma _3 \in \mathbb{R }$ such that $\gamma _1 A_1+\gamma _2 A_2+\gamma _3A_3 \succ 0.$ Then the set $\{(x^TA_1x,x^TA_2x,x^TA_3x):x \in \mathbb{R }^n\}$ is convex.

1.2 S-lemma and approximate S-lemma

Using Dine’s Theorem, Yakubovich (cf [24]) obtained the following fundamental ${S}$-lemma which has played a key role in many areas of control and optimization.

Lemma 7.3

(${S}$-lemma [24]) Let $A_1,A_2 \in S^{n},\,a_1,a_2 \in \mathbb{R }^n$ and $\alpha _1,\alpha _2 \in \mathbb{R }$. Suppose that there exists $x_0 \in \mathbb{R }^n$ such that $x_0^TA_2x_0+a_2^Tx_0+\alpha _2<0$. Then the following statements are equivalent:

• (i) $x^TA_2x+a_2^Tx+\alpha _2 \le 0 \Rightarrow x^TA_1x+a_1^Tx+\alpha _1 \ge 0$ ;

• (ii) $(\exists \lambda \ge 0) (\forall x \in \mathbb{R }^n) (x^TA_1x+a_1^Tx+\alpha _1 )+\lambda (x^TA_2x+a_2^Tx+\alpha _2 ) \ge 0.$

For a homogeneous quadratic system with multiple convex quadratic constraints, Ben-Tal et al. [7] derived the following approximate S-lemma which provides an estimate between an associated quadratic optimization problem and its SDP relaxation.

Lemma 7.4

(Approximate S-lemma [7, Lemma A.6]) Let $R,H_0,H_1,\ldots ,H_K$ be symmetric $(p \times p)$ matrices such that $H_i \succeq 0,\,i=1,\ldots ,K$ and $\sum _{k=0}^K\lambda _iH_i \succ 0,$ for some $\lambda _i \ge 0, i=0,\ldots ,K$. Consider the following quadratically constrained quadratic problem

$$\begin{aligned} ({ QCQ}) \quad \max \limits _{y \in \mathbb{R }^p}\{y^TRy: y^TH_0y \le 1, y^TH_iy \le 1,\quad i=1,\ldots ,K\} \end{aligned}$$

and the semidefinite optimization problem

$$\begin{aligned} ({ SDP}) \quad \min \limits _{\mu _0,\ldots ,\mu _K \ge 0}\left\{ \sum _{i=0}^K \mu _i : \sum _{i=0}^K\mu _kH_k \succeq R\right\} . \end{aligned}$$

Then, $\max ({ QCQ}) \le \min ({ SDP}) \le \rho ^2 \max ({ QCQ})$ where $\rho =\sqrt{2\log (6 \sum _{i=1}^K\mathrm{rank}H_k)}.$

1.3 Tractable classes of robust optimization problems

The following tractable classes of robust optimization problems are known.

1.3.1 Robust least squares problems [3, 13]

Consider the following robust least squares programming problem:

$$\begin{aligned} ({ RLSP}) \quad \displaystyle \min \limits _{x \in \mathbb{R }^n}\max \limits _{(A,a) \in \mathcal{U }}&\Vert Ax-a\Vert ^2, \end{aligned}$$

where $\mathcal{U }\subseteq \mathbb{R }^{k \times n} \times \mathbb{R }^{k}= \mathbb{R }^{k \times (n+1)}$, is an uncertainty set. Then, (RLSP) can be equivalently rewritten as a semidefinite programming problem under the following two cases:

• (i) $\mathcal{U }$ is an ellipsoid (see [13]), i.e., $\mathcal{U }=\{(A^{(0)},a^{(0)}) + \Delta : \Delta \in \mathbb{R }^{k \times (n+1)}, \Vert \Delta -\overline{\Delta }\Vert _F \le \rho \}$;

• (ii) $k \ge 2$ and $\mathcal{U }$ is the intersection of two ellipsoids (see [3]), i.e, $\mathcal{U }= \{(A^{(0)},a^{(0)}) + \Delta : \Delta \in \mathbb{R }^{k \times (n+1)}, \, \mathrm{Tr}(\Delta B_j \Delta ) \le \rho _j^2,\ j = 1,2\}$ where $B_j \in S^{n \times n}$ satisfying $\gamma _1 B_1+\gamma _2 B_2 \succ 0$ for some $\gamma _1,\gamma _2 \ge 0$.

1.3.2 Robust second-order cone programming problems [4, 7]

Consider the following robust second order cone programming problem:

$$\begin{aligned} \begin{array}{lcl} ({ RSOCP}) &{} \min \limits _{x \in \mathbb{R }^n} &{} a^Tx \\ &{} \text{ s.t. } &{} \Vert B_ix-b_i\Vert \le d_i, \ \forall (B_i,b_i) \in \mathcal{U }_i,\quad i=1,\ldots ,m, \end{array} \end{aligned}$$

where $\mathcal{U }_i\subseteq \mathbb{R }^{k_i \times n} \times \mathbb{R }^{k_i}= \mathbb{R }^{k_i \times (n+1)},\,i=1,\ldots ,m$, is an uncertainty set. Then, (RSOCP) can be equivalently rewritten as a semidefinite programming problem under the following two cases:

• (i) $\mathcal{U }_i$ is an ellipsoid (see [7]), i.e., $\mathcal{U }_i=\{(B_i^{(0)},b_i^{(0)}) + \Delta _i: \Delta _i \in \mathbb{R }^{k_i \times (n+1)}, \Vert \Delta _i-\overline{\Delta }_i\Vert _F \le \rho _i\}$;

• (ii) $\mathcal{U }_i$ is the intersection of at most $k$ many ellipsoids (see [4]), i.e, $k_i=k$ with $k \in \mathbb{N }$ and $\mathcal{U }_i= \{(B_i^{(0)},b_i^{(0)}) + \Delta : \Delta \in \mathbb{R }^{k \times (n+1)}, \, \Vert C_j \Delta ^T\Vert _F^2 \le \rho _j^2,\ j = 1,\ldots ,k\}$, where $C_j \in \mathbb{R }^{(n+1) \times (n+1)}$ such that there exist $\mu _j \in \mathbb{R }$ such that $\sum _{j=1}^k\mu _jC_j^TC_j \succ 0.$