NONCONVEX QUADRATIC REFORMULATIONS AND SOLVABLE CONDITIONS FOR MIXED INTEGER QUADRATIC PROGRAMMING PROBLEMS

. In this paper, we study a mixed integer constrained quadratic programming problem. This problem is NP-Hard. By reformulating the problem to a box constrained quadratic programming and solving the reformulated problem, we can obtain a global optimal solution of a sub-class of the original problem. The reformulated problem may not be convex and may not be solvable in polynomial time. Then we propose a solvability condition for the reformulated problem, and discuss methods to construct a solvable reformulation for the original problem. The reformulation methods identify a solvable subclass of the mixed integer constrained quadratic programming problem.

1. Introduction. In this paper, we consider a mixed integer constrained quadratic programming problem, which is defined as follow.
where Q is an n × n real symmetric matrix, c is a vector in R n , x ∈ R n , sets I and J satisfy I ∪ J ⊆ {1, 2, ..., n} and I ∩ J = ∅. The feasible domain of MQP is denoted by F = {x ∈ R n |x i ∈ {0, 1} for i ∈ I, x i ∈ [0, 1] for i ∈ J}. This mixed integer constrained quadratic programming problem (MQP in short) is important in both theoretical researches and real applications. We can find many industrial and management applications for this general problem. For example, in [5] Bienstock showed that MQP could play very important role in the portfolio optimization. And two subclasses of MQP such as box constrained quadratic programming and 0-1 constrained quadratic programming also have many applications. For example, in [3] [4] Billionnet et al. studied the quadratic 0-1 knapsack problem which is a quadratic 0-1 problem; in [14] [21] researchers used quadratic box constrained problem in numerical simulation of friction problems in rigid body mechanics and image reconstruction from projections.
Since MQP is an NP-Hard problem (see [13]), there is no polynomial algorithm for solving it if N P = P . Many studies try to identify polynomial time solvable subclass of MQP. For example, for 0-1 constrained quadratic programming problem, which can be denoted as QIP. Many researchers have done great works and provide different global optimality conditions. For example, Allemand et al. discussed a polynomial time solvable case in [1]. They adopted a well-designed enumeration algorithm to achieve a global optimal solution in polynomial time. Jeyakumar et al. provided global optimality conditions for Non-convex quadratic minimization problems with quadratic constraints in [15]. They obtained necessary global optimality conditions for weighted least squares with ellipsoidal constraints, and quadratic minimization with binary constraints. Fang et al. used a canonical duality approach to provide a polynomial-time solvable sub-class of QIP in [10]. Further, Wang et al. extended the result and proposed an approximation algorithm for solving the max-cut problem in [25]. Burer et al. modeled nonconvex quadratic program having a mix of binary and continuous variables as a linear program over the dual of the cone of copositive matrices in [8]. They provided a relationship among some convex sets of symmetric matrices. Based on this relationship, they showed that these two problems are equivalent. In this way, they packaged all the difficulty completely inside the convex cones of completely positive matrices. Besides this, they also showed the possibility to reduce the dimension of the completely positive representation and established extension to complementarity constraints over bounded variables. Moreover, Bomze et al. interpreted Burer's work from a topological point of view in [7]. They defined a weak key condition and showed that under this condition the Minkowski sum of the lifted feasible set and the lifted recession cone will give exactly the closure of the former. Sun et al. gave new results on the duality gap between the binary quadratic optimization problem and its Lagrangian dual or semidefinite programming relaxation in [24]. They derived a necessary and sufficient condition for the zero duality gap and discussed its relationship with the polynomial solvability of the primal problem.
Our work, motivated by these works, identifies a new solvable sub-class of MQP and provides a practical algorithm to get a global optimal solution of a subclass of MQP.
We study MQP from the viewpoint of quadratic reformulations, that is, we reformulate MQP to another quadratic programming problem, and by solving the reformulated problem to obtain an optimal solution of MQP. Since the MQP problem is NP-Hard in general, there is no polynomial-time algorithm for solving it, unless P=NP. However, for some subclasses of MQP, it may be transformed to a polynomial-time solvable convex programming problem. Then we can design algorithms for solve these subclasses with hidden convexity. Besides, for another subclass, the problem may not be transformed to a convex problem. Instead, it can only be transformed to an equivalent nonconvex quadratic programming problem.
Since finding a local optimal solution of a nonconvex quadratic programming problem is also not hard, then if the local optimal solution of the problem is unique, and by conventional local-search algorithms, the problem is still solvable. In this paper, we will give conditions that cases of MQP can be transformed to a solvable problem.
This paper is arranged as follows. Firstly we introduce the definition of quadratic reformulation from the viewpoint of Lagrangian dual theory. Then based on the quadratic reformulation, we discuss conditions for solvability of the reformulations. Next, we discuss the relationship between the Lagrangian multiplier and a conic programming problem to design algorithms for obtaining a solvable reformulation. Lastly, we provide an algorithm to compute solvable reformulations.
In this paper, we adopt the following notations: For an n × n matrix Q and a set K ⊆ {1, 2, ..., n}, Q KK denotes the sub-matrix of Q, with sub-rows and subcolumns of Q whose index belonging to K. For two matrices A and B, A · B denotes trace(A T , B). Let e i be a vector in R n , with its i-th element being equal to 1 and other elements being equal to 0, and e be a vector in R n with all elements being equal to 1. For a given optimization problem ( * ), its optimal objective value is denoted by V ( * ).
2. Lagrangian dual and quadratic reformulation. The Lagrangian function for MQP is defined as where Λ denotes the n × n diagonal matrix with λ i being the i-th diagonal element, and λ i 0 for i ∈ J.
Since MQP is non-convex, the duality gap between MQP and its conventional Lagrangian dual problem is not always zero. So we consider the extended Lagrangian duality. The extended Lagrangian dual function is defined as P e (λ) = min x∈[0,1] n L(x, λ). By this definition, the value of the extended Lagrangian dual function is no larger than the primal optimal value for any λ ∈ G, as stated in the following theorem.
Proof. By definition, we have P e (λ) = min x∈[0,1] n L(x, λ) ≤ min x∈F L(x, λ) (noting F ⊂ [0, 1] n ). For any x ∈ F, it is easy to verify that L(x, λ) Then the extended Lagrangian dual problem is defined as max {λ∈R n , λi 0 for i∈J.} P e (λ). ( Different from the conventional Lagrangian dual problem, the extended Lagrangian dual problem always satisfies the strong dual principle, which can be stated as the following theorem.

YE TIAN AND CHENG LU
Theorem 2.2. The gap between problem (2) and problem (1) is always zero.
Then we have the next corollary.
Corollary 1. For any optimal solution λ * of problem (2), the problem is a reformulation of problem (1) in the sense that any optimal solution of problem (1) is optimal to problem (3).
This corollary implies that the Lagrangian function L(x, λ * ), with λ * being an optimal solution of problem (2), is a quadratic reformulation of problem (1). This reformulation transforms the original problem to a continuous optimization problem.
Since the optimal solution of problem (2) is not unique, there are many reformulations in such a way. Our object is to find a solvable reformulation to solve the original problem.
3. Solvable conditions. We have discussed reformulations of problem (1). In this section, we will discuss the solvability of these reformulations.
The simplest solvable case is the convex case, i.e., if L(x, λ * ) is a convex function, then the reformulation is solvable. Hence we have the next theorem: has a unique optimal solution x * , which is also an optimal solution of problem (2).
Proof. If Q+2Λ * 0, then problem (3) is a convex quadratic programming problem, which is polynomial-time solvable. Besides, if Q + 2Λ * 0, then the objective function of problem (3) is strictly convex, and their exists a unique optimal solution for problem (3).
The Convex Solvable Condition is the simplest solvable condition for reformulations.
For problem (1), any optimal solution x * must satisfies the following KKT condition, that is, there exists a Lagrangian multiplier λ * The KKT condition is not a sufficient optimality condition for problem (1). In the literature, we can find the following Positive Semidefinite Condition: (1) has a KKT solution x * with its Lagrangian multiplier λ * satisfy Q + 2Λ * 0.
Then we have the following theorem.
Hence, the Lagrangian multiplier in the Positive Semidefinite Condition can be interpreted as a convex reformulation of problem (1). Now we consider nonconvex cases. Let x * be a KKT solution of problem (1) with its corresponding Lagrangian multiplier λ * . Define the tangent cone T (x * ) at x * as We give the next condition to verify the global optimality of x * for problem (1) and optimality of λ * for problem (2).
Under Theorem 3.3, it is easy to verify that the reformulation problem is solvable, since any local optimal solution is global optimal for the reformulated problem. However, Under conditions of Theorem 3.3, there is no guarantee to obtain the optimal solution x * by local search, since there may be multiple local optimal solutions.
The following stronger condition guarantees the uniqueness of local optimal solution for reformulation. Proof. The proof is similar to that of Theorem 3.2. For any x ∈ [0, 1] n such that x = x * , let d = x − x * and f d (t) being defined the same as that in Theorem 3.2, and using a similar method, we can prove f d (1) > f d (0) and f d (t) is a strictly increasing function on [0, 1], which implies L(x, λ * ) > L(x * , λ * ), and x is not a local optimal solution of L(x, λ * ) (since −d is a decreasing feasible direction at x for the reformulated problem). Hence x * is the unique local optimal solution for min x∈[0,1] n L(x, λ * ).
Under condition of Theorem 3.4, the reformulation may not be convex, but it has only one local optimal solution, so it can be solved by any local optimization algorithms to find its unique local optimal solution. This condition is also a sufficient condition for global optimality. Here, we give a name for it.
Condition 2. (Second order solvability condition) There exists a KKT solution x * of problem (1), with its corresponding Lagrangian multiplier λ * , such that d T (Q + 2Λ * )d ≥ 0 for all d ∈ T (x * ), then we say problem (1) satisfies the second order solvability condition. If d T (Q + 2Λ * )d > 0 for all d ∈ T (x * ), then we say problem (1) satisfies the second order strong solvability condition.
This solvable condition is more general than Positive Semidefinite Condition. In [6], I.M.Bomze provided a global optimality condition for quadratic programming problem with a polyhedron feasible set. The above solvable condition on problem (1) situation is similar to their conditions. In our work, we will not only give the solvability condition in theory, but also provide methods to find reformulations that satisfies the above conditions to design practical algorithms.

KKT system and Conic reformulation problem.
To obtain a solvable reformulation, we will discuss the relationship between the Lagrangian multiplier and a conic programming problem, and compute the reformulation by solving the conic programming problem.
Define a cone For a KKT pair (x * , λ * ), define an corresponding matrix Proof. Since x * is a KKT solution of problem (1), thus x * i = 0 or 1 for all i ∈ I, 0 ≤ x * i ≤ 1 for all i ∈ J. For any x ∈ [0, 1] n , let d = x − x * . We can verify that d ∈ T (x * ). And for any d ∈ T (x * ), we always can multiply a positive scale to make the new d satisfy that 0 ≤ x * + d ≤ 1. Then we have that The above theorem transformed the second order solvable condition to the condition D(x * , λ * ) ∈ D n+1 . Now we give more discussions for this condition.
Denote D * n+1 as the dual cone of D n+1 . We define the conic programming min 0 with its dual problem Then we have the next theorem.
, then we can easily verify Thus the complementary condition satisfies. From the optimality theory of conic programming, we know that x * is a global optimal solution of problem (5), and (σ * , λ * ) is an optimal solution of problem (6), where σ * = −F (x * ). And since problem (5), problem (6) and problem (1) are equivalent, we also know that x * is a global optimal solution of problem (1).
Hence, if there exists a KKT solution x * with its corresponding Lagrangian multiplier λ * satisfy D(x * , λ * ) ∈ D n+1 , then (σ * , λ * ) with σ * = −F (x * ) is an optimal solution of problem (6). However, by solving problem (6), there is no guarantee to obtain λ * The above theorem also provides a sufficient condition of global optimality for problem (1). We give a name here.

Condition 3. (Extended global optimality condition)
The problem (1) has a KKT solution x * with its Lagrangian multiplier λ * such that D(x * , λ * ) ∈ D n+1 . Lemma 4.3. Under the Extended Global Optimality Condition with λ * and x * being defined, if (σ D , λ D ) is an optimal solution of the problem (6), then λ D ≤ λ * . (1)). And (1)) for any x ∈ [0, 1] n . So x * is optimal for the problem min x∈[0,1] n L(x, λ D ). Notice that the gradient of the function Due to the special property of optimal solution σ D , λ D and the relationship between λ D , λ * , we have the following key result: Under the Extended Global Optimality Condition with λ * and x * being defined, if (σ D , λ D ) is an optimal solution of the problem (6), we have that σ D = −V ((1)), and λ * is the unique optimal solution of the following linear conic programming problem: Proof. We notice that any feasible solution (σ D , λ) of problem (7) is optimal to problem (6) with λ ≤ λ * implied by lemma 1. Because (σ D , λ * ) is feasible to problem (7), and the linear independence for every constraint for x i , λ * is the unique optimal solution of problem (7).
The above theorem indicates that under the Extended Global Optimality Condition with λ * being defined, λ * is the unique optimal solution of problem (7). Hence, if we can obtain the optimal solution λ * of problem (7), then we get a solvable reformulation min x∈[0,1] n L(x, λ * ) for problem (1). 5. Proposed algorithm and numerical example. Because problem (6) is equivalent to problem (1), we pack the whole difficulty of the original problem into the cone D n+1 and D * n+1 , thus there is no polynomial time algorithm for solving problem (1) unless P=NP. Then in order to solve the problem (6), we need to choose a computable cone C n+1 which satisfy C n+1 ⊆ D n+1 to substitute the cone D n+1 which is difficult to compute. Given such a cone C n+1 , we define the following conic programming problem: Assuming that σ d is the optimal solution of C1, we define an additional conic programming problem: Correspondingly, we have the following result.
Remark 1. The primary purpose of this paper is theoretical-to provide a more general global optimality condition for problem (1) and identify a bigger solvable sub-class of problem (1). We do not aim to discuss how to find a computable cone C n+1 to approximate the cone D n+1 . This is a big issue in recent research ([9] [20] [22] [26]). The performance of the lower bound is determined by the choice of C n+1 . A tighter inner approximation of the cone D n+1 not only provides a tighter lower bound, but also a higher chance of getting the global optimal solution of problem (1).
With problem (8) and problme (9) being defined, we design the following algorithm to compute a reformulation of problem (1).
Step 2: Solve the problem (8) to get its optimal value σ d .
Step 5: Solve min x∈[0,1] n L(x, λ * ) to obtain a local optimal solution x * . If F (x * ) = σ d , then return x * as an optimal solution of problem (1); otherwise, the algorithm fails to obtain a solvable reformulation.
The next theorem validates Algorithm 1.
Theorem 5.2. If Algorithm 1 returns a solution x * successfully, then x * is a global optimal solution of the problem (1).
Proof. At step 5 of Algorithm, if x * is returned successfully, then F (x * ) = σ d guarantees that x * is a global optimal solution of problem (1).

Remark 2.
For all known methods in literature, the largest solvable subclasses of mixed integer constrained quadratic programming problem are those satisfying the Positive Semidefinite condition with Q + 2Λ being invertible. But for Algorithm 1, a solvable subclass of nonconvex quadratically constrained quadratic programs are those satisfying the condition of Theorem 5.1 with Q + 2Λ being invertible. Under this condition, we can obtain a reformulation L(x, λ * ) satisfies the second order solvability condition. If C n+1 = S n+1 , then these two conditions become equivalent. Whereas, the cone C n+1 can be better chosen than S n+1 , based on the structure of the feasible domain F. For example we can choose C n+1 = S n+1 +N n+1 which is bigger. In this way, the performance of Algorithm 1 could be improved. And in this sense, the newly proposed Algorithm 1 extends the known solvable subclass to more general cases.
Example 2. Here we choose C n+1 = S n+1 + N n+1 . Using Algorithm 1, we find the optimal solution λ The above examples demonstrate that Algorithm 1 is applicable for a larger subclasses of solvable MQP problems. Based on the different structures of some given problems, we may find some tighter computable cones for relaxation. Then we may improve the performance of our algorithm. 6. Conclusion. In this paper, we studied quadratic reformulation methods for mixed integer constrained quadratic programming problem. We gave a more general global optimality condition based on KKT system. It is the generalization of the known Positive Semidefinite Condition in Literature. Under this condition, a bigger sub-class of the problems can be polynomial-time solved. Then based on this condition, we reformulated the original problem into a conic problem. And for the solvable sub-class problems, we proposed a practical algorithm to get the corresponding optimal KKT pair. The proposed Algorithm 1 sheds some light on designing effective algorithms for a new solvable sub-classes of MQP.
Some researchers have studied the relationship between quadratic programming and conic programming in literature. For instance, Burer reformulated the mixedbinary QPs in copositive representation in [8]; Strum and Zhang reformulated the general nonconvex quadratic programming problem as a conic programming problem in [23]. Their works mainly discussed on the relationship between the solutions of the original quadratic optimization problem and the reformulated conic programming problem. Compared to their works, the main difference of this paper is that we consider the relationship between the Lagrangian multipliers of the quadratic optimization problem and several corresponding conic programming problems. The Lagrangian multipliers gave us some new insights, which is not easy to see from the original problem and its conic reformulation directly, and led us to discover a larger solvable sub-class of MQP.
The relationship between the Lagrangian multipliers and the SDP relaxation method can also be found in [19]. In this paper, we have refined and extended this relationship to more general cones which include the positive semidefinite cones. In this sense, our work is an extension of the previously know results. Our simple numerical example clearly indicates that the proposed Algorithm 1 indeed can solve some new solvable subclass which can not be solved by using previously reported algorithms in literature.
The proposed algorithm is based on the choice of the computable cone C n+1 . We are currently investigating new techniques to approximate the cone D n+1 from inside.