Extended canonical duality and conic programming forsolving 0-1 quadratic programming problems

An extended canonical dual approach for solving 0-1 quadratic 
programming problems is introduced. We derive the relationship 
between the optimal solutions to the extended canonical dual problem 
and the original problem and prove that there exists no duality gap 
in-between. The extended canonical dual approach leads to a 
sufficient condition for global optimality, which is more general 
than known results of this kind. To solve the extended canonical 
dual problem, we construct corresponding conic programming problems 
and study their relationship to the extended canonical dual problem. 
Using this relationship, we design an algorithm for solving the 
extended canonical dual problem. Our work extends the known solvable 
sub-class of 0-1 quadratic programming problems.

1. Introduction. In this paper, we study the following 0-1 quadratic integer programming problem: where Q is an n × n real symmetric matrix and c is a vector in R n . The 0-1 quadratic integer programming problem (QIP in short) appears in many applications [19]. For example, the well studied max-cut problem in combinatorial optimization is a special sub-class of QIP. It is known that QIP is NP-Hard [7]. In other words, it is impossible to be solved generally in polynomial time, unless P=NP.
To find an exact global optimal solution to QIP, we usually count on some well performed enumeration strategies. The branch-and-bound algorithm with a lower bound estimation [14] applies such a strategy. However, in the worst case, the branch-and-bound algorithm may still need to enumerate through all the feasible points. Taking the computation time into consideration, researchers study approximation algorithms for finding an approximate solution to some classes of quadratic integer problems [8,12]. For example, Goemans et al. proposed an efficient approximation algorithm to achieve an approximation solution to the max-cut problem with a guaranteed approximation ratio of 0.878 [8].
Another effort on solving QIP is to identify some sub-classes that can be solved in polynomial time. Identifying polynomially solvable subclasses of binary quadratic programming problems not only offers theoretical insights into the complex nature of the problem, but also provides platforms for designing relaxation schemes to obtain exact solutions. For example, Allemand et al. discussed a polynomial time solvable case in [1]. With the assumption that the quadratic objective function has a low rank spectral decomposition, they adopted a well-designed enumeration algorithm to achieve a global optimal solution in polynomial time. Recently, Fang et al. used a canonical duality approach to provide a polynomial-time solvable sub-class of QIP [3]. Wang et al. further extended the result and proposed an approximation algorithm for solving the max-cut problem [18]. Canonical duality theory was firstly proposed by Gao [5]. It has been developed for solving global optimization problems [3,4,10,17,20,6]. Our work, motivated by these works, extends the canonical duality theory to solve the 0-1 quadratic programming problem, discovers a new global optimality condition for solving QIP and identifies a new solvable sub-class of QIP.
The rest of this paper is arranged as follows. We first briefly review the main results of the canonical duality theory and define an extended canonical dual problem. Then we present a perfect dual relationship between QIP and its extended canonical dual problem in Section 2. Followed by constructing a conic relaxation problem for QIP in Section 3, we discuss the relationship between the dual of this conic programming problem and the extended canonical dual problem. In Section 4, we propose an efficient algorithm for solving the extended canonical dual problem to generate an exact global optimal solution to QIP under appropriate assumptions. Some concluding remarks are given in the last section.
2. Canonical duality theory and extended canonical dual problem. In this paper, we adopt the following notations: M n denotes the set of all n × n real symmetric matrices, S n the set of all n×n symmetric positive semidefinite matrices, and N n the set of all n × n matrices with non-negative elements. Given a vector x ∈ R n , [x] i or x i denotes the i-th component of x, Diag(x) denotes the n × n diagonal matrix with x i being the i-th diagonal element. Especially, for a vector λ ∈ R n , Λ denotes the diagonal matrix Diag(λ). For two vectors x, y ∈ R n , x • y is a vector in R n with [x] i [y] i being its i-th component. For a real symmetric matrix U , U 0 means U is positive semidefinite and U ≻ 0 means U is positive definite. For two matrices A and B, denote A · B = trace(A T B). For a given optimization problem ( * ), its optimal objective value is denoted by V ( * ). Now we focus on the problem QIP. Since its 0-1 constraints can be equivalently written as x 2 i − x i = 0 for i = 1, 2, · · · , n, the Lagrangian function of QIP is defined by for λ ∈ F = {λ ∈ R n |det(Q + 2Λ) = 0}. From Theorems 1 and 2 in [3], we know the following results: The above theorem says that every critical point of the canonical dual function corresponds to a KKT point of the primal problem. The next theorem indicates that if a critical point λ * satisfies the "positive definite condition" of Q + 2Λ * ≻ 0, then its corresponding primal solution x λ * is indeed a global optimal solution to the problem QIP. Theorem 2.3. If P d (λ) has a critical point λ * such that Q + 2Λ * ≻ 0, then x λ * = −(Q + 2Λ * ) −1 (c − λ * ) is a global optimal solution of the problem QIP.
With the definition of the original canonical dual function and its properties in mind, we now consider an extended canonical dual problem. We first extend the feasible domain from F = {λ ∈ R n | det(Q + 2Λ) = 0} to G = {λ ∈ R n | (Q + 2Λ)x + (c − λ) = 0 has a solution x ∈ R n }. Evidently, we know F ⊆ G. Hence the set G is an extension of F . Next we want to define a function P c (λ) on G such that P c (λ) = P d (λ) for any λ ∈ F . For this purpose, we introduce the following lemma: Lemma 2.4. Let A be a real symmetric matrix in M n and b ∈ R n . If A is not invertible and the system of linear equations Ax + b = 0 is solvable, then Proof. Since x a and x b are solutions of Ax + b = 0, we have 1 2 With Lemma 2.4, we can define an extended canonical dual function P c (λ) on G as below.
In this way, we know P c (λ) = P d (λ) for λ ∈ F and P c (λ) is uniquely defined for any λ ∈ G − F . Therefore, we say P c (λ) is an extended canonical dual function. For function P c (λ), we would like to define its extended critical points. To do so, we need to introduce the extended gradient as well.
Definition 2.6. For any λ ∈ G, the extended gradient of the extended canonical dual function P c (λ) is defined as Moreover, we say λ * is an extended critical point of P c (λ) if 0 ∈ ∂P c (λ * ).
By this definition, we see a conventional critical point of P d (λ) is also an extended critical point of P c (λ). Meanwhile, the extended critical point has the following property: Thusλ is an extended critical point of P c (λ). On the other hand, ifλ is an extended critical point of P c (λ), then there exists anx such that (Q+2Λ)x+c−λ = 0 Definition 2.8. For an extended critical pointλ of P c (λ), define any xλ ∈ {x ∈ R n | (Q + 2Λ)x + c −λ = 0} {0, 1} n as a primal solution of QIP corresponding toλ (note any such xλ has the same primal objective value by Lemma 2.4).
From Lemma 2.7 and Definition 2.8, we see a perfect dual relationship in the next theorem.
Proof. Lemma 2.7 implies that, for an extended critical point λ * , there exists a corresponding primal point x λ * such that (Q + 2Λ * )x λ * + c − λ * = 0 and x λ * ∈ {0, 1} n . Hence x λ * is a feasible solution of the problem QIP and it satisfies the KKT condition. Meanwhile, we have F ( The next lemma further specifies the relationship between the feasible solutions of the problem QIP and the extended critical points of P c (λ).
Proof. For anyx ∈ {0, 1} n , notice that the system of linear equations ( is not zero, we know (Q + 2Λ)x + c − λ = 0 for λ always has a unique solution λx ∈ G, which is exactly the extended critical point.
Remark 1. Theorem 2.9 and Lemma 2.10 indicate that for any extended critical point λ * ∈ G, there always exists at least one primal solution corresponding to On the other hand, for any primal solutionx ∈ {0, 1} n , there always exists a unique dual extended critical point λx corresponding to it. This correspondence leads to the perfect dual relation in the sense of P c (λ * ) = F (x λ * ). Now, we define the extended canonical dual problem (ECD in short) of the problem QIP as below.
ECD is to find an extended critical point of the extended canonical dual function with a minimum objective value. Combining Theorem 2.9 and Lemma 2.10, we have the next theorem.
Theorem 2.11 (perfect dual relation). There is a perfect dual relation between problems QIP and ECD in the sense that V (QIP ) = V (ECD). Moreover, for any dual optimal solution λ * to problem ECD, there exists a corresponding primal optimal solution Proof. From Theorem 2.9 and Lemma 2.10, we know that each primal feasible solution has a corresponding dual extended critical solution, and each dual extended critical solution has at least one corresponding primal solution with the same objective value. Thus, the optimal solution of problem QIP must correspond to an optimal extended critical solution of problem ECD and V (ECD) = V (QIP ).
Note that the extended canonical dual problem ECD is well defined with a perfect dual relation in the extended definition. This new setting extends the previous results shown in [3], in which the canonical dual function is defined only when Q + 2Λ is invertible and the perfect dual relationship is not always satisfied.
3. Extended canonical duality and conic programming. Our goal is to solve the dual problem ECD for obtaining a global optimal solution to the primal problem QIP. In general, this is an NP-hard problem, because solving QIP is NP-hard. What we are interested in doing is to identify a solvable sub-class of QIP such that its dual problem ECD can be solved efficiently.
Our approach is based on the concept of linear conic relaxation. We start from reformulating the primal problem QIP as a linear conic programming problem.

CHENG LU, ZHENBO WANG, WENXUN XING AND SHU-CHERNG FANG
First, let us define a cone which is convex, and a set of matrices Let Cone(Z) be the convex cone generated by Z and denote D * n+1 = Cone(Z). Then we see the relation of D n+1 and D * n+1 in the next result.
This means that U ∈ D n+1 and, consequently, (D * n+1 ) * ⊆ D n+1 . Therefore, D n+1 = (D * n+1 ) * . Note that Lemma 3.1 also implies that D * n+1 is the dual cone of D n+1 . Remark 2. Similar definitions for D n+1 and D * n+1 can also be found in Strum and Zhang [15]. Now, we consider the following conic optimization problem with the rank one constraint: An immediate result can stated as below. Proof. If x ∈ {0, 1} n and X = xx T , then it is easy to verify that (x, X) satisfies all constraints of the problem QIP2. Hence (x, X) is a feasible solution to QIP2.
On the other hand, if (x, X) is a feasible solution of the problem QIP2, then the constraint of rank(Y ) = 1 leads to X = xx T and X ii = x 2 i for i = 1, 2, ..., n. From the constraint of X ii = x i , we can get x 2 i = x i . Consequently, x ∈ {0, 1} n . The above lemma shows that problems QIP2 and QIP have an equivalent feasible domain. Noting that they have the same objective function, we have the following result. Note that the rank one constraint of the problem QIP2 is not a convex constraint. By dropping this constraint, we may relax QIP2 to the following linear conic programming problem (COP in short): Using the conic duality theory, we define the dual problem of COP as Since the dual conic programming problem COD itself is hard to solve, we do not tackle it directly. Instead, we study its relation to the canonical dual problem ECD first. This may provide hints for us to solve the canonical dual problem using a conic programming method.
To do so, we first define a dual matrix corresponding to λ * , for any feasible solution λ * ∈ G of ECD, as Then we have an immediate result.
Theorem 3.4. If the canonical dual problem ECD has an extended critical point λ * ∈ G such that D(λ * ) ∈ D n+1 , then (σ * , λ * ) with σ * = 2P c (λ * ) is an optimal solution of the conic dual problem COD. Moreover, the corresponding primal solution x λ * is a global optimal solution of the original problem QIP.
Proof. Since the corresponding primal solution x λ * ∈ {0, 1} n , (x λ * , X * ) with X * = x λ * x T λ * must be feasible to both of QIP2 and COP. Noticing that we know (σ * , λ * ) is a feasible solution of the problem COD. Using the perfect dual relation P c (λ * ) = F (x λ * ) of Theorem 2.11, we can verify that Therefore, the complementarity condition is satisfied. From the optimality condition for linear conic programming, we know that (x λ * , X * ) is optimal to the problem COP and, consequently, optimal to the problem QIP2. Since problems QIP2 and QIP are equivalent, we know that x λ * is an optimal solution of the problem QIP.
The above theorem provides a new sufficient condition of global optimality for solving problem QIP. This condition actually extends the global optimality condition stated in Theorem 2.3 (or Theorem 3 in [3]), since each case that satisfies the condition in Theorem 2.3 also satisfies the condition in Theorem 3.4. The next example illustrates the difference of the global optimality conditions stated in Theorems 2.3 and 3.4. Example 1. Consider the following quadratic programming problem: The canonical dual function is defined as It is easy to verify that λ * = 0 is a critical point of P d (λ) with x λ * = 0. Then we have Q + 2Λ * = 0 1 1 0 , which is invertible but not positive semidefinite. Hence the condition in Theorem 2.3 is not satisfied. However, we see that since every term of D(λ * ) is non-negative. Applying Theorem 3.4, we know x λ * is globally optimal.
Since the rest of this section is centering around identifying a sub-class of QIP that can be globally solved, we state the following extended global optimality condition as an assumption: Extended Global Optimality Condition. The extended canonical dual function has an extended critical point λ * ∈ G with σ * = 2P c (λ * ) such that D(λ * ) = −σ * (c − λ * ) T c − λ * Q + 2Λ * ∈ D n+1 . The corresponding primal solution of λ * is denoted by x λ * .
It is interesting to note that, from Theorem 3.4, the λ * in the Extended Global Optimality Condition must be an optimal solution of the problem COD. But, solving COD directly may not provide λ * , because COD may have multiple optimal solutions in general. This situation is illustrated by the next example.
Example 2. Consider the following quadratic programming problem: The canonical dual function is defined as where e = [1, 1] T . It is easy to verify that λ * = [1, 1] T is an extended critical point of P c (λ) with x λ * = 0. The corresponding conic dual problem becomes It is not difficult to see that any (σ, λ 1 , λ 2 ) with σ = 0 and 0 λ 1 , λ 2 1 is an optimal solution of this conic relaxation problem. In particular, by Theorem 3.4, (σ * , λ * 1 , λ * 2 ) = (0, 1, 1) corresponding to the extended critical point of P c (λ) is an optimal solution of the conic relaxation problem. But solving this problem may or may not lead to λ * directly.
In order to find the λ * as stated in the Extended Global Optimality Condition, we need to conduct further analysis. Lemma 3.5. Under the Extended Global Optimality Condition with λ * and x λ * being defined, if (σ D , λ D ) is an optimal solution of the problem COD, then x λ * is a global optimal solution of the following mathematical optimization problem: Proof. From the given condition, we know that is an optimal solution of COP, by the complementarity condition of conic programming, we have This completes the proof.
The optimality condition of the problem MOP results in the next Lemma.
Lemma 3.6. Under the Extended Global Optimality Condition with λ * and x λ * being defined, let (σ D , λ D ) be an optimal solution of the problem COD.
Proof. Notice that the gradient of the objective function of the problem MOP at Combining Lemmas 3.5 and 3.6, we have the next result.
Now we can establish the relationship between the optimal solutions of problems COD and ECD as follows.
Theorem 3.8. Under the Extended Global Optimality Condition with λ * and x λ * being defined, if (σ D , λ D ) is an optimal solution of the problem COD, then λ * is the unique optimal solution of the following conic program: Proof. Notice that, for any feasible solution λ of the problem COP2, (σ D , λ) is an optimal solution of the problem COD. From Lemma 3.7, we have λ λ * . Since λ * is a feasible solution of COP2, it must be the unique optimal solution of COP2.
In summary, under the Extended Global Optimality Condition, the extended critical point λ * corresponds to one of the optimal solutions of the problem COD and λ * is actually the unique optimal solution of the conic program COP2.

Practical algorithm and numerical experiments.
Conic programming problems with different underlying structures have been studied recently, for example, Strum and Zhang treated cones of non-negative quadratic functions in [15] and Burer handled copositive cones in [2]. However, to our knowledge, there is no known algorithm that can efficiently solve the conic programming problem COP and its dual problem. In order to develop a practical computational algorithm, we will restrict this conic programming problem to a better structured cone. The idea is to substitute the cone D n+1 by some computable cone C n+1 such that C n+1 ⊆ D n+1 and D * n+1 ⊆ C * n+1 for the dual cones. Given such a cone C n+1 , we define the following conic programming problem: Then we have the next result.
Theorem 4.1. The optimal value of the problem P1 is a lower bound for the problem QIP. Moreover, if there exists an extended critical point λ * ∈ G such that D(λ * ) ∈ C n+1 , then this lower bound coincides with the optimal value of the problem QIP.
Proof. Since C n+1 ⊆ D n+1 , we have V (P 1) V (COD) V (QIP ). If there exists an extended critical point λ * such that D(λ * ) ∈ C n+1 ⊆ D n+1 , then, by Theorem 3.4, its corresponding primal solution x λ * is a global optimal solution of QIP. Hence V (P 1) = V (QIP ) and the lower bound becomes the optimal value of the problem QIP.
Assuming that σ D is optimal to the problem P1, we define another conic programming problem: Then we have the next theorem.
Theorem 4.2. If P c (λ) has an extended critical point λ * ∈ G such that D(λ * ) ∈ C n+1 then λ * is the unique optimal solution to the problem P2.
Proof. From Theorem 4.1, we know the value σ D is equal to 2V (QIP ), which also equals to σ D as defined in the problem COP2. Since D(λ * ) ∈ C n+1 ⊆ D n+1 , from Theorem 3.8, we know λ * is optimal to COP2 and any feasible solution of the problem P2 is also feasible for COP2. Hence λ * is the unique optimal solution of P2.
Now, let (Q + 2Λ * ) + denote the Moore-Penrose pseudoinverse of the matrix (Q + 2Λ * ) [13]. With problems P1 and P2, we have the following algorithm which provides either a global optimal solution to the problem QIP or a lower bound for QIP: Step 1: For a given problem QIP, construct the conic programming problem P1.
Step 2: Solve the problem P1 for an optimal solution σ D .
Step 3: Construct the conic programming problem P2 using σ D .
Step 4: Solve the problem P2 to obtain an optimal solution λ * .
Remark 3. For those cases satisfying the condition of Theorem 4.2, if Q + 2Λ * is invertible, then (Q + 2Λ * ) + becomes the conventional inverse matrix, and The next theorem validates Algorithm 1.
Theorem 4.3. If Algorithm 1 returns a solution x λ * successfully, then x λ * is a global optimal solution of the problem QIP. Otherwise, Algorithm 1 will return a lower bound.
Proof. If x λ * is returned successfully in Step 5 of Algorithm 1, then , we know λ * is an extended critical point of P c (λ). By Theorem 3.4, x λ * must be a global optimal solution of the problem QIP. Otherwise, from Theorem 4.1, a lower bound 1 2 σ D for QIP is returned successfully. It is important to note that, for practical computations, there are many choices for C n+1 . For example, the positive semidefinite cones S n+1 are computable cones for consideration. We may also consider some cones with better approximation effects (than semidefinite cones), for example, the cones of S n + N n = {A + B | A ∈ S n , B ∈ N n }. Moreover, the sequence of cones in [11] used by Klerk et al. to approximate the copositive cone from inside are also potential candidates.
The following three examples are used to illustrate how Algorithm 1 works.
Example 3. First, we consider the three-dimensional example used in [3] with    Example 5. Lastly, we consider a ten-dimensional example with  It is interesting to mention that the matrices Q+2Λ * are not positive semidefinite for Examples 4 and 5. In this situation, the canonical dual method proposed in [3] fails to find the global optimal solution, but Algorithm 1 still works. 5. Concluding remarks. In this paper, we have presented an extended canonical dual approach for solving 0-1 quadratic integer programming problems. By studying the relationship between the extended canonical dual problem and corresponding conic programming problems, we have discovered a new global optimality condition for solving QIP and identified a new solvable sub-class of QIP with a practical algorithm.
Our new global optimality condition is more general than those reported in literature, including Theorem 3 of [3] for the {0, 1}-constrained QIP, Lemma 2 of [16] for the {−1, 1}-constrained QIP, Proposition 3.2 of [9] for the quadratically constrained quadratic programs, and other articles of this kind.
To the best of our knowledge, a known solvable subclass of QIP is the set of those satisfying the Positive Definite Condition of Q + 2Λ * ≻ 0 (See [3]). In contrast, our proposed Algorithm 1 extends this solvable sub-classes to those QIP satisfying the condition in Theorem 4.2 with x λ * = −(Q + 2Λ * ) + (c − λ * ) ∈ {0, 1} n . This is certainly more general than the Positive Definite Condition. The performance of Algorithm 1 depends on the choice of the cone C n+1 . The positive semidefinite cone S n+1 is only one acceptable choice. As Examples 4 and 5 show, we may choose a cone tighter than S n+1 for better performance than previously known algorithms.
To continue this research, we are currently extending our work to handle the quadratically constrained quadratic programming problems. We are also investigating better computable approximation of the cone D n+1 to further improve the performance of Algorithm 1.