Sequential Lifting of General Integer Variables for Integer Programs

Lifting integer variables is a widely used technique to create strong cutting planes. In 1975, Wolsey introduced a method to compute the exact sequential lifting coefficients of bounded integer variables by solving many integer programs. This paper presents a new technique to perform exact sequentially up and down lifting of general integer variables. The technique requires solving only a single branching tree. Some computational results demonstrate that this new sequential lifting technique performed approximately 11 times faster than Wolsey’s technique. used technique [7, 8, 25-29]. Sequential lifting requires |F |=1 and up lifting assumes α β ∈ ≤ ∑ i E i i x is valid inequality for 1 0 = IP x P where kj=0 for all j ∈ F. Therefore, sequential up lifting assumes that α β ≤ ∑ i i is valid for 1 * = IP x k P and seeks to create an inequality of the form 1 1 2 . α α β ∧ = + ≤ ∑ n i i i x x Sequential down lifting assumes 2 α β = ≤ ∑ i i i n x is valid for 1 1 = IP x u P and obtains and inequality of the form 1 2 α α γ ∨ = + ≤ ∑ n i i i x x where γ is typically equal to 1 1 β α ∨ − u . There is also a sequential lifting when 2 α β = ≤ ∑n i i i x is valid for 1 * = IP x k P where k∗∈ {1... u1-1}, which is roughly a combination of both up and down lifting. Lifting can be approximate or exact. Exact lifting finds the strongest ' α and/or ' β possible. Thus, exact sequential up lifting finds the maximum value of α1∧ that still maintains the validity of the inequality. If such a value of α1∧ is obtained, then the dimension of the face induced by the sequentially lifted inequality increases by at least 1 in the unrestricted polyhedron. Approximate lifting techniques obtain coefficients that maintain valid inequalities, but at times these coefficients could be strengthened. These approximate lifting techniques do not necessarily increase the dimension of the induced face, but rather trade the computational effort required for exact lifting for a theoretically weaker inequality. Some such approximate lifting results include sequential up lifting [7] and sequence independent lifting [6,30-32]. Other work has used a linear relaxation or just a portion of the original problem to approximate the lifting coefficients [29,33]. Simultaneous lifting requires |F | ≥ 2. Zemel [34] provided an exact technique to simultaneously up lift sets of binary integer variables. This technique solves an exponential number of integers programs and then finds extreme points of the polar created from the solutions to these integer programs. This method yields numerous inequalities, Industrial Engineering & Management I n d u s tri al En gin eering &Mnagem e n t


Introduction
Define a bounded integer program (IP) as max c T x subject to Ax ≤ b, 0 ≤ x ≤ u, n x ∈  . IPs are NP-complete [1] and IPs are typically solved by branch and bound [2], which has an exponential run time. A common method that typically decreases the computational effort of branch and bound involves the addition of valid inequalities to the IP formulation. This paper presents a new technique to perform exact sequential lifting, which modifies a weak valid inequality into a stronger valid inequality. is r+1. A face of dimension r is a facet if the dimension of P IP is equal to r+1. Nemhauser and Wolsey [3] provide more information about polyhedral theory with respect to integer programs.
Generating a weak valid inequality is trivial. Lifting, introduced by Gomory [4], is a technique used to modify a weak inequality into a stronger inequality. Numerous researchers have used lifting to create useful cutting planes for various problems. A small subset of these articles is . Briefly, let E ⊆ N={1, 2 . . .n} and F ⊆ N\E be any nonempty set. Now let be a valid inequality of P IP where 0 ≤ k j ≤ u j for all j ∈ F. Lifting seeks to create a valid inequality of P IP , which takes the form There are various types of lifting, such as sequential, simultaneous, approximate and exact, up, middle and down lifting techniques. For distinction, α ∧ and α ∨ denote the coefficients that are obtained through up and down lifting, respectively, and α represents a lifted coefficient that is either up or down lifted.
Of these lifting techniques, sequential up lifting is the most widely used technique [7,8,[25][26][27][28][29]. Sequential lifting requires |F |=1 and up lifting assumes where γ is typically equal to There is also a sequential lifting when , which is roughly a combination of both up and down lifting.
Lifting can be approximate or exact. Exact lifting finds the strongest ' α and/or ' β possible. Thus, exact sequential up lifting finds the maximum value of α 1∧ that still maintains the validity of the inequality. If such a value of α 1∧ is obtained, then the dimension of the face induced by the sequentially lifted inequality increases by at least 1 in the unrestricted polyhedron.
Approximate lifting techniques obtain coefficients that maintain valid inequalities, but at times these coefficients could be strengthened. These approximate lifting techniques do not necessarily increase the dimension of the induced face, but rather trade the computational effort required for exact lifting for a theoretically weaker inequality. Some such approximate lifting results include sequential up lifting [7] and sequence independent lifting [6,[30][31][32]. Other work has used a linear relaxation or just a portion of the original problem to approximate the lifting coefficients [29,33].
Simultaneous lifting requires |F | ≥ 2. Zemel [34] provided an exact technique to simultaneously up lift sets of binary integer variables. This technique solves an exponential number of integers programs and then finds extreme points of the polar created from the solutions to these integer programs. This method yields numerous inequalities, but is computationally intensive. Recently, [17] created a linear-time algorithm that exactly up lifts sets of binary variables into a cover inequality generated from a single binary knapsack constraint (a single nonnegative less than or equal to constraint, {x ∈ {0, 1} n :ax ≤ b, a ≥ 0}). Observe that the aforementioned sequence independent lifting references could also be viewed as an approximate method to up lift sets of integer variables.
Prior to this research Wolsey [29] provided the only known technique to exactly perform sequential lifting of general integer variables. His method requires the solution to u 1 integer programs. This paper presents a new technique to perform exact sequential lifting of general integer variables. This new technique requires the solution to only a single branching tree for both up and down lifting and two branching trees if lifting over a valid inequality when x 1 =k * where k * ∈ {1, ..., u 1 -1}.
The remainder of the paper is organized as follows. Section 2 presents the new technique to sequentially up lift integer variables and compares this to Wolsey's existing technique. Some computational results are contained in Section 3 that demonstrate that this new method is faster than Wolsey's method. A conclusion and some directions for future research are discussed in Section 4.

Up Lifting Integer Variables
Wolsey [29] introduced a method to exactly lift general integer variables. This technique is derived directly from his theorem which states: If, for a specific k, the problem is infeasible, then let The sequentially lifted inequalities are As mentioned in the introduction, of the many possible values of k * the most frequently used is k * =0, which is called up lifting. That is, a valid inequality is obtained when x 1 =0, and x 1 is up lifted into this inequality. From Theorem 2.1 it is easy to see that Wolsey's up lifting method requires the solution to u 1 integer programs and the right hand side β does not change. Now the attention turns toward the purpose of this paper, which produces a new technique to up lift general integer variables. The algorithm creates an inequality that increases the dimension of the face, but may not necessarily be a valid inequality (guess too high of a value for α 1∧ ). A modifiable branching tree is then used to check to see if the inequality is valid. If the inequality is not valid or equivalently there exists a point violating the proposed inequality, then α 1∧ is decreased according to this feasible point and the objective function is reset with this new α 1∧ . This process continues until all nodes are fathomed.
The input to the Sequential Lifting Algorithm (SLA) is composed of the constraints and bounds of a general integer program and an SLA up lifts x 1 by finding an α 1∧ that is the maximum value such that The Sequential Lifting Algorithm (SLA) Begin a modified branch and bound tree by letting the following LP be the unfathomed root node of the tree.
While there exist unfathomed nodes in the branch and bound tree, begin. Select an unfathomed node and solve the linear relaxation with the solution denoted by * x that is non-integer.
One node has the parent's LP with the added inequality The other child's node has the parent's LP and adds the inequality The following example demonstrates this algorithm and provides some fundamental insights into the differences between Wolsey's method and SLA. The branching tree is explored according to a depth first left strategy. .
First observe that x 2 ≤ 4 is a valid inequality of 1 IP x P that defines a face of dimension 0 from the point (0, 4). Figure 1 provides the branching tree to exactly up lift x 1 . SLA begins with the root node given by the LP .
. Two new child nodes are created. One adds on the constraint x 1 ≤ 4 and the other . This is an integer solution and 4M>β=4. So α 1∧ is changed so that 4 × α 1∧ +0=4 and α 1∧ =1. The objective function of all remaining LPs is changed to x 1 +x 2 . Unlike branch and bound, this node is not fathomed; instead a single child node is added with the same LP as its parent, but with the updated objective value. The solution to node 4 is * 4 = The method then takes the minimum of these values as α 1∧ =1, which guarantees a valid inequality and increases the dimension by at least 1. In Figure 1, values, because they are on the same line. Essentially, Wolsey's method checks every possible extreme point and accepts the best α 1∧ value that maintains validity. Clearly some work is typically waisted.
In contrast, SLA begins by starting with an objective function that is nearly parallel to the x 2 axis. As soon as an integer solution is discovered with value larger than 4, α 1∧ is changed to intersect this point. In this case, it happens at node 4 with the point (4, 0). The inequality now tested for validity is x 1 +x 2 ≤ 4, which intersects (4,0). If no integer solutions satisfy x 1 +x 2 ≤ 4, then the inequality is valid. However, node 5 finds such a violating point (2,3) and the objective function changes to 1 2 . Observe that (2, 3) meets this inequality at equality.
Eventually, the tree is fathomed and there are no integer solutions with a value larger than 4 to the objective function. Thus, the inequality is valid.
Essentially SLA maintains inequalities that guarantee to increase the dimension of the face (an additional point meets the inequality at equality), but may not necessarily be valid. Once the algorithm terminates, the inequality is clearly valid since there does not exist a feasible point with a value larger than β. Since α 1∧ is calculated from a feasible point, the dimension of the lifted inequality's face also increases in the unrestricted polyhedron. SLA seeks to obtain the lifting coefficient by coming through the inside of the polyhedron; whereas, Wolsey's method seeks to obtain the lifting coefficient by identifying all extreme points.
One may attempt to incorrectly argue that SLA is Wolsey's method. The erroneous argument states that the root node should have u children with each branch having x i set to a different integer between 0 and u. Such an IP would only generate a single objective function. This single IP could not accurately calculate the lifting coefficient. Thus, SLA is not merely an extension of Wolsey's method, but a new method to perform sequential lifting.
The main theoretical result of the paper, which states that SLA terminates with a valid inequality, can now be presented. Furthermore, this inequality results in the same coefficient as Wolsey's method, and, under a feasibility assumption, the dimension of the inequality increases over the non-restricted polyhedron.  in P IP as long as α 1∧ is finite. is strictly less than the previous value of α 1∧ . Thus, the values of α 1∧ monotonically decrease as the branching tree progresses.
Due to the monotonically decreasing nature of the α 1∧ , any integer solution contained in the feasible solution space of a node that is fathomed because z ≤ β satisfies the returned inequality ( In such a scenario, the solution to every child would either be integer or infeasible. Any solution to an integer with value larger than β, creates an additional child. The solution x values of this child node must be identical to its parent's solution and thus the objective value is equal to β. This new node is then fathomed. Thus, each node is fathomed in a finite number of steps and SLA terminates.
Assume SLA terminates with α 1∧ =∞. In such a case, no integer solutions were encountered in the branching tree. Thus, the solution to Maximize For contradiction, assume Wolsey's coefficient is less than α 1∧ . Then there exists x ′ be the solution to Wolsey's IP when x 1 =k'. Clearly since k x ′ is integer, this contradicts the prior argument on the validity of SLA. Consequently, SLA and Wolsey's method both produce identical coefficients.
Finally, assume An additional advantage to SLA is that it may enable exact up lifting over unbounded integer programs. Wolsey's method does not enable this type of lifting as solving an infinite number of IPs is not possible. However, SLA may not terminate for an unbounded integer program as a critical assumption for the finite termination proof is no longer valid. For instance, implementing SLA to lift x 1 into x 2 ≤ 0 for would never terminate. Consequently, if SLA terminates for an unbounded integer program, then SLA determines the correct up lifting coefficient.

Sequential Down Lifting and Lifting when x 1 =k *
With a fundamental understanding of up lifting, it is now straightforward to modify SLA into algorithms that can down lift and lift over a valid inequality when x 1 =k * . For brevity two new algorithms and theorems for these other types of lifting are not presented. Rather changes are described that enable the reader to trivially extend SLA to these other two sequential lifting types through variable substitution. An example of each algorithm is also provided to aid with comprehension.
Down lifting can be viewed as substituting 1 ' x for x 1 where When an integer solution is found that is larger than β, the linear relaxation point, * LR x , is substituted into the equation Everything else is identical and trivial to determine with knowledge of this substitution. The following example provides a demonstration of down lifting using SLA.

Example 3.1:
Returning to Example 2.2, if x 1 =4, then x 2 ≤ 0 is a valid inequality that defines a face of dimension 0 due to the point (4, 0). The down lifted branching tree is shown in Figure 2 using a depth first left node evaluation strategy. The first integer solution is found at node 2 with a Z=4M+4>0=β and x=(0, 4). Solving for α 1v in α 1v (4-0)+ 4=0 results in α 1v =-1. Node 3 also is integer with Z=1>β and x= (2,3). Solving for α 1v in  To help clarify the idea that middle lifting may not result in a valid inequality, observe that x 2 ≤ 1 is valid if x 1 = k * =3. This inequality defines a face of dimension 0 from the point (3,1). To find the proposed up lifted coefficient, set the objective function to α 1∧ (x 1 -3)+x 2 and add the constraint x 1 ≥ k * +1=4. The optimal integer solution is (0,4), which sets α 1v =1. Any remaining nodes are easily fathomed. For the down lifting portion, the optimal integer solution is (2, 3), which sets 1 2 α ∧ = − . Since

Computational Results
This section demonstrates that SLA is substantially faster than Wolsey's method. All computational results were performed on an Intel Core i7-2600 chip at a 3.4 GHz processor with 8 Gb of RAM and the integer programs were solved using CPLEX 10.0 [35] at its default settings.
The problems chosen for this study were generated from random multiple knapsack polytope  Table 1.
The computational study starts with the valid inequality x 1 ≤ u 1 and sequentially up lifts every other variable until the inequality becomes facet-defining. Fifty random knapsack instances are created for each problem size. Table 1 reports the total time in seconds required to lift all of the variables. Thus, when n=10, the total time columns represent the time required to sequentially up lift 9 variables for each of the 50 problems or 450 variables with either Wolsey's Algorithm or SLA. In  addition, the total number of integer programs that Wolsey's method solved is also reported. Table 1 shows the computational superiority of SLA. On average SLA lifted variables eleven times faster than Wolsey's method. SLA was at least as fast in every instance. As expected, the larger the average upper bound the greater improvement of SLA. For instance, the largest bounds occurred with the slack coefficient of 2 and when there are fifty variables. The average of these 150 instances shows that SLA is about 25 times faster than Wolsey's method. Some additional computational results [36] demonstrate SLA's superiority over Wolsey's method.

Conclusion and Future Research
Since Wolsey's paper in 1975 and prior to this work, there has only been one method to perform exact sequential lifting. This existing method requires solving u i+1 integer programs where u i is the upper bound for variable x i . This paper presents a new technique, SLA, to perform exact sequential lifting of bounded integer programs. SLA requires solving a single branching tree for both up and down lifting, and only two branching trees for middle lifting. SLA, under reasonable assumptions, becomes the first method for sequential lifting of unbounded integer programs.
A computational study compared the amount of computational effort required to sequentially up lift many variables using both Wolsey's method and SLA. SLA performed 11 times faster than Wolsey's method. Thus, SLA is both theoretically and computationally superior to the existing method. It is recommended that researchers performing sequential lifting implement SLA instead of Wolsey's method.
The creation of SLA also raises several important research questions. Can SLA's new method and theory be extended to perform exact simultaneous lifting? Can the concepts behind SLA be extended into a new method to obtain approximate lifting coefficients (terminating SLA prior to each node being fathomed)? Can polynomial time methods be developed to create sequentially lifted inequalities?