Knapsack constraint reformulation: A new approach that significantly reduces the number of sub-problems in the branch and bound algorithm

: The paper presents a new approach to significantly reduce the number of sub-problems required to verify optimality in the branch and bound algorithm. The branch and bound algorithm is used to solve linear integer models and these models have application in areas such as scheduling, resource allocation, transportation, facility allocation and capital budgeting. The single constraint of the knapsack linear integer problem (KLIP) is reformulated in such a way that the number of standard branch and bound sub-problems required to verify optimality is significantly reduced. Computational results of the proposed approach on randomly generated KLIPs are also presented.

ABOUT THE AUTHORS Elias Munapo and Santosh Kumar have interest in discrete optimization methods and they have put forward several ideas that may provide further insight into discrete optimization and linear programming methodology. Their approaches need an appropriate software support, which is a challenge not only for the authors but for the field of optimization. They have made direct contributions to the field of integer programming and special models like quadratic assignment, travelling salesman, network routing by link weight modification. The current paper on knapsack constraint reformulation opens up many interesting questions that can only be answered provided this paper reaches researchers in discrete optimization. We believe, our work challenges software developers to raise funds, support university research for higher degrees. We can provide supervision, if required and hope universities will welcome such a move from the software companies.

PUBLIC INTEREST STATEMENT
Resource constraints often arise in all studies. Mathematical classification is generally based on intrinsic property of the situation. One such problem in optimization has been described by a generic name, "Knapsack Problem", meaning that the knapsack has a finite limited capacity and our interest is to fit in that knapsack as much as possible from the available opportunities. Its applications arise in various areas. A typical knapsack constraint, which deals with only positive numbers, can be written as: Here, a 1 , a 2 + … + a n ≥ 0 are the known amounts of resource that will be used per unit and x 1 , x 2 + … + x n ≥ 0 represent the unknown integers to be determined with some objective to be achieved. This paper reformulates the above constraint and reduces the required computational effort. This reformulation may have applications in other areas, which requires further attention.
a 1 x 1 + a 2 x 2 + … + a n x n ≤ b

Introduction
The general linear integer programming (LIP) problem has many applications in real life, for example, they arise in set covering, travelling salesman, assignment, transportation, knapsack, capital budgeting, facility location, timetabling and airline scheduling. Many of these applications have been presented in Taha (2004) and Winston (2004), where these problems have been formulated as a linear integer programming (LIP) model. Because of real-life applications, see Chinneck (2004), the LIP model has attracted so much attention from researchers, yet a consistent and efficient general purpose method has not been developed. We are not aware of any polynomial time algorithm for the general LIP model. In fact, the LIP model is NP complete and a polynomial algorithm for the LIP is believed not to exist. This paper presents a new approach, which takes advantage of the presence of a single constraint to solve a knapsack linear integer problem (KLIP). The problem is reformulated in such a way that the number of standard branch and bound sub-problems required to verify optimality is significantly reduced. Computational results of the proposed approach on randomly generated KLIPs are also presented.
A mathematical statement of the knapsack problem is given in Section 2 and in Section 3, a few problems are discussed where the branch and bound (B & B) performance is poor. We have categorized these problems according to the possible source leading to poor performance. These problems have been subdivided in seven classes. Knapsack constraint reformulation is presented in Section . A summary of the computational experiments on randomly generated problems is presented in Section 5 and finally, the paper has been concluded in Section 6.

The knapsack linear integer problem
such that: where a j , b and c j are given nonnegative constants, and and x j ≥ 0 are integer valued quantities and j = 1, 2, … , n.

Standard branch and bound algorithm
The B & B algorithm was first proposed by Land and Doig (1960) for solving integer programs. The algorithm was further modified by Dakin (1965) to solve both pure and mixed integer programs. The B & B algorithm in general relies on the usual strategy of first relaxing the integer problem into a linear programming (LP) model. If the linear programming optimal solution is an integer then, the optimal solution to the integer problem has been obtained and the search concludes. If the LP optimal solution is not an integer, then a variable with a fractional value is selected to create two subproblems such that part of the feasible region is discarded without eliminating any of the feasible integer solutions. The process is repeated on all variables with fractional values until an integer solution is found, see Bealie (1979), Beasley (1996), Mitchell and Lee (2001), Taha (2004) and Winston (2004). In this paper, the standard B & B refers to that version of the B & B algorithm proposed by Dakin (1965) and we assume that there are no state-of-art branching rules, cuts or pricing.
In the following, several problems have been presented where B & B performance is poor.

Complexity of the standard B & B method with numerical illustrations
The worst case complexity of the B & B algorithm is discussed for the LIP, which is NP Complete. The number of sub-problems can easily reach unmanageable levels even for very small problems. In this a 1 x 1 + a 2 x 2 + … + a n x n ≤ or ≥ b, section, we present some classes of the LIP models that cause serious challenges for the standard B & B algorithm.
such that: where x j = 0 or 1∀j and n is even.
The behaviour of the standard branch and bound method for n = 4, 6, 8, 16 and 40 is given in Table 1.
Class 2: Step pattern formed by negative signs.
The second class is the integer problems that have constraints with negative signs forming a step pattern. Also, the coefficients in the objective function as well as the constants on the right-hand side of the constraints are significantly different. The standard B & B algorithm on this class of problems can behave in a very bizarre way if the branching is not properly managed. The following three numerical illustrations 3.2.1-3.2.2 were taken from Kumar et al. (2007). Numerical illustration 3.2.3 was slightly modified.
Numerical illustration 3.2.1 such that: Note the step formed by the minus signs and the fluctuating values of constants in the right-hand side. Using the standard B & B algorithm, it generated 209 sub-problems to verify optimality. The optimal solution is: x 1 = 590, x 2 = 544, x 3 = 897 and Z = 10, 769.
Numerical illustration 3.2.2 such that: Using the standard B & B algorithm, it generated 605 sub-problems to verify optimality. The optimal solution is: The standard B & B requires 2101 sub-problems to verify optimality, which is given by: The other classes of linear integer problems that make the B & B an unreliable method are presented in Classes 3 to 7.
Class 4 has an alternate form also as was the case in Class 1. This alternate form will have the objective function as given below and constraints remain unchanged. This alternative objective is For the Class 4 problem, the behaviour of the B & B method for n = 4, 6, 8 and 16 becomes worse as given in Table 2.
Class 5: Knapsack problem: A pure integer case.
Mere changing of variables from binary to pure integer makes any LIP worse for the standard B & B algorithm. For example, consider: such that: where x j ≥ 0 and integer ∀j, 1 ≤ ≤ n − 1, is odd and n is even.
Once again, an alternate problem for class 5 is when objective function is changed but constraints remain unchanged. This alternative objective function is given by: The computational behaviour of the standard branch and bound method for n = 4, 6, 8 and 16 is given in Table 3.
Class 6: Hard Knapsack Problems.  The following general integer model is also very difficult to solve by the standard B & B algorithm on its own. such that : where ∀j, 1 = 1 or some large number. There must be one large number in the objective row and ∀ij, a ij = 2 or some large odd number. There must be one large odd number in every row and in every column. Further, it must satisfy i1 + i2 + … + in < i ∀i.
Minimize Z = 1 x 1 + 2 x 2 + … + n x n 11 x 1 + 12 x 2 + … + 1n x n ≥ 1 21 x 1 + 22 x 2 + … + 2n x n ≥ 2 ⋮ 1m x 1 + m2 x 2 + … + mn x n ≥ m Minimize Z = x 1 + 141x 2 + x 3 55x 1 + 2x 2 + 2x 3 ≥ 73 2x 1 + 91x 2 + 2x 3 ≥ 97 2x 1 + 2x 2 + 85x 3 ≥ 99 There are so many other classes of difficult integer models that occur in real life. These include the travelling salesman problem (TSP) and the generalized assignment problem (GAP). At the moment, it may be very difficult to come up with an efficient general purpose algorithm. Instead, it makes sense to study one class and then propose an efficient way of solving it. In this paper, the KLIP is targeted because of its special features.

Reformulation of the knapsack problem
The knapsack problem has special features that we can take advantage of for reformulation. This problem has only one constraint and all coefficients are nonnegative. The coefficients in the constraint of any KLIP can be arranged in ascending order i.e. This is possible for all KLIP models since all coefficients are nonnegative, and since a 1 is the smallest one can rewrite the expression (2) as follows: Let a 1 j = a j − a 1 for j = 2, 3, … , n then (2) can be expressed as given by (3).
Repeating the process given in (3), where smallest coefficient is a 1 2 , we obtain If the process repeated until there is no term left, the transformed constraint will become as shown in (5).
Note, y j are integer valued quantities and j = 1, 2, … , k The KLIP becomes such that: (2) a 1 x 1 + a 2 x 2 + … + a n x n where a 1 ≤ a 2 ≤ … ≤ a n .
The standard B & B method takes 231 sub-problems to verify the optimal solution of (8), which is Note the coefficients, when arranged in an increasing order are given by 12, 18, 20, 24 and 29.
Phase 1: Where y j ≥ 0 and integer ∀j . In phase 1, there is no integral restriction on the variables x j . The number of sub-problems necessary to verify the optimal solution reduces to 37, resulting once again in the solution given by: Such that : where x j ≥ 0 and integer ∀j.
This KLIP is a special case from Class 6. The number of sub-problems reduces from 7449 to only 5 after reformulation. Optimal solution was obtained in Phase 1.

Numerical illustration 4.3
such that: where x j ≥ 0 and integer ∀j and n = 16. This KLIP comes from Class 1 but in this case, variables are integer and not necessarily binary. The number of sub-problems reduces from over 30, 000 to only 11 after reformulation and the optimal solution was obtained in Phase 1.

Computational experiments
Fifty randomly generated knapsack problems of different sizes were used in the analysis. The objective of the computational experiments was to determine whether the number of sub-problems decrease after the reformulation. The computational results were tabulated as given in Tables 4 and  5. MATLAB R2013 (version 8.2) running on an Intel Pentium Dual desktop (Dual core G2020 2.9 GHz CPU, 2GB DDR3 1333 RAM) was used for the computational experiments. In all the fifty cases, it was observed that the number of standard B & B sub-problems decreased significantly after reformulation, as can be seen from the results in Tables 4 and 5.

Conclusions
What has emerged from the computational experiments is that the number of standard B & B subproblems required to verify optimality can be significantly reduced by reformulation. In all the fifty cases analysed, the number of sub-problems were significantly reduced after reformulation. So many improvements have been done on the branch and bound algorithm in terms of addition of cuts to get the branch and cut algorithm (Brunetta, Conforti, & Rinaldi, 1997;Mitchell, 2001;Padberg & Rinaldi, 1991), pricing to get the branch and price algorithm (Barnhart, Johnson, Nemhauser, Savelsbergh, & Vance, 1998;Salvelsbergh, 1997) and also combining the two improved versions to get the branch cut and price hybrid algorithm (Barnhart, Hane, & Vance, 2010;Fukasawa et al., 2006;Ladanyi, Ralphs, & Trotter, 2001). Here, in this paper, we have we have achieved improvement in the standard B & B algorithm on its own before combining it with other approaches. In addition to using cuts and pricing within the context of a branch and bound algorithm, preprocessing given in Savelsbergh (1994) can reduce the number of sub-problems needed to verify optimality. Reformulation proposed in this paper significantly reduces the number of B & B sub-problems required to verify optimality.
In subsequent publications, attempt will be made to: (1) Extend the proposed constraint reformulation to the general LIP model, when applicable.
(2) Search for the mathematical reasons that give rise to the efficiency observed.
(3) Extend the reformaulation concept to other situations.