A Heuristic for a Mixed Integer Program using the Characteristic Equation Approach

While most linear programming (LP) problems can be solved in polynomial time, pure and mixed integer problems are NP-hard and there are no known polynomial time algorithms to solve these problems. A characteristic equation (CE) was developed to solve a pure integer program (PIP). This paper presents a heuristic that generates a feasible solution along with the bounds for the NP-hard mixed integer program (MIP) model by solving the LP relaxation and the PIP, using the CE


Introduction
An integer linear program is a linear program which is further constrained by integer restrictions on some or all variables. When all variables are integer restricted, it is called a pure integer program (PIP) model and when only some of the variables are restricted to integer values, it becomes a mixed integer program (MIP) model. Integer programming (IP) models frequently arise in human resource planning, facility location, assignment problems, production planning, time-tabling, warehouse location, scheduling and capital budgeting, just to mention a few. While most linear programming (LP) problems can be solved in polynomial time, PIP and MIP are NP-complete problems, which have no known polynomial time algorithm to solve them. In this paper, the PIP is solved by using the characteristic equation (CE) with the hope that this approach may provide insight into MIP solution procedures and applications. Generally, MIP problems have been solved

Branch and Bound
The approach for tackling a MIP is the branch and bound (BB) algorithm, which relies on the iterative solution of the LP relaxation. The reason for dropping integer constraints is to be able to use LP general-purpose solution methods (Kaibel, 2011). The standard way to represent a choice between n alternatives in a mixed integer programming model is through n binary variables that add up to one. Unfortunately, this approach commonly leads to unbalanced BB trees and diminished solver performance (Dey and Vielma, 2010).

Branch and Cut
The branch and cut (BC) algorithm incorporates the cutting-plane algorithm in the BB scheme, and the idea behind integrating the two algorithms is that LP relaxations do not naturally well approximate the convex hull of mixed integer solutions to MIPs (Jozefowiez et al., 2012). Research in this area is continuing, along with the development of sophisticated new software packages that incorporate these new techniques.

Decomposition Techniques
Decomposition techniques are another good alternative to apply exact optimization methods to solve large integer programming formulations. The basic idea here is to transform or split the problem into smaller sub-problems. This technique has been applied in various problem domains. For example, Benders' (1962) decomposition was used to produce solutions for large instances of the aircraft routing and crew scheduling problems (Laesanklang et al., 2015). Decomposition techniques have also been applied within heuristic approaches using some form of clustering. For example, Jozefowiez et al. (2012) tackled a large vehicle routing problem by decomposing it into sub-problems. The sub-problem size is controlled by splitting a large sub-problem to shrink the corresponding cluster.

Stochastic Programming
Stochastic programming becomes an increasingly popular technique to model decision making under uncertainty. It is able to model uncertainties in a flexible way and impose real-world constraints relatively easily (Sherali and Adams, 1990). Sourd and Spanjaard (2008) proposed a novel decomposition method based on the particular structure of the problem concerned. It decomposes the problem geographically into security and bond sub-problems, which are then further broken into smaller sub-problems.

Lagrangian Relaxation
Lagrangian (sometimes spelled as Lagrangean) relaxation is a popular relaxation technique to compute bounds on the optimal solution of a MIP (Sridhar, 2014). These algorithms are either combinatorial or LP-based. The combinatorial algorithms include greedy algorithms, randomized algorithms and reduction to problems with known poly-time algorithms (Sridhar, 2014).

Proposed Method 4.1 Mathematical Development for The Proposed Method
Consider a general mixed integer programming problem where is (1x ), is ( x1), is ( x ) and is ( x1). Let 1 < represent integer restricted non-negative variables and the remaining ( -1 ) variables are such that ≥ 0. In equation (1), 1 X represents integer restricted non-negative variables in , The pure integer programming model of (1) After obtaining the PIP solution to (3), one can also develop a modified LP from the given MIP (equation (1)) when all integer restricted variables are replaced by their values and the problem reduces to ( -1 ) variables, where all these variables are non-negative restricted real variables. Here onwards, we will call it a modified LP model.
The three problems (1), (2) and (3), and the modified LP have close relationships among themselves, which can be used to develop a method for solving the MIP model (1). Some of these relationships that may be of immediate interest to us are discussed next.

Observation 1
The LP relaxation model (2) is a least constrained model among models (1), (2) and (3), hence the LP optimal solution of model (2) will be an upper bound to the MIP model (1).

Observation 2
A feasible solution to the PIP model (3) will also be feasible to the MIP model (1). Proof -Since a feasible solution X to model (3) will also satisfy the requirement for the MIP model (1), where 1 of the variables are required to have integer restricted values, it then follows that International Journal of Mathematical, Engineering andManagement Sciences Vol. 2, No. 1, 1-16, 2017 ISSN: 2455-7749 5 all feasible solutions to the PIP model (3) will also be feasible to the MIP model (1). Therefore, the PIP optimal solution will act as a lower bound to the MIP model (1).

Observation 3
The MIP model involves two-way distribution of resources. the first distribution of resources involves the division of the resource vector b among the integer and continuous variables and the second distribution gives rise to values of the basic variables. the distribution of the resource vector b is achieved by determining ordered optimal solutions to the PIP model. These ordered optimal solutions can be obtained by using the characteristic equation developed by Kumar et al. (2007), and Kumar and Munapo (2012). The distribution of resources to continuous variables is obtained by the LP.

Observation 4
The optimal PIP solution has a property that if = is an element of this optimal integer solution, then = + 1 along with other variables at their optimal values will always lead to an infeasible solution.

The Method
In the proposed approach, for the given MIP, we first develop two parallel LP and PIP models. The optimal value of the relaxed objective is denoted by LP Z which acts as the upper bound to the given MIP. For any optimization problem Z, let F(Z) denote its set of feasible solutions. The only requirement for LP which is true in this case. The relaxed problem is easier to solve than the original problem, and the LP Z gives the upper bound (Geoffrion and Marsten, 1972). From the optimal solution of LP Z a characteristic equation (CE) is formed to resolve the PIP model (3), where all variables are restricted to integer values. The advantage of the CE is that it can provide the best, second best, third best solutions, etc. for the PIP model (3). The optimal PIP solution acts as the LB to the given MIP model. The gap between the UB given by LP Z and the LB given by PIP Z can be decreased by the LP modified model that can be formulated by substituting values of the integer restricted variables in the MIP model (1) and solving the remaining model as a LP model in ( -1 ) variables. The combined solution will be a feasible solution to the MIP model (1). This will give rise to an improved LB, denoted by . If is insignificant or is zero then either the optimal solution or near optimal solution to the MIP is obtained.

The Characteristic Equation
The characteristic equation (CE) is obtained from the final table of the LP relaxation. It is a mapping of the integer hyper-plane on feasible integer points and the mapping of the objective function on interior integer points in a descending order.
The CE is based on the following three basic ideas (i) The objective value must be an integer.
Non-basic variables are either zero (as in the LP solution) or if some of them are not zero, they must be an integer quantity.
Basic variables are also functions of the non-basic variables, and for the non-zero nonbasic variables, the basic variables must also become integer values in a PIP model.
The major advantage of using the CE is that convergence is guaranteed and it can be used to obtain ordered optimal solutions for the PIP problem. However, the solution of the CE can be a challenge, and also if there are more than one solution to the CE, all those solutions have to be tested for integer solutions. The LP extreme points are an intersection of at most m constraints and the LP optimal solution is an intersecting point of one more hyper-plane represented by the objective function. The CE is a mapping of the hyper-plane on feasible integer points. From the optimal relaxation LP solution, one can write the objective function row as where k represents the number of non-basic variables, j  represents the integer coefficients, D is the lowest common factor for all terms and R is the remainder in the RHS value. The CE is then given by When the LHS is equal to the RHS, the objective function Z is guaranteed to be an integer value. Since we are looking for an integer solution to the CE, we also got the condition (2) in Section 4.3 satisfied. For the solution to be an acceptable feasible solution, the solution of the characteristic solution must also convert the values of the basic variables to non-negative integer values. The next best solution is obtained by further reduction in the value of the objective function, which is possible by increasing the value of .
It may also be noted that when the PIP model (3) is solved for the optimal solution, all variables have non-negative integer values. We substitute from this solution, values of 1 integer restricted values in the MIP model (1) and get a modified LP in ( -1 ) real variables, which is a LP model and can be solved by any known method. The combined integer restricted values from the PIP and LP solution for the modified LP gives a feasible solution to the MIP, hence acts as a LB.

Algorithmic Steps of the Method
The method is comprised of the following steps.
Step 1. Solve the relaxed LP model (2), find the value of LP Z which will be an UB to the given problem (1).
Step 2. Obtain the characteristic equation from the solution of LP Z . Set k =1.
Step 3. Solve the characteristic equation and obtain kth best integer solution for minimum . This will be a solution to the PIP, i.e. when all are integer restricted variables.
Step 4. The PIP solution from Step 3 will be a lower bound to the given MIP. Let this LB be denoted by PIP Z .
Step 5. If   PIP LP Z Z  equal to zero, or approximately equal to zero go to Step 10. Else go to Step 6.
Step 6. Substitute the integer solution for the integer restricted variables obtained from Step 3 into the original MIP model (1) and get a modified LP in ( -1 ) variables.
Step 7. Solve the LP in ( -1 ) real variables obtained at Step 6. Step 8. Combine the integer solution of Step 3 for the integer variables and real solution for the real variables from Step 7 to get a feasible solution to the MIP. This value is likely to be less than the UB and more than the LB and if the difference is insignificant, one can stop the search, else one has to improve the feasible solution. Check if this solution can be declared as the optimal solution. If yes, go to Step 10, else go to Step 9.
Step 9. Set k = k+1 and go to Step 3.
Step 10. Conclude the search process as the optimality condition has been satisfied.

Numerical Illustrations 5.1 Example 1
This example is taken from Hillier and Lieberman (2001).
Using LIPS programme, the LP solution found is shown in Table 1 (6) again as a PIP and for that the CE will be required, which is given by (7) 14 ., ,......... 2 , 1 , 0 12 3 5 17 12 The non-basic variables , 4 x 5 s , 6 s and 7 s have to move from the current value zero to some nonnegative integer value such that (7) remains satisfied for the minimum and also converts the current basic variables from the real values to integer values. From Table 2, the relations between the basic and non-basic variables are given by The only integer solution from equation (9) The above solution can be concluded as an optimal solution to the given MIP, as currently we have 13 ≤ ZMIP ≤ 14.25, and a feasible solution of 13.5. Note that the only variation possible is to increase the values of the variables x1, x2, x3, but the solution will exceed the current UB. Similarly, an increase in x4 will worsen the current solution. Hence an optimal solution has been obtained and the search is terminated.
The above solution can be concluded as an optimal solution to the given MIP, as currently we have 13 ≤ ZMIP ≤ 14.25, and a feasible solution of 13.5. Note that the only variation possible is to increase the values of the variables x1, x2, x3, but the solution will exceed the current UB. Similarly, an increase in x4 will worsen the current solution. Hence, an optimal solution has been obtained and the search is terminated.

Example 2
Consider the following MIP Details of the LP optimal solution are given in the Appendix A, Tables 3 and 4. From the optimal LP solution, the UB for the given MIP is = 2528 57 = 44.351.
The CE for the ZPIP will be given by The non-basic variables have to move from the current value of zero to some non-negative integer value such that (13) The only solution is when all variables are zero and 20 20  S . However, this solution does not lead to an integer solution as basic variables do not satisfy integer requirements. When = 1, the RHS of the CE (13) becomes 77, and the equation (13)   There are no sub-problems as is in the case of branch and bound or branch and cut methods.  The method does not depend on the initial relaxed solution, but uses the relaxed solution as the upper bound.  The method searches the optimal solution using the simplex algorithm, but moves over the integer polyhedron.  This approach is suitable for changes in the input values.  It utilises and makes use of the characteristic equation to adjust the lower bound.

Further Challenges
A solution of a CE can be demanding for larger values of . Further work is required on how to solve a CE. The method uses the concept of ordered solutions, but for most of the ordered solutions, the value of will be high. For high values of , the solution to the CE becomes more demanding.
In the branch and bound approach for a MIP, when a large number of variables are integer restricted, the number of sub-problems can be very high. Sometimes even a feasible solution is not easily obtainable. However, the proposed approach guarantees determination of a feasible solution and also gives its bounds. The proposed approach works better when integer restricted variables are relatively large (see Example 1 above).
The branch and bound technique may work better if integer restricted variables are only a few in number. In fields like machine learning, computer vision, advertising, and statistics, it is quite common to encounter MIP formulations with millions of binary decision variables. These algorithms can yield good results in practice but do not offer any theoretical bounds on runtime and solution quality. The proposed method may prove to be useful for these problems.