A Rough Sets based modified Scatter Search algorithm for solving 0-1 Knapsack problem

Article history: Received December 10, 2014 Received in revised format: January 4, 2015 Accepted February 20, 2015 Available online February 24 2015 This paper presents a new search methodology for different sizes of 0-1 Knapsack Problem (KP). The proposed methodology uses a modified scatter search as a meta-heuristic algorithm. Moreover, rough set theory is implemented to improve the initial features of scatter search. Thereby, the preliminary results of applying the proposed approach on some benchmark dataset appear that the proposed method was capable of providing better results in terms of time and quality of solutions. Growing Science Ltd. All rights reserved. 5 © 201


Introduction
0-1 Knapsack Problem (0-1 KP) is a special case of general 0-1 linear problem in which the allocation of items to a knapsack is discussed.Knapsack problems appear in real-world decision-making processes in a wide variety of fields such as production, logistics, distribution and financial problems (Marchand et al., 1999;Kellerer et al., 2004;Gorman & Ahire, 2006;Wascher & Schumann, 2007;Granmo et al., 2007;Nawrocki et al., 2009;Vanderster et al., 2009).Dantzig (1957) is believed to be the first who introduced the knapsack problem and proved that the complexity of this problem is NPhard (Garey & Johnson, 1979).There are literally many practical applications for knapsack problem and it has become the object of numerous studies and a great number of papers have been proposed for solving this problem.In this problem, different items with various profits (p) and weights (w) are considered.There is also capacity limit in knapsack problem (C).The general KP model is a binary problem, stated as follows: x {0,1} , i 1,..., n In this problem xi is a binary variable, which is one, when the item i is chosen and zero, otherwise.
In this paper, a scatter search methodology evolved by Rough Sets theory to solve 0-1KP for problems in different sizes, is developed.The proposed model of this paper takes advantage of rough sets theory, which relies on Meta models to reduce the objective function evaluations.
The rest of the paper is organized as follows: in the next section, the literature review on 0-1KP is presented.Brief description of scatter search and rough sets theory are presented in section 3. Section 4 describes the proposed RSSS approach for optimization of 0-1KP in more detail and experimental results are presented in section 5. Finally, the advantages of the proposed structure, conclusions, and possible future work are discussed in section 6.

Literature review
Over the last few decades, there have been extensive studies on the 0-1KP.Martello et al. (2000) presented a survey of recently developed solution approaches for 0-1KP.Previous studies of 0-1KP have been predominantly focused on the profit decision under certain conditions.In the context of the research reported have, research work dealing with the mathematical modeling and solving approach aspects of 0-1KP is presented.A majority of 0-1KP problems have been studied using operations research techniques classified in two categories: (1) Single objective and (2) Multi objective models, but most of the studies are based on single objective models.From the late of 20th century, the 0-1KP has captured interests of many researchers.Albano and Orsini (1980) were among the early researchers address the 0-1KP with single objective of profit.They developed a tree search approach to solve the 0-1 KP.Lin and Chen (1994) proposed a systolic algorithm for solving KPs.Pisinger (1995) applied an expanding core algorithm for the exact 0-1KP.Wang et al. (1999) suggested a chaotic annealing neural network with gain sharping for solving 0-1KP.Martello et al. (1999) applied dynamic programming and strong bounds for 0-1KP.Yamada et al. (2005) studied combination of the knapsack problem and the minimum spanning tree problem and developed an undirected graph for solving this problem.Wilbaut et al. (2008) stated a survey of effective heuristics and their applications to a variety of KP.Belgacem and Hifi (2008) proposed sensitivity analysis of the optimum to perturbation of the profit of a subset of items in the 0-1KP.Lin (2008) studied the KP with imprecise weight coefficients using genetic algorithm.Chunha et al. (2010) proposed a new Lagrangian based Branch and Bound algorithm for the 0-1KP.Archetti et al. (2010) proposed α-approximation algorithm bounds for solving 0-1KP.Kaparis and Letchford (2010) proposed exact and heuristic separation algorithms for 0-1 KP.Yang and Wang (2011) presented a rough set based genetic algorithm for solving the 0-1KP.Lin et al. (2011) suggested an exact algorithm for the 0-1KP with a single continuous variable.Zou et al. (2011) applied a novel global harmony search algorithm for solving the 0-1KP.Layeb (2011Layeb ( , 2013) ) proposed two hybrid approaches based on Harmony Search algorithm and Cukoo Search Algorithm with Quantum Computing (QC) for solving 0-1KP.Lasserre and Thanh (2012) solved 0-1 programs (such as 0-1KP) using a joint + marginal heuristic.Bansal and Deep (2012) proposed a modified binary particle swarm optimization for KP and report promising results.Guler et al. (2012) presented algorithm with guarantee value for KPs.Truong et al. (2013) applied chemical reaction optimization algorithm with greedy strategy for the 0-1KP.Zhang et al. (2013) proposed amoeboid organism algorithm for solving 0-1KP.Bhattacharjee and Sarmah (2014) suggested a shuffled frog-leaping algorithm for solving 0-1KP.Kulkarni and Shabir (2014) proposed Cohrt intelligence algorithm for solving 0-1KP.
Regarding these variations, it is clear that each researcher has studied the KP from different aspects.
For instance, there could be environmental or social concerns as well as economic goals.Moreover, in some cases, KP problem can be studied in the context of portfolio and logistics problems.Silva et al. (2006) proposed a scatter search method for bi-criteria 0-1KP.Silva et al. (2006) stated the core concept in bi-criteria 0-1 KPs.Beausoleil et al. (2008) suggested multi-start and path re-linking methods to deal with multi objective KPs.Taniguchi et al. (2009) applied a virtual pegging approach to the max-min optimization of the bi-criteria KP.Kumar and Singh (2010) proposed an assessing solution quality of multi-objective 0-1 KP using evolutionary and heuristic algorithms.Sato et al. (2012) applied Variable Space Diversity, crossover and mutation in multi objective evolutionary algorithm for solving manyobjective KPs.Lu and Yu (2013) proposed an adaptive population multi objective quantum-inspired evolutionary algorithm for multi objective KPs.

Scatter search
Glover (1998) is believed to be the first who presented Scatter Search.Unlike other similar algorithms, Scatter Search uses results to search in solution space purposefully and deterministically.This algorithm, directs vectors with a set of solutions called Reference Set (RefSet) and obtains optimal solutions from the prior solutions.RefSet includes a set of solutions that have both variety and quality that an algorithm needs them for covering the whole solution space for getting near the optimum solution (Glover, 1998).
Scatter Search methodology has been best known for its flexibility in solving variety of problems.Even the internal sub methods of it are very flexible.Consequently, this procedure can be implemented to solve different problems in various scales.In this section, we propose a general description of Scatter Search steps, but these five steps will be adapted to the proposed problem in the next sections.Note that the complexities of these methods are being "changed" and not only "reduced or expanded" according to the problem.The general descriptions of these steps are as follows: 1) Diversification Generation Method: Generates diverse solutions using a random (or a set of random) solutions called seed solutions.2) Improvement Method: Transforms the diverse solutions produced in the prior method into more qualified solutions (neither the input nor the output solutions are required to be feasible, but it is rather that outputs be feasible) 3) Reference Set Update Method: keeps the "best" solution in the RefSet (in the first place and during the algorithm).Meaning that, the "best" solution satisfies both quality and diversity of solutions.4) Subset Generation Method: Produces a subset of RefSet as a basis for Combination Method.5) Solution Combination Method: transforms the members of subset generated in the prior step to new solutions by combining solution vectors.

Rough sets theory
Rough Sets theory is a new mathematical approach to imperfect knowledge that is proposed by Pawlak (1992) and works on vague and imprecise environments.The theory works on the notion of sets and the relationships among them.The Rough Sets theory has its basis on the information that we have about every member of universe, the collection of objects we am interested in, and tries to present a way to transform data to knowledge and gives us a useful method for discovering hidden patterns in the raw data (Mrozek, 1992).
The main privilege of Rough Sets theory is that, there is no need for additional knowledge such as probability in statistics or membership grade in fuzzy logic.The basis of theory stands on the notions of "upper and lower approximations of set" and set modeling (Slowinski, 1992).In this method, the superfluous data in aspect of reasoning will be removed and the indispensable data will be on board to derive the decision rules from them.We now describe the fundamental concepts of Rough Sets theory that is used in the proposed methodology in this paper.

Relational Systems (Knowledge Base)
Suppose that there is a finite set U ≠ ∅ (the universe) of subjects, we are observing or interested in.Any subset X U ⊆ of the universe is called a concept or a category in U and any family of concepts in U will be referred to knowledge about U. If R is an equivalence relation over U, then by / U R , we mean that family of equivalence classes of R referred to as categories of R and

If
Thus / ( ) U IND P or in short / U P , denotes knowledge associated with the family of equivalence relations P, called P-basic knowledge about U. In fact, P-basic categories have those of basic properties of the universe, which can be expressed employing knowledge P.

Approximation of sets
As we have demonstrated before, some categories cannot be expressed exactly by employing available knowledge.By Rough Sets theory, we are able to make an approximation of a set by other sets.These sets are called lower and upper approximations.The lower and upper approximations can be presented in an equivalent form as follows The set X R is the set of all elements of U, which can be certainty classified as elements of X, in knowledge R. The Set X R is the set of elements of , U which can be possibly classified as elements of X, employing knowledge R and Set ) X ( BN R is the set of elements, which cannot be classified to X or to -X having knowledge R.
The positive region or lower approximation of X is the collection of those objects that can be classified with full certainty as members of the set X, using knowledge R. Similarly, the negative region ( ) R NEG X is the collection of objects with which it can be determined without any ambiguity, employing knowledge R that belongs to the compliment of X (Pawlak, 2002).Fig. 1 shows a schematic overview to the notion of approximations.

Proposed algorithm
Scatter Search is a meta-heuristic algorithm that applies a global search in solution space with a set called RefSet.Three methods of five general scatter search methods depend on the type and complexity of the problem including Diversification Generation Method, Improvement Method, and Combination Method.Therefore, it is essential to introduce the problem I am going to solve so that I can justify three methods referred.

Diversification generation method
As mentioned before, in Scatter Search methodology first step is to generate a set of diverse solution with a random mechanism.To this end, we applied a systematic mechanism proposed by Glover (1977) to produce a diverse set of binary vectors.First consider max 2,3,..., h h = , which max 1 h n ≤ − , which must be assigned a value of one or a set of seed solutions to build up the other answers on it.Our seed solution is a vector that picks zero for every array of it."div" number of random solutions is being produced here and they are presented by ' x that: As it can be noticed, the proposed method creates " 2 div " of diverse solutions and other " 2 div " will be the complement of these solutions so that: '' ' Now, we have our diverse solutions that they might be feasible or not.Finally, in the last step of this method, we make them feasible by a random technique.We randomly choose variables that accepted one and switch them to zero until the solution becomes feasible.Therefore, the idea of random solution creation in Diversification Generation Method will stay untouched.

Improvement method
We apply this step to enhance the quality of provided answers.First, the proposed method calculates a ratio as a criterion for each variable's importance in solution quality.As the approach is black box, so we just used objective function evaluation for computing ratio as below which i x represents the decision variable that picks 0 or 1 in the diverse solutions.

{ | 1}
f is the average of objective function in " 2 div " of solutions that have a better objective value when the i'th decision variable picks 1. i B f is the average of objective function in " 2 div " of solutions that have a better objective value when the i'th decision variable picks 0.
represents weight coefficient or the coefficient of i'th variable in constraint.
Consequently, we have a ratio for each decision variable, which helps us decide about every variable.However, the procedure goes like this that in first step in case of infeasible solutions; the proposed study sorts the ratios of variable and variables with values of one with the least ratio transform to zero and this procedure continues until the solutions become feasible.In the next step, variables with value of zero with the most ratio are switched to one.The feasibility is required in both conditions.

Reference Set Update Method
This method first sorts the solutions according to their objective function values.As mentioned in general scatter search description, RefSet is a set of "b" (

Subset generation and Combination method
We simply choose two of solutions using permutation for combining in next method.Suppose we have two solutions from RefSet selected by Subset generation method.For creating new solutions, we assign a value of 1 for variables that have the value of 1 in both picked solutions and pick 0 or 1 randomly for the rest of variables.To make new solution, the feasibility is a required condition.In the next step, the Improvement Method improves generated solutions and their objective function values will be compared with the members of 1 b and in the case of improvement in new solutions, the replacement will be settled with the worst member of 1 b .For diverse solutions, we compare the distance of new solutions from the 1 b with the temporary members of 2 b and in case of improvement in diversity, new solutions will be substituted.As it can be noticed, this method is a loop and the algorithm ends when b is not updated during a loop.In this condition, new Population will be produced to run the entire method again.

Pop size update with rough sets theory
As mentioned before, if b is remained unchanged, the population will be reproduced and all of methods will be repeated on new set of solutions in population but in case of speed and quality of obtained solutions, the population update can be very important.To this end, we use Rough Sets Theory for updating Population and the idea is described as follows.
Suppose that we have an n sized problem, meaning the objective function has n variables, say 1 2 , ,..., n x x x .After a loop, we have an imprecise knowledge about every variable's effect in objective function and the knowledge is obtained from the best solutions in b1, so in Rough Sets Theory terms, if my universe is that classifies b1members to two equivalent classes, respectively, variables which picked value of one in b1, and variables which picked value of zero in b1.Therefore, for every solution we have a Relational System as follows, ( , ) / i U R denotes knowledge associated with the family of equivalence relations In other words, the equivalence relations of i R are the basic concept of universe that can be represented in i R .To illustrate the idea we will serve an example.
Consider that we have 10 decision variables (n=10), 1 3 b = which is as follow, By classification of (1 or 0), we define three equivalence relations These are elementary categories (concepts) in the knowledge based Theoretically, basic concepts are sets that are intersections of elementary categories, so we have According to the set of variables that values are one we have: {1,1,..,1} X = (10) As we can notice, some categories ( 1 2 5 9 10 , , , , x x x x x ) cannot be expressed exactly by employing available knowledge.Consequently, a rough set arrives at the idea of approximation of a set by other sets.Let us define X ⊆ U as a set of variables where all of them having the value of one, considering the general tendency of algorithm for maximizing the objective function value.With each subset X ⊆ U and an equivalence relation S, we associate the subset S X .As demonstrated before, according to the obtained knowledge, the variables 3 4 { , } x x can certainly classified as elements of X, in the knowledge S. Now we can calculate 6 7 8 ( ) { , , } S NEG X x x x = that refers to variables, which cannot be a member of set X, in the knowledge of S.
We use this procedure for updating Population.First, we put the b members of RefSet in new Population, then by applying rough Sets Theory data reasoning on 1 Re fset , we calculate the RX and pick value of one for the members of RX and oppositely pick value of zero for the members of ( ) S NEG X .Other variables, in terms of rough Sets Theory, are called ( ) S BOUNDRY X pick 0 or 1 with a random procedure.Now we have a new Population that the whole Scatter Search must be implemented on.

Experimental results
We have coded the proposed Rough Sets based modified Scatter Search (RSSS) algorithm in Matlab on an Intel Core i3, 1.736 Ghz processor with 4Gb RAM.To evaluate the performance of RSSS algorithm, two sets of Ill-known benchmark problems in the 0-1KP literature have been considered: (1) Small sizes: The data set consists of ten collected instances [f1-f10] from Zou et al.(2011) with number of items ranging from 4 to 23 and ten instances [f11-f20] from Kulkarni and Shabir (2014) with number of items ranging from 30 to 75.
(2) Large-scale sizes: The data set consists 96 problems from Pisinger (1995) with a variety of variable sizes (n= 100, 300, 1000, 3000).To cover all kinds of possible conditions, the problems have been categorized into different classes.We have run this algorithm 20 independent times for each instance from Zoa data and 10 independent times for each instance from Pisinger data.

Results of small and medium size problems
The first data set under study includes small-scale problems.The proposed model can be optimally solved within noticeable time for small instances.Thus, to solve these problems, we have considered a global optimum value of objective function (GOV) obtained from branch and bound algorithm (B&B) to compare with the solution of the RSSS algorithm.

Avg(|BSRSSS-MSRSSS|)= Average of the absolute difference between BSRSSS and MSRSSS.
The obtained results and comparison of B&B and RSSS algorithms corresponding to the 20 problems are shown in Table 2.
In Table 3, the first column symbolizes the name for each instance; the second column shows the information of instance; the third column lists the global optimum solution (GOV) and the remaining columns describe the computational results of RSSS, CI, MDSFLA, NGHS, QICSA and QIHSA respectively.

Results of Large scale problems
The second data set investigated is associated with Pisinger data (Pisinger 1995).We have compared the results of proposed RSSS algorithm with commercial software's like OptQuest, Solver, Evolver and the methods proposed in Gortázar et al. (2010) called BinarySS and Pisinger (1995) called Expknp.In a similar vein, we have used the range of data sets as R = 100, 1000, 10000 and used Pisinger's exact method, which uses the objective function coefficients as a best-known value solver.By solving the problems with various variable numbers (n) and proposed ranges (R), the number of feasible solutions in each method is shown in Table 4.As far as the knowledge of researchers is concerned, one of the features of algorithm quality is the number of objective function evaluations that solver uses to find the best solution.In Table 5, the numbers of function evaluations for the problems in different sizes are proposed.It is obvious that by using the Rough Sets theory, the superfluous evaluations have been eliminated and there are a reasonable number of objective function evaluations.
Comparing to the last research in this field, there is a great progress in our results regarding the fact that we used only one Diversification Generation Method and one Combination Method while in Gortázar et al. (2010), three Diversification Generation Methods and seven Combination Methods were used and we obtained better results.Best-known values and results of commercial software such as OptQuest, Evolver, Solver and ExpKnap, Binary SS and RSSS algorithms are shown in Tables 6-17.The results of Fig. 2 demonstrates the calculated APD for these three methods in various variable numbers.In fact, this figure shows the deviation of obtained solution by each method from the bestknown value, in which the efficiency of RSSS is obvious.

Conclusions
This paper has proposed a Scatter Search based method for solving 0-1KP in different scales and ranges, which has a specific approach called RSSS.To achieve a more efficient, a Rough Sets Theory concept of "lower and upper approximations" has been combined with two main procedures of Scatter Search.
We have also studied two sets of Ill-known benchmark problems and solved different sizes of them, which were categorized into several types regarding the coherency of objective function coefficients and constraint coefficients.For the first data set, results were compared with Zou (2011) and for second data set, both feasibility and optimality of commercial software's and results of Gortázar et al. (2010) have been studied.As it is demonstrated in results, the proposed method is more practical and successful in solving 0-1KP.In first data set, the RSSS method almost have similar behavior with NGHS and the APD calculated on second data set between best-known values and our obtained solution is less than comparable methods, especially there is a considerable gap between our idea and Commercial software packages.We expect that our idea of combining Rough Sets as a powerful tool to improve meta-heuristic features will help researchers discover more efficient methods for solving optimization problems.
that 1 b number of them are solutions with high quality and 2 b number of solutions are the most diverse solutions in solution space.Because of sorting these solutions, 1 b is updated with the solutions that have the best objective function value.In this step, the ratio is updated based on the new members of 1 b .Next step is to update 2 b .For maximum diversity according to high quality solutions; we define a distance function that calculates the "distance" of two solutions and by using the function.Regarding these values, we will be able to figure out which solutions in population have the maximum distance from the members of 1 b .The proposed study calculates the minimum distance of every solution in population with the members of 1 b and solutions that have the maximum of minimum distances will be chosen to fill 2 b .

Fig. 2 .
Fig. 2. The value of APD for each number of variables

Table 1
RSSS parameter settings

Table 4
Number of feasible solutions that solved

Table 5
Number of function evaluations by RSSS

Table 18
The average GAP of several methods According to the results of Table18, it is obvious that the three methods, RSSS, BinarySS, and OptQuest have provided more qualified solutions.For comparing the efficiency of these methods, we have defined Absolute Percent Deviation (APD), which divides the GAP to the best-known value for each problem as follows,