Reducing calibration effort for clonal selection based algorithms: A reinforcement learning approach

Abstract

In this paper we introduce (C, n)-strategy which improves the former C-strategy for on-line calibration of Clonal Selection based algorithms. In this approach, we are focused on a trade-off between the intensification and the diversification of the algorithm search. By using our approach, it allows us to reduce the number of the parameters of the algorithm respecting both the original design of the algorithm and its performance. The number of selected cells and the number of clones are dynamically controlled on-line, according to the algorithm’s behavior. We report statistical comparisons using well-known clonalg based algorithms for solving combinatorial optimization problems. From the tests, we conclude that the tuning effort for Clonalg based algorithms is strongly reduced using our technique. Moreover, the dynamic control does not decrease the performance of the original version of the algorithm. On the contrary, it has shown to improve it.

Introduction

When we design a bio-inspired algorithm to address a specific problem we need to define a representation, and a set of components to solve the problem. We also need to choose parameter values which we expect will give the algorithm the best results. This process of finding adequate parameter values is a time-consuming task and considerable effort has already gone into automating this process [14], [24], [29]. Researchers have used several methods to find good values for the parameters, as these can affect the performance of the algorithm in a significant way. The best known mechanism to do this, is tuning parameters on the basis of experimentation, something similar to a generate and test procedure [2], [16], [22]. Considering that an immune algorithm run is an intrinsically dynamic adaptive process, we can expect that a dynamic adaptation of the parameters during the search could help to improve the performance of the algorithm. Dynamic adaptation of parameter values has shown to be able to improve the performance of other metaheuristics [4], [8], [30], [36], [10], [21], [23].

The idea of adapting parameters during the run is not something new, but we need to manage a trade-off between the improvement of the search and the adaptation cost. In this case, the idea is to monitor the search to be able to trigger actions for an adaptive parameter control strategy, in order to improve the performance of the well-known immune algorithm clonalg [5].

We present a brief revision of the related work in parameter control domain in the next section. The algorithm clonalg is briefly described in Section 3 with remarks on the aspects of parameter control. We propose a method in Section 4 which includes a monitoring and an updating procedure. We have tested the algorithm using our strategy with instances of the traveling salesman problem and of the university course timetabling problem. The aim of these tests is:

• To evaluate if using our control strategy the immune algorithm does not decrease its performance but requires less time for tuning.

• To discriminate which parameters are really required when the immune algorithm uses our control strategy.

Statistical analysis of these results are reported in Section 5. The conclusions, in Section 6, confirm our experience from this study and the following trends of this research. It is important to note that our goal is not to present the new best algorithm that solves hard combinatorial problem instances. Our research is focused on proposing strategies for controlling parameters respecting, as much as possible, the original design of the algorithm proposed by the authors, without decreasing its performance.