Enhancing the Learning Capacity of Immunological Algorithms: a Comprehensive Study of Learning Operators

Immunological algorithms are a kind of bio-inspired intelligence methods which draw inspiration from natural immune systems. The problem-solving performance of immunologi-cal algorithms mainly lies on the utilization of learning (i.e. mutation) operators. In this paper, nine different learning operators in a standard immune algorithmic framework are investigated. These learning operators consist of eight existing operators and a newly proposed search direction based operator. Experiments are conducted based on nine variants of immunological algorithms that use different learning operators. Simulation results on a large number of benchmark optimization problems give a deep insight into the characteristics of these operators, and further verify that the proposed new learning operator can greatly improve the performance of immunological algorithms.


Introduction
Many difficulties such as dimensionality, differentiability and multimodality are associated with the optimization of large scale problems.To address such problems, bioinspired intelligence algorithms (Da Silva Santos et al., 2010;Gao, 2012) have attracted more and more interest, among which the immunological algorithm (IA) is a particular class of optimization methods inspired by the basic features of adaptive immune response to antigenic stimulus.Most IAs mimic the metaphors of clonal selection principle (de Castro and Zuben, 2002), hypermutation (Freitas and Timmis, 2007), receptor editing (Gao et al., 2007) and lateral interaction effect (Whitbrook et al., 2007), providing a promising search mechanism by exploiting and exploring the solution space in parallel and effectively (Dasgupta et al., 2011).The main unique property of IAs is the utilization of the clonal proliferation, and the clonal selection which returns promising solutions acquired in the learning process.It is evident that IAs possess good features of maintaining population diversity, and capability of allocating multiple optimal solutions (Haktanirlar Ulutas and Kulturel-Konak, 2011).Although IAs have achieved good performance in solving various kinds of practical problems, such as digital signal processor (Mitra and Venayagamoorthy, 2010), nonlinear classification (Ozsen et al., 2009), fault diagnosis (Hao and Sun, 2007), etc, their performance is limited in solving optimization problems (McEwan and Hart, 2009).Compared with other bio-inspired algorithms, such as the well-known evolutionary computation (Yao and Xu, 2006), IAs still greatly suffer from the issues of stagnation and slow convergence.The reason seems to be that the learning capacity (involving hypermutation and receptor editing) has not been fully exploited, i.e., no sophisticated learning operator can be found in the literature (Jansen and Zarges, 2011).Based on the above consideration, we review and analyze the existing learning operators commonly used in IAs, and propose a new search direction based learning operator (L sd ) to encourage the antibodies to utilize the information of its surrounding antibodies, by means of moving the antibody toward the nearby antibodies with higher affinities and meanwhile away from the antibody with lower affinities.Therefore, the L sd operator can not only evolve antibodies into promising search areas to accelerate convergence speed, but also prevent antibodies from entering undesired regions to jump out of local optimal solutions.The experiments of using all learning operators in IAs are conducted based on a large number of benchmark numerical optimization problems.The results show the characteristics of each learning operator, and further indicate that the proposed L sd operator manipulates the best performance.

Immunological algorithm
To investigate the effect of learning operators, a standard immunological algorithm framework (called IA) is utilized (de Castro and Zuben, 2002;Kelsey and Timmis, 2003).I-A evolves a population of antibodies (B cells) towards a global optimum through a process of evaluation, cloning, learning (i.e.mutation) and selection.The evaluation procedure computes the affinity function values for all antibodies.Affinity is an important measure to represent the fitness of antibody to antigen.For a minimization optimization problem, higher affinity values of antibodies correspond to better solutions for the problem needed to be solved.The cloning proliferation is a mitotic procedure whereby the cells divide themselves, creating a set of clones identical to the parent cell.Generally, the proliferation rate is directly proportional to the affinity level.The learning procedure (involving hypermutation or receptor editing) performs the exploration and exploitation within solution search space, guiding antibodies to the global optimum.It plays a key effect on the search performance of the algorithm.The selection procedure picks up the antibodies with higher affinities and meanwhile eliminates those with lower affinities to reduce the computational complexity.
From the perspective of optimization, a learning operator L mutate a candidate solution Ab = (x 1 , x 2 , ..., x D ) to a trial one , where D is the dimension of the problem.In the literature, there are several learning operators commonly used in IA.They are summarized in the following.
• Cauchy mutation L cy (Yao et al., 1999;Xu and Zhang, 2007;Khilwani et al., 2008): • Static Hypermutation L h1 (Cutello et al., 2004;Gong et al., 2008): the number of mutations is independent from the affinity of the antibody.That is to say, Ab ′ will undergo a constant number c of mutation times.Each mutation act on Ab is implemented through replacing a certain number of Ab at a random dimension with a random integer between 0 and 9 (Gong et al., 2008).
• Proportional Hypermutation L h2 (Cutello et al., 2004): the number of mutations is proportional to the normalized affinity value, that is, f (Ab) × c × D, where f (Ab) is the normalized affinity distributed in the interval of [0, 1].c is a constant number, representing the maximum mutation intensity.
• Inversely proportional hypermutation L h3 (Cutello et al., 2004(Cutello et al., , 2005(Cutello et al., , 2006)): the number of mutations is inversely proportional to the normalized affinity value, i.e., the higher affinity of an antibody, the less times of mutations will be carried out on it.It is reasonable to make such an inverse choice, since better antibodies usually contain more useful information for evolution.Too many mutations might have higher probability to destroy these information, thus depressing the learning performance.
• Hypermacromutation L m (Cutello et al., 2004): the number of mutations is independent from the affinity and the parameter c.Instead, the operator mutates at most j −i+1 values in the interval of [i, j], where two randomly generated integers i and j satisfy the condition of i < j ≤ D.
• Lateral interaction mutation L li (Cutello et al., 2006;Pavone et al., 2011): in addition to hypermutation and receptor editing, the lateral interaction during different antibodies also takes place according to the idiotypic network theory (Gao et al., 2008).In other words, each paratope on an antibody can not only recognize a foreign antigen, but also can be recognized by external idiotopes.Motivated by this mechanism, similar to the crossover operator in evolutionary computation, an antibody is attracted by other antibodies, i.e., , where Ab j ̸ = Ab i is a randomly selected antibody in the population.
• Baldwinian learning L bl (Gong et al., 2010): learning mechanism can provide an easy evolutionary path towards co-adapted alleles in environments, by means of employing differential information during other antibodies.It is realized as ) in a probability of p, where i ̸ = j ̸ = k, Ab j and Ab k are randomly selected from the population, and s represents the Baldwinian learning strength.
Intuitively, all the above eight learning operators are able to evolve antibodies to matured ones in semi-blind manners, although some of the matured ones might possess lower affinities.However, as the parallel feature of the immune algorithm, there did exist a probability of making progress to improve the affinity of antibodies.After the clonal selection progress, the most improved antibodies are reserved and enter into the next generation of evolution.

Search direction based learning operator
Even though the above learning operators used in IA can explore/exploit the solution space in an effective manner, as we observed, they are not fully developed from the aspect of utilizing the information in environment.In Fig. 1, we summarized the characteristics of the learning operators.The solid rectangle S show the solution space of the optimization problem.The dashed circles denote contour lines of affinity, and the inner circles indicate that they represent higher affinities than the outer ones.
From Fig. 1, we can notice that the learning mechanisms used in ( 1)-( 6) on the antibody Ab only utilize random perturbation on the antibody itself, while those in ( 7)-( 8) make use of information in the environment.As reported in (Cutello et al., 2006;Gong et al., 2010), learning from the environment provides an encouraging alternative method, probably a more easy way to achieve better search performance.In details, the mechanism in (7) uses the information of a randomly selected antibodies in the population to guide the current search.A successful guide is strongly depending on the quality of the selected guiding antibody Ab j , implying that there must be amount of redundant search if the guiding antibody is far away from the global optimal solution.Instead, the mechanism in (8) utilizes the differential information between two other antibodies Ab j and Ab k in the population.The learning acting on this differential information might have ability to use the mutual beneficial components, thus exhibiting more promising properties for searching.
Based on the above analysis, we can find that the mechanisms used in ( 7) and ( 8) don't use any measurable knowledge from the population, i,e, the guiding antibodies (Ab j and Ab k ) are randomly selected without any relevance of the current antibody Ab, therefore hindering the effectiveness of the learning performance.In view of the limitations of the above learning operators, we propose a new search direction based learning operator L sd , not only to provide another alternative mutation method, but also aiming to achieve a better search capacity.The L sd is formulated in Eqs. ( 1)-( 3): In these equations, ∆ repulsion and ∆ attraction are repulsion and attraction effect on Ab i respectively.In a single learning procedure of L sd , if a randomly selected guiding antibody Ab j whose affinity is lower that the base antibody Ab i , then the repulsion effect will be implemented with a probability of p r , for the purpose of preventing the base antibody from entering undesired region of the search space.On the other hand, the attraction effect takes place with a probability of p a when the affinity of the selected guiding antibody Ab k is higher, thus enhancing the capacity of exploiting the promising regions of the search space.In addition, to improve the randomness of the learning mechanism, the attraction and repulsion scaling factors α 1 and α 2 are set as random numbers generated in the interval [0, 1].From Eq.
(3), it is clear that the search direction of the base antibody is guided to move towards the regions with higher affinity, and meanwhile away from the regions with lower affinity, thus enabling the algorithm to possess better exploitation capacity and the ability of jumping out of local optimum.

Experimental results and discussions
The computational progresses of nine learning operators used in IA described above have been implemented in C++ program under Visual Studio 2010.In order to evaluate the performance of the proposed L sd learning operator, it is validated using some well-known benchmark numerical optimization problems obtained from the literatures (Yao et al., 1999;Cutello et al., 2006;Gong et al., 2010).Table 1 lists the details of the benchmark functions.f 1 −f 5 are unimodal functions which are relatively easy to be optimized, but the difficulty increases as the dimension size increases.f 6 is the step function, while f 7 is a noisy quartic function.f 8 − f 13 are multimodal functions with plenty of local minima which represent the most difficult class of problems for many op-timization algorithms; f 14 − f 23 is a multimodal function with only a few local optima.The different type of benchmark functions test the searching ability of learning operators from different aspects, that is, unimodal functions trend to reflect the convergence speed of the opearator in a direct manner, while multimodal functions are likely to estimate the operator's capacity of escaping from local optima.
Owing to the random nature of the IA and the learning operators, to evaluate the performance of each learning operator, their performance cannot be judged by the result of a single run.Many trials with independent population initialization should be made to obtain a useful conclusion of the performance of the approach.Therefore, in this study the results are obtained in 30 trials.In the experiment, the userdefined parameters are set as follows: the population size is set to be 30, the clone size is 5.It is worth mentioning that we use equal cloning strategy in this study to reduce the influences of cloning operator.By doing so, each antibody in the population has the same probability to undergo the learning mechanism, thus we can make a direct comparison of the performance during all learning operators.In addition, the termination condition of the algorithm is set to be that when the maximum number of function evaluations reaches 150000.Fig. 2 depicts the sketch of the Sphere function f 1 when its dimension D is set to be 2, and the corresponding convergence graphs of each learning operator.It is obvious that, for such unimodal function, all learning operators can evolve the antibodies effectively.In particular, the learning operator L sd possesses the fast learning speed and the most precise solution.
To further demonstrate the effectiveness and robustness of the proposed L sd , all learning operators are carried out on all tested 23 benchmark functions.One of the effective strategies to perform a comparative study between the variants of IAs is to use the oracle-based view of computation (Wolpert and Macready, 1997).Based on this method, the best solution should be found within a certain number of function evaluations.Herein, the best values can be used for comparison because of the equal number of function evaluation for all operators in all cases.To make an intuitive comparison during the variants, the results of best run are normalized between 0 and 1, therefore the worst and best values of each best solution are changed to 0 and 1, respectively.The normalized results for all benchmark functions are presented in Table 2. To achieve a general conclusion based on the oracle-based view, three kinds of the sum of scores and rank of each learning operator are presented in this table.The symbol ∑ u denotes the sum of scores on unimodal function f 1 − f 5, while ∑ m is the sum on multimodal functions f 8 − f 13 .
∑ represents the total sum on all tested benchmark functions.At first glance, it is clear that the learning operator L sd works very well because it has the best performance with the score of 22.940 to 23, and has a rank of 1 among 9 operators.Furthermore, as the L sd is 4.941 which is also the best one among all operators, thus confirming that L sd has a fastest convergence speed for unimodal functions.On the other hand, the biggest value of ∑ m of L sd also verifies the capability of escaping from local optimum for multimodal functions.
Based on Table 2, there are some remarks should be emphasized concerning all learning operators.The first six learning operators mutate the base antibody only utilizing random perturbation, while the last three ones make use of information in the population either semi-blind or search direction based.The population information utilization based learning operators have ranks of 1, 2, 3, significant better than the others, suggesting that the interaction of information in the population is likely to improve the search performance.Thus, we can conclude that the former operators mainly act as the exploitation in the search space, while the latter ones mainly employed as exploration.In the further, it is a promising research direction to combine one of former six operators with the one of latter ones, and it can be expected to achieve a better performance.

Conclusions
In this paper, we made a comprehensive study on the learning operators used in the immunological algorithms.Nine different learning operators, maturing the antibody either by utilizing random perturbation or by utilizing the guiding information from other antibodies, are implemented and analyzed.In view of the limitations of the existing operators, the newly proposed search direction based learning mechanism can not only attract the antibody to promising regions in search space, but also preventing from entering undesired regions by means of the information contained in the antibodies with worse affinities.Experimental results on a large number of benchmark numerical optimization problems verified the effectiveness and robustness of L sd , suggesting that the useful information during the whole population should be sufficiently utilized to improve the search performance of the algorithm.

Figure 1 :
Figure 1: The learning characteristics of all nine operators.

Figure 2 :
Figure 2: (a) the 2D sketch of the Sphere function f 1 , (b) the convergence graphs of each learning operators on f 1 .
where δ k is a Cauchy random variable with the scale parameter t = 1 and satisfies the density function f Cauchy (x) =

Table 1 :
Benchmark problems used in the experiments.