Non-Stationary Stochastic Global Optimization Algorithms

Abstract

Studying the theoretical properties of optimization algorithms such as genetic algorithms and evolutionary strategies allows us to determine when they are suitable for solving a particular type of optimization problem. Such a study consists of three main steps. The first step is considering such algorithms as Stochastic Global Optimization Algorithms (SGoals ), i.e., iterative algorithm that applies stochastic operations to a set of candidate solutions. The second step is to define a formal characterization of the iterative process in terms of measure theory and define some of such stochastic operations as stationary Markov kernels (defined in terms of transition probabilities that do not change over time). The third step is to characterize non-stationary SGoals, i.e., SGoals having stochastic operations with transition probabilities that may change over time. In this paper, we develop the third step of this study. First, we generalize the sufficient conditions convergence from stationary to non-stationary Markov processes. Second, we introduce the necessary theory to define kernels for arithmetic operations between measurable functions. Third, we develop Markov kernels for some selection and recombination schemes. Finally, we formalize the simulated annealing algorithm and evolutionary strategies using the systematic formal approach.

1. Introduction

In global optimization studies, a general question is which type of algorithms are more fit for which type of optimization problems, so, we can decide when to use an evolutionary algorithm to solve a class of optimization problems by understanding its theoretical properties and characterizing its observable behavior [1].

According to Zilinskas and Zhigljavsky in [2], stochastic global optimization algorithms (SGoals in short) are inseparable from their presentation and analysis. Several researchers have done active development of the field from long time ago. Such as: Torn and Zilinskas in [3], Mockus and Zilinskas in [4], or Neimark and Strongin in [5]. However, it remains an active field of research including mathematical analysis of problems.

SGoals have been also studied from a Markovian perspective: Zhigljavsky and Zilinskas in Sections 3.3 and 3.4 in [6] and Tikhomirov in [7] studied the convergence rate of some homogeneous Markov monotone random search optimization algorithms. Also, H Al-Mharmah et al. in [8] studied some random non-adaptive algorithms for finding the maximum of a continuous function on the unit interval. An analysis of selection algorithms was done by Chakraborty et al. in [9] and an analysis for evolutionary strategies for global minimization was described by François in [10].

As can be noticed, these studies do not use measure theory to formalize probabilistic concepts or are developed around a specific optimization problem. Gomez in [11], describes a formal and systematic approach for characterizing stochastic global optimization algorithms. There, the required theory of probability to characterize SGoals such as measure theory, Markov kernel, operations between kernels, products and conditions to study convergence is presented. In addition, it is proved that some algorithmic functions like projection and sort can be represented by kernels.

Moreover, the notion of join-kernel has been introduced in that paper as a way to characterize the combination of stochastic methods. Also, it is defined a formal structure of an optimization space for studying SGoals. Finally, Gomez formalizes algorithms that have a next-population stochastic method, which does not change transition probabilities over time. Such algorithms can be viewed from the perspective of stationary Markov processes. This viewpoint applies among others for, standard versions of hill-climbing, parallel hill-climbing, steady-state genetic, generational genetic, and differential evolution algorithms.

This work continues such a systematic formal approach. First, we review the theory done by Gomez in [11]. Next, we generalize the sufficient conditions convergence Lemma 71 in [11] from stationary to non-stationary Markov processes. Third, we develop arithmetic kernels to characterize arithmetic operations between measurable functions. We develop Markov kernels for some selection and recombination schemes. Finally, in order to show some applications of the concepts developed, we formalize both simulated-annealing and evolutionary-strategies using the systematic formal approach, which are classical algorithms and can be found several studies in the literature such as Romeijn et al. in [12] or Weise in [13].

2. Preliminaries

This section provides a brief introduction to the systematic formalization proposed by Gomez in [11]. Such systematic formalization of SGoals, is carried on Markov kernels terms. We formalize SGoals with stationary next population stochastic method, i.e., SGoals that can be characterized as stationary Markov processes and do not change transition probabilities of the next population over time. That is the case of the hill-climbing [14], the parallel hill-climbing, the generational genetic [15,16,17], the steady-state genetic [18], and the differential evolution [19,20] algorithms. However, SGoals such as the Simulated Annealing [21], Evolutionary Strategies [22], or any algorithm using parameter control/adaptation techniques [23] cannot be characterized as stationary Markov processes.

We clarify that in this section we only review the concepts and not the proofs. The proof of each concept can be found in [11].

2.1. Systematic Formalization Theory

We review the concepts used to characterize SGoals and necessary concepts to extend the theory to characterize adaptive SGoals.

We consider an optimization problem with an objective function $f\Phi \to \mathbb{R}$, which is defined over a feasible region $\Omega \subseteq \Phi$, where $\Phi$ is the solution space and f is a function that is looking for a global optimizer $p^{*}$ described by:

$$\underset{}{min}\left(f:\Phi \to \mathbb{R}\right)=\{p^{*}\in \Omega \subseteq \Phi \mid \left(\forall p\in \Omega \right)\left(f\left(p^{*}\right)\le f\left(p\right)\right)\}.$$

(1)

2.1.1. Stochastic Global Optimization Algorithm

In this work, we focus on algorithms that are not deterministic but stochastic. e.g., simulated annealing and evolutionary strategies. A generic way to describe such algorithms in pseudocode is given by Algorithm 1. Here the main difference between algorithms is the way the ${\mathrm{N}}_{\mathrm{EXT}}{\mathrm{P}}_{\mathrm{OP}}$ operation is carried out.

The ${\mathrm{N}}_{\mathrm{EXT}}{\mathrm{P}}_{\mathrm{OP}}$ method generates new populations, the ${\mathrm{I}}_{\mathrm{NIT}}{\mathrm{P}}_{\mathrm{OPN}}\left(n\right)$ method generates an initial population and ${\mathrm{B}}_{\mathrm{EST}}\left(P_t\right)$ chooses the best set of individuals from $P_t$.

Algorithm 1. Stochastic Global Optimization Algorithm

SGoal(n) 1. $t_0=0$ 2. P = InitPop(n) 3. while¬End($P_t$, t) do 4. $P_{t+1}$ = NextPop( $P_t$ ) 5. $t=t+1$ 6. returnBest($P_t$)

2.1.2. Measure and Probability Theory

Gomez in [11] used probabilistic kernels to formalize SGoals. This work is an extension of Gomez’s work. We review some concepts that Gomez uses and that we are going to use to characterize adaptive SGoals.

Definitions of Measure Theory

Probability theory uses measure theory to formalize the concepts. Measure theory defines elementary events $\Omega \ne \varnothing$ and the system of observable events $\mathcal{A}\subseteq 2^{\Omega }$, where $\mathcal{A}$ is a family of sets. These concepts can be translated in the context of SGOAL where the set of elementary events contains all possible populations and, a family of sets of populations is a system of observable events.

Probability theory operates over sets to measure the probability of some observable event. The structure used to measure subsets of $\Omega$ is a $\sigma$-algebra that meets the following conditions:

1.

$\Omega \in \mathsf{\Sigma }.$$\Omega$ is considered a universal set.

2.

$\mathsf{\Sigma }$ is $\overline{\mathrm{C}}$. $\mathsf{\Sigma }$ is closed under complement.

3.

$\mathsf{\Sigma }$ is $\overline{\mathrm{C}\mathrm{U}}$. $\mathsf{\Sigma }$ is closed under countable unions.

When we deal with a continuous space, a topological space is used. Hence, $\left(\Omega ,\tau \right)$ is a topological space, the sigma-algebra ($\sigma$-algebra) $\mathcal{B}\left(\Omega \right)\equiv \mathcal{B}\left(\Omega ,\tau \right)\equiv \sigma \left(\tau \right)$ is the Borel $\sigma$-algebra on $\Omega$. Consider set $\Omega =\mathbb{R}$, in that case $\mathcal{B}\left(\mathbb{R}\right)$ is the $\sigma$-algebra on $\mathbb{R}$. In this paper, a tuple of the form $\left(\Omega ,\mathsf{\Sigma }\right)$, refers to a measurable space, where $\mathsf{\Sigma }$ is a $\sigma$-algebra.

In probability theory, we generally need to operate between measurable spaces using certain functions, precisely measurable functions. Let $\left({\Omega }_1,{\mathsf{\Sigma }}_1\right)$ and $\left({\Omega }_2,{\mathsf{\Sigma }}_2\right)$ be two measurable spaces. A function f is defined as measurable if $f:{\Omega }_1\to {\Omega }_2$, and $f^{-1}\left(B\right)$ maps a subset of the domain. i.e., if $B\in {\mathsf{\Sigma }}_2$ then $f^{-1}\left(B\right)\in {\mathsf{\Sigma }}_2$.

We call this function f measurable because we can define a measure $\mu :\mathsf{\Sigma }\to \overline{\mathbb{R}}$ on $\left({\Omega }_2,{\mathsf{\Sigma }}_2\right)$ in terms of $\left({\Omega }_1,{\mathsf{\Sigma }}_1\right)$. Where $\mu$ has the following conditions:

1.

$\mu \left(Ø\right)=0$.

2.

$\mu \left(B\right)\ge 0$ for all $B\in \mathsf{\Sigma }$.

3.

$\mu \left({\bigcup }_{i\in I}{B}_i\right)={\sum }_{i\in I}\mu \left(B_i\right)$ for all ${\left\{B_i\in \mathsf{\Sigma }\right\}}_{i\in I}$ Countable Disjoint Family.

If $\left(\Omega ,\mathsf{\Sigma }\right)$ is a measurable space and $\mu$ is a measure, we call $\left(\Omega ,\mathsf{\Sigma },\mu \right)$ a measure space, if $\mu$ is a probability measure i.e., $\mu \left(\Omega \right)=1$, then $\left(\Omega ,\mathsf{\Sigma },\mu \right)$ is a probability space.

Let $\left({\Omega }_1,{\mathsf{\Sigma }}_1,Pr\right)$ be a probability space and $\left({\Omega }_2,{\mathsf{\Sigma }}_2\right)$ be a measurable space. If $X:{\Omega }_1\to {\Omega }_2$ is a measurable function, then X is called a random variable with values in $\left({\Omega }_2,{\mathsf{\Sigma }}_2\right)$.

2.1.3. Kernel

Stochastic processes can model the process of generating one population from another population. A transition kernel is used to characterize each iteration in a stochastic process given by:

$$K\left(x,A\right)=P(x,A)=Pr\left[X_t\in A\mid X_{t-1}=x\right].$$

(2)

where x is the current population and A a subset of possible populations.

Definition 1. 

(Markov kernel) Let $\left({\Omega }_1,{\mathsf{\Sigma }}_1\right)$ and $\left({\Omega }_2,{\mathsf{\Sigma }}_2\right)$ be measurable spaces. A function $K:{\Omega }_1\times {\mathsf{\Sigma }}_2\to \left[0,1\right]$ is called a (Markov) kernel if the following two conditions hold:

1.

Function $K_{x,•}A\mapsto K(x,A)$ is a probability measure for each fixed $x\in {\Omega }_1$

2.

Function $K_{•,A}x\mapsto K(x,A)$ is a measurable function for each fixed $A\in {\mathsf{\Sigma }}_2$.

The transition kernel of an existing transition density $K:{\Omega }_1\times {\Omega }_2\to \left[0,1\right]$ is defined by:

$$K\left(x,A\right)={\int }_{A}K\left(x,y\right)dy.$$

(3)

Kernels linked with deterministic methods used by SGoals play a significant role in developing a systematic formal theory. Next, we review several characterizations of stochastic methods using transition kernels.

Deterministic Kernel

Let $\left({\Omega }_1,{\mathsf{\Sigma }}_1\right)$ and $\left({\Omega }_2,{\mathsf{\Sigma }}_2\right)$ be measurable spaces, and $f{\Omega }_1\to {\Omega }_2$ be ${\mathsf{\Sigma }}_1-{\mathsf{\Sigma }}_2$ measurable. The function $1_f{\Omega }_1\times {\mathsf{\Sigma }}_2\to \left[0,1\right]$ is a kernel defined by:

$$1_f\left(x,A\right)=\left\{\begin{array}{cc}1\hfill & \mathrm{if}f\left(x\right)\in A\hfill \\ 0\hfill & \mathrm{otherwise}.\hfill \end{array}\right.$$

(4)

Kernel Indicator

Let $\left(\Omega ,\mathsf{\Sigma }\right)$ be a measurable space. The indicator function 1: $\Omega \times \mathsf{\Sigma }\to \left[0,1\right]$ defined as $1\left(x,A\right)=1_{id\left(x\right)}\left(A\right)$, with $id\left(x\right)=x$ is a kernel.

Random Scan (Mixing)

The mixing update mechanism of a set of n Markov transition kernels $K_1$, …, $K_n$, each of them with a probability of being picked $p_1,p_2,\dots ,p_n$ ($\sum p_i=1$), is defined by:

$$\left(\sum _{i=1}^n{p}_i{K}_1\right)\left(x,A\right)=\sum _{i=1}^n{p}_i{\int }_A{K}_i\left(x,y\right)dy.$$

(5)

Composition

The composition update mechanism is built using the kernel multiplication operator. It is built on the concept of applying one kernel after another. Kernel composition of $K_1$, $K_2$ is defined by:

$$\left(K_2\circ K_1\right)\left(x,A\right)=\int K_2\left(y,A\right)K_1\left(x,dy\right).$$

(6)

The composition of update mechanisms that correspond to a set of n transition kernels $K_1$, …, $K_n$ is defined as the product kernel $K_n\circ K_{n-1}\circ \dots \circ K_1$. Based on the fact that kernel multiplication is an associative operation.

Transition Kernel Iteration

The transition probability of iteration t (application) of a Markovian kernel K, describes the probability to transit to set $A\in \mathsf{\Sigma }$ within t steps when starting at state $x\in \Omega$ as defined by:

$$K^{\left(t\right)}\left(x,A\right)=\left\{\begin{array}{cc}K\left(x,A\right)\hfill & ,t=1\hfill \\ {\displaystyle \underset{\Omega }{\overset{}{\int }}}{K}^{\left(t-1\right)}\left(y,A\right)K\left(x,dy\right)\hfill & ,t>1.\hfill \end{array}\right.$$

(7)

Let $p:\mathsf{\Sigma }\to \left[0,1\right]$ be the initial distribution of subsets, in that case the probability that the Markov process is in set $A\in \mathsf{\Sigma }$ at step $t\ge 0$ is given by:

$$Pr\left\{X_t\in A\right\}=\left\{\begin{array}{cc}p\left(A\right)\hfill & ,t=0\hfill \\ {\displaystyle \underset{\Omega }{\overset{}{\int }}}{K}^{\left(t\right)}\left(x,A\right)p\left(dx\right)\hfill & ,t>0.\hfill \end{array}\right.$$

(8)

2.1.4. Kernels on Cartesian Products

SGoals work with populations, so we need to work with probability theory concepts that can handle them; To do so, we use a probability space $\left({\Omega }^n,{\mathsf{\Sigma }}^{\otimes n},⨂{}_{i=1}^n{\mu }_i\right)$, where ${\Omega }^n$ is the set of elementary elements, i.e., the set of all possible populations of size n. ${\mathsf{\Sigma }}^{\otimes n}$, is the product $\sigma$-algebra and $⨂{}_{i=1}^n{\mu }_i$ a product probability measure [11].

We can use probability theory on cartesian products to use several new kernels by joining simple kernels. Hence, we can join stochastic methods used in SGoals to generate new populations from other populations.

Swap Kernel

We can define a kernel $1_{\underleftarrow{}}:\left({\Omega }_1\times {\Omega }_2\right)\times \left({\mathsf{\Sigma }}_2\otimes {\mathsf{\Sigma }}_1\right)\to \left[0,1\right]$ that characterizes the Swap method, a deterministic method often utilized by SGoals when we need to select individuals. Where $\underleftarrow{}$ is defined as follows:

$$\begin{array}{cccc}\underleftarrow{}:& {\Omega }_1\times {\Omega }_2& \to & {\Omega }_2\times {\Omega }_1\\ & \left(x,y\right)& \mapsto & \left(y,x\right).\end{array}$$

(9)

Projection Kernel

We can define a kernel $1_{{\pi }_I}:{\prod }_{i=1}^n{\Omega }_i\times {⨂}_{i=1}^m{\mathsf{\Sigma }}_i\to \left[0,1\right]$ that characterizes a method that selects m individuals from a population of size n. where ${\pi }_I$ is defined as follows:

$$\begin{array}{cccc}\hfill {\pi }_I:& \prod _{i=1}^n{\Omega }_i& \to & \prod _{i=1}^m{\Omega }_{k_i}\\ \hfill & \left(x_1,\dots ,x_n\right)& \mapsto & \left(x_{k_1},\dots ,x_{k_n}\right).\end{array}$$

(10)

Join Kernel

The join of methods that generate a subpopulation of the next population can be characterized by a join kernel $⊛K{\Omega }^{\eta }\times {\mathsf{\Sigma }}^{\otimes \upsilon }\to \left[0,1\right]$ with $\upsilon ={\sum }_{k=1}^m{\upsilon }_k$. Where the join stochastic method is defined as follows:

$$\begin{array}{cccc}\hfill F:& {\Omega }^{\eta }& \to & {\Omega }^{\upsilon }\\ \hfill & \left(x_1,\dots ,x_n\right)& \mapsto & \prod _{i=1}^m{F}_i.\end{array}$$

(11)

Permutation Kernel

Methods that generate permutations of populations can be characterized by the kernel $K_I{\Omega }^n\times {\mathsf{\Sigma }}^{\otimes n}\to \left[0,1\right]$ defined as $K_I={⊛}_{k=1}^n{\pi }_{i_k}$. Where $I=\left[i_1,i_2,\dots ,i_n\right]$ is a fixed permutation of the set $\left\{1,2,\dots ,n\right\}$. If we let $\mathcal{P}$ as the set of permutations of the set $\left\{1,2,\dots ,n\right\}$. then, the kernel $K_{\mathcal{P}}{\Omega }^n\times {\mathsf{\Sigma }}^{\otimes n}\to \left[0,1\right]$ defined as $K_{\mathcal{P}}=\frac{1}{\left|\mathcal{P}\right|}{\displaystyle \sum _{I\in \mathcal{P}}}{\pi }_I$ characterizes a stochastic method that generates a population from another one, looking for a fixed set of permutations.

Sorting Kernel

Methods that sorts populations according to closeness of objective function are commonly used in SGoals, so we need to characterize them with a kernel $s_{n,n-1}{\Omega }^n\times {\mathsf{\Sigma }}^{\otimes n}\to \left[0,1\right]$. As defined in Proposition 63 in [11].

VR Kernel

A common pattern in most SGoals is that they can be described by two consecutive stochastic methods: A variation V: ${\Omega }^{\eta }\to {\Omega }^{\varpi }$ and a replacement R: ${\Omega }^{\eta +\varpi }\to {\Omega }^{\upsilon }$ method. Gomez in [4] defined these methods as a single variation-replacement F: ${\Omega }^{\eta }\to {\Omega }^{\upsilon }$ method named (VR) method. These methods can be characterized by kernels $K_R:{\Omega }^{\eta +\varpi }\times {\mathsf{\Sigma }}^{\otimes \upsilon }\to \left[0,1\right]$ and $K_V:{\Omega }^{\eta }\times {\mathsf{\Sigma }}^{\otimes \varpi }\to \left[0,1\right]$, hence $K_{VR}=K_R\circ \left[1_{{\Omega }^{\eta }}⊛K_V\right]$ and $K_{VR}=K_R\circ \left[K_V⊛1_{{\Omega }^{\eta }}\right]$.

2.2. Characterization of a SGoal Using Probability Theory

Definition 2. 

(optimality) $d\left(x\right)=f\left({\mathrm{B}}_{\mathrm{EST}}\left(x\right)\right)-f^{*}$ (Where $f^{*}$ is the optimal value of the objective function f in Ω).

Gomez in [11], defines an optimization space using sets that include optimal elements. It can be seen that these sets can be related to the concept of level set. We establish optimal individuals as individuals that have an objective function less than $ϵ\in {\mathbb{R}}^{+}$. The optimal elements are defined as follows.

1.

($ϵ$-state) x is $ϵ$-optimum element if $d\left(x\right)<ϵ$,

2.

($\overline{ϵ}$-state) x is an $ϵ$-optimum element if $d\left(x\right)\le ϵ$, and

3.

($\widehat{ϵ}$-state) x is an $ϵ$-element if $d\left(x\right)=ϵ$.

Gomez in [11] defines a ($f$-optimization $\sigma$-algebra) as $\sigma$-algebra that contains strict $ϵ$-optimum states ${\left\{{\Omega }_{ϵ}\right\}}_{ϵ>0}\subseteq \mathsf{\Sigma }$. This in order to finally define an optimization space $\left({\Omega }^n,{\mathsf{\Sigma }}^{\otimes n},f\right)$ to study the convergence of SGoals. There $\Omega$ is the feasible region, $\mathsf{\Sigma }$ is a f-optimization $\sigma$-algebra and f is an objective function.

2.3. Kernels on Optimization Spaces

Elitist Stochastic Methods

Some SGoals use elitist stochastic methods that guarantee that the best $\upsilon$ solutions in the next generation are equal to or better than the $\upsilon$ solutions of the current generation. It is to capture the notion of improving the solution.

Definition 3. 

(elitist method) A stochastic method F: ${\Omega }^{\eta }\to {\Omega }^{\upsilon }$ is called elitist if $f\left({\mathrm{B}}_{\mathrm{EST}}\left(\mathrm{F}\left(P\right)\right)\right)\le f\left({\mathrm{B}}_{\mathrm{EST}}\left(P\right)\right)$.

Definition 4. 

(elitist kernel) A kernel $K:{\Omega }^{\eta }\times {\mathsf{\Sigma }}^{\otimes \upsilon }\to \left[0,1\right]$ is called elitist if $K\left(x,A\right)=0$ for each $A\in {\mathsf{\Sigma }}^{\otimes \upsilon }$ such that $d\left(x\right)<d\left(y\right)$ for all $y\in A$.

Lemma 1. 

If $K:{\Omega }^{\eta }\times {\mathsf{\Sigma }}^{\otimes \upsilon }\to \left[0,1\right]$ is elitist then

1.

$K\left(x,{\left({\Omega }_{\overline{d\left(x\right)}}^v\right)}^c\right)=0$ and $K\left(x,{\Omega }_{\overline{d\left(x\right)}}^v\right)=1$.

2.

Let $x\in {\Omega }^{\eta }$, if $d\left(x\right)<\alpha \in \mathbb{R}$ then $K\left(x,{\left({\Omega }_{\overline{\alpha }}^v\right)}^c\right)=0$ and $K\left(x,{\Omega }_{\overline{\alpha }}^v\right)=1.$

Definition 5. 

(optimal strictly bounded from zero) A kernel $K:{\Omega }^{\eta }\times {\mathsf{\Sigma }}^{\otimes \upsilon }\to \left[0,1\right]$ is called optimal strictly bounded from zero iff $K\left(x,{\Omega }_{ϵ}\right)\ge \delta \left(ϵ\right)>0$ for all $ϵ>0$.

2.4. Convergence of a SGoal

2.4.1. Convergence

Let $\left(D_t\right)$ be a random sequence, i.e., a sequence of random variables defined on a probability space $\left(\Omega ,\mathsf{\Sigma },P\right)$. Then $\left(D_t\right)$ is said to

1.

converge completely to zero, denoted as $D_t\stackrel{c}{\to }0$, if for every $ϵ>0$

$$\underset{t\to \infty }{lim}{\displaystyle \sum _{i=1}^{t}Pr\left\{\left|D_t\right|>ϵ\right\}}<\infty .$$

(12)

2

converge in probability to zero, denoted as $D_t\stackrel{p}{\to }0$, if for every $ϵ>0$

$$\underset{t\to \infty }{lim}{\displaystyle Pr\left\{\left|D_t\right|>ϵ\right\}}=0.$$

(13)

Gomez in [11] follows the approach proposed by Günter Rudolph in [24], to study the convergence properties of a SGoal. This concept is also studied by Zhigljavsky and Zilinskas in [6], where there is an extensive study of SGoals. We clarify that in the rest of this paper, $\mathsf{\Sigma }$ is an optimization $\sigma$-algebra. First, Rudolph defines a convergence property for a SGoal in terms of the objective function.

Definition 6. 

(SGoalconvergence). Let $P_t\in {\Omega }^n$ be the population maintained by a SGoal$\mathcal{A}$ at iteration t. Then $\mathcal{A}$ converges to the global optimum if the random sequence $\left(D_t=d\left(P_t\right):t\ge 0\right)$ converges completely to zero.

Lemma 2. 

(Lemma 1 in [24]) If $\exists ϵ>0$ such that $K\left(x,{\Omega }_{ϵ}\right)\ge \delta >0$ for all $x\in {\Omega }_{ϵ}^c$ and $K\left(x,{\Omega }_{ϵ}\right)=1$ for all $x\in {\Omega }_{ϵ}$ then, holds for $t\ge 1$

$$K^{\left(t\right)}\left(x,{\Omega }_{ϵ}\right)\ge 1-{\left(1-\delta \right)}^t.$$

(14)

Using Lemma 2, Rudolph establishes a theorem for convergence of evolutionary algorithms. Which is specified towards SGoals in Theorem 1.

Theorem 1. 

(Theorem 1 in Rudolph [24])

Let a SGoal fulfill the condition everywhere dense sampling condition of Lemma 2. Then will converge to the global optimum ($f^{*}$) of a real valued function $f:\Phi \to \mathbb{R}$ with $f>-\infty$, defined in an arbitrary space $\Omega \subseteq \Phi$, regardless of the initial distribution $p\left(·\right)$.

2.4.2. Convergence of aVR-SGoal

Gomez [11] follows the approach used by Günter Rudolph in [24], to study the convergence properties of a VR-SGoals but Gomez rewrites it in terms of kernels VR variaton-replacement.

Theorem 2.

AVR-SGoalwith $K_{\mathrm{V}}$ an optimal strictly bounded from zero variation kernel and $K_{\mathrm{R}}$ an elitist replacement kernel, will converge to the global optimum of the objective function.

3. Materials and Methods

In this section, we begin by generalizing the theory developed by Gomez in [11] to generalize the concept of Stationary Markov process to non-stationary ones. Next, we study the necessary theory to characterize arithmetic methods that are useful in some recombination and mutation schemes described in [25].

3.1. Generalization to Non-Stationary Algorithms

For a non-stationary (or non-homogeneous) Markov process, the transition probabilities (kernel) may change over time ([26]). Suppose that $K_t$ is the transition kernel applied at time $t>0$ of a non-stationary Markov process. Then, the transition kernel of such non-stationary Markov process at time t is defined as $K^{\left(t\right)}=K_t\circ K_{t-1}\circ \dots \circ K_1$. Clearly, we can rewrite Equation (7). The transition kernel of a non-stationary Markov process is given by:

$$K^{\left(t\right)}\left(x,A\right)=\left\{\begin{array}{cc}{K}^1\left(x,A\right)\hfill & \mathrm{if}t=1\hfill \\ {\displaystyle \underset{\Omega }{\overset{}{\int }}}{K}^{\left(t-1\right)}\left(y,A\right)K^t\left(x,dy\right)\hfill & \mathrm{if}t>1.\hfill \end{array}\right.$$

(15)

Now we are in the position of generalizing Lemma 71 in [11] to non-stationary Markov processes.

Lemma 3.

If $\exists \delta >0$, such that for all $x\in {\Omega }_{ϵ}^c$, $K_t\left(x,{\Omega }_{ϵ}\right)\ge \delta >0$ and, for all $x\in {\Omega }_{ϵ}$, $K_t\left(x,{\Omega }_{ϵ}\right)=1$, then $K^{\left(t\right)}\left(x,{\Omega }_{ϵ}\right)\ge 1-{\left(1-\delta \right)}^t$ holds for $t\ge 1$.

Proof. 

We just rewrite the proof of Lemma 71 in [11] (Gomez uses induction on t) but taking care of the non-stationary property of the Markov process. For $t=1$ we have that $K^{\left(t\right)}\left(x,{\Omega }_{ϵ}\right)=K_t\left(x,{\Omega }_{ϵ}\right)$ (Equation (7)), so $K^{\left(t\right)}\left(x,{\Omega }_{ϵ}\right)\ge \delta$ (condition lemma), therefore $K^{\left(t\right)}\left(x,{\Omega }_{ϵ}\right)\ge 1-{\left(1-\delta \right)}^t$ ($t=1$ and numeric operations). Here, we will use the notation (as Gomez did) $K^{\left(t\right)}\left(y,{\Omega }_{ϵ}\right)=K_y^{\left(t\right)}\left({\Omega }_{ϵ}\right)$ to reduce the visual length of the equations.

$K_x^{\left(t+1\right)}\left({\Omega }_{ϵ}\right)$
$={\displaystyle {}_{\Omega }}{K}_y^{\left(t\right)}\left({\Omega }_{ϵ}\right)K_t\left(x,dy\right)$	(Equation (7))
$={\displaystyle {}_{{\Omega }_{ϵ}}}{K}_y^{\left(t\right)}\left({\Omega }_{ϵ}\right)K_t\left(x,dy\right)+{\displaystyle {}_{{\Omega }_{ϵ}^c}}{K}_y^{\left(t\right)}\left({\Omega }_{ϵ}\right)K_t\left(x,dy\right)$	($\Omega ={\Omega }_{ϵ}\bigcup {\Omega }_{ϵ}^c$)
$={\displaystyle {}_{{\Omega }_{ϵ}}}{K}_t\left(x,dy\right)+{\displaystyle {}_{{\Omega }_{ϵ}^c}}{K}_y^{\left(t\right)}\left({\Omega }_{ϵ}\right)K_t\left(x,dy\right)$	(If $y\in {\Omega }_{ϵ},K_y^{\left(t\right)}\left({\Omega }_{ϵ}\right)=1$)
$=K_t\left(x,{\Omega }_{ϵ}\right)+{\displaystyle {}_{{\Omega }_{ϵ}^c}}{K}_y^{\left(t\right)}\left({\Omega }_{ϵ}\right)K_t\left(x,dy\right)$	(def kernel)
$\ge K_t\left(x,{\Omega }_{ϵ}\right)+\left[1-{\left(1-\delta \right)}^t\right]{\displaystyle {}_{A_{ϵ}^c}}{K}_t\left(x,dy\right)$	(Induction hypothesis)
$\ge K_t\left(x,{\Omega }_{ϵ}\right)+\left[1-{\left(1-\delta \right)}^t\right]K_t\left(x,{\Omega }_{ϵ}^c\right)$	(def kernel)
$\ge K_t\left(x,{\Omega }_{ϵ}\right)+K_t\left(x,{\Omega }_{ϵ}^c\right)-{\left(1-\delta \right)}^t{K}_t\left(x,{\Omega }_{ϵ}^c\right)$
$\ge 1-{\left(1-\delta \right)}^t\left(1-K_t\left(x,{\Omega }_{ϵ}\right)\right)$	(Probability)
$\ge 1-{\left(1-\delta \right)}^t\left(1-\delta \right)$	(condition lemma)
$\ge 1-{\left(1-\delta \right)}^{t+1}$.

   □

Finally, Theorem 72 in [11] also holds for non-stationary Markov processes. So, in order to show convergence of a non-stationary SGoal it is sufficient to prove that the SGoal satisfies the condition of Lemma 3.

Theorem 3. 

(Theorem 72 in [11]–a corrected version of Theorem 1 in [24]) ASGoalwhose stochastic kernel satisfies $K^{\left(t\right)}\left(x,{\Omega }_{ϵ}\right)\ge 1-{\left(1-\delta \right)}^t$ for all $t\ge 1$ will converge to the global optimum ($f^{*}$) of a well-defined real-valued function $f:\Phi \to \mathbb{R}$, defined in an arbitrary space $\Omega \subseteq \Phi$, regardless of the initial distribution $p\left(·\right)$.

Proof. 

See proof of Theorem 72 in [11].   □

3.2. Arithmetic between Measurable Functions

Arithmetic operations can be found in schemes of mutation and recombination [25]. So to characterize an algorithm with kernels in its entirety we must characterize all methods that can alter the generation of new populations.

According to Theorem 22 in [11], to characterize arithmetic methods as deterministic kernels it is enough to prove that these methods are measurable. Proposition 1 provides the sufficient conditions for a function $f:\Omega \to \mathbb{R}$ to be measurable.

Proposition 1.

Let $(\Omega ,\mathsf{\Sigma })$ be a measurable space, then $f:\Omega \to \mathbb{R}$ is $\mathsf{\Sigma }-\mathcal{B}\left(\mathbb{R}\right)$ measurable if and only if one of the following conditions holds:

1.

$\{x\in \Omega :f(x)<b\}\in \mathsf{\Sigma }$ for every $b\in \mathbb{R}$

2.

$\{x\in \Omega :f(x)\le b\}\in \mathsf{\Sigma }$ for every $b\in \mathbb{R}$

3.

$\{x\in \Omega :f(x)\ge b\}\in \mathsf{\Sigma }$ for every $b\in \mathbb{R}$

4.

$\{x\in \Omega :f(x)>b\}\in \mathsf{\Sigma }$ for every $b\in \mathbb{R}$.

Proof. 

Note that $\{x\in \Omega :f\left(x\right)<b\}=f^{-1}\left((-\infty ,b)\right)$, and use $\left\{\right(-\infty ,b):b\in \mathbb{R}\}$ family to generate $\mathcal{B}\left(\mathbb{R}\right)$. Details see [27] proposition 1 chapter 3.    □

Lemma 4 gives a useful equality between sets to characterize arithmetic methods.

Lemma 4.

Let $(\Omega ,\mathsf{\Sigma })$, be a measurable space and $f:\Omega \to \mathbb{R}$ and $g:\Omega \to \mathbb{R}$ be $\mathsf{\Sigma }-\mathcal{B}\left(\mathbb{R}\right)$ measurable, then

$$\{(x,y)\in \Omega \times \Omega :f\left(x\right)+g\left(y\right)<c\}=\bigcup _{q\in \mathbb{Q}}{L}_{q,c}\times R_{q,c},$$

(16)

where,

$$L_{q,c}=\{x\in \Omega :g\left(x\right)<c-q\}$$

and

$$R_{q,c}=\{y\in \Omega :f\left(y\right)<q\}.$$

Proof. 

$[\supseteq ]$ Consider a tuple $(x,y)$ such that:

$$(x,y)\in \bigcup _{q\in \mathbb{Q}}{L}_{q,c}\times R_{q,c}$$

then, for some $q\in \mathbb{Q}$ we have that $g\left(x\right)<c-q$ and $f\left(y\right)<q$. Hence, applying an arithmetic operation we have that $f\left(y\right)+g\left(x\right)<c$. So $(x,y)\in \left\{\right(x,y)\in \Omega :f(y)+g(x)<c\}$.

$[\subseteq ]$

Let $(x,y)\in \left\{\right(x,y)\in \Omega :f(y)+g(x)<c\}$ so there exists some $q\in \mathbb{Q}$ that by the density of rational numbers holds: $f\left(y\right)<q<c-g\left(x\right)$ applying some arithmetic $f\left(y\right)<q$ and $g\left(x\right)<c-q$. From,

$$(x,y)\in \bigcup _{q\in \mathbb{Q}}[\{x\in \Omega :g\left(x\right)<c-q\}\times \{y\in \Omega :f\left(y\right)<q\}$$

follows that

$$\{(x,y)\in \Omega \times \Omega :f\left(x\right)+g\left(y\right)<c\}\subseteq \bigcup _{q\in \mathbb{Q}}{L}_{q,c}\times R_{q,c}.$$

   □

3.2.1. Method Product by a Scalar

Proposition 2.

Let $(\Omega ,\mathsf{\Sigma })$, be a measurable space and $f:\Omega \to \mathbb{R}$ be $\mathsf{\Sigma }-\mathcal{B}\left(\mathbb{R}\right)$ measurable, then $h:\Omega \to \mathbb{R}$ defined as $h\left(x\right)=\alpha \ast f\left(x\right)$ where $\alpha \in \mathbb{R}$, is $\mathsf{\Sigma }-\mathcal{B}\left(\mathbb{R}\right)$ measurable.

Proof. 

For every $\alpha \in \mathbb{R}$ and $c\in \mathbb{R}$ we want to show that $h^{-1}\in \mathsf{\Sigma }$, we prove by cases.

$[\alpha =0]$	Note that $h^{-1}\left((-\infty ,c)\right)=\{x\in \Omega :0\ast f\left(x\right)=0\}$
	$\{x\in \Omega :0\ast f(x)=0\}=\Omega$	Because $f\left(x\right)\in \mathbb{R}$
	$\Omega \in \mathsf{\Sigma }$	Definition of $\sigma$-algebra
	$h^{-1}\left(\left\{0\right\}\right)\in \mathsf{\Sigma }$	$h^{-1}\left(\left\{0\right\}\right)=\{x\in \Omega :0\ast f\left(x\right)=0\}$
$[\alpha >0]$	Note that $h^{-1}\left((c,\infty )\right)=\{x\in \Omega :\alpha \ast f\left(x\right)>c\}$
	$h^{-1}\left((c,\infty )\right)=\{x\in \Omega :f\left(x\right)>c/\alpha \}$	Arithmetic operations
	$\{x\in \Omega :f(x)>c/\alpha \}\in \mathsf{\Sigma }$	Measurable by proposition 2.
	$h^{-1}(c,\infty )\in \mathsf{\Sigma }$	$h^{-1}(c,\infty )=\{x\in \Omega :\alpha \ast f\left(x\right)>c\}$
$[\alpha <0]$	Note that $h^{-1}\left((-\infty ,c)\right)=\{x\in \Omega :\alpha \ast f\left(x\right)<c\}$
	$h^{-1}\left((-\infty ,c)\right)=\{x\in \Omega :f\left(x\right)<c/\alpha \}$	Arithmetic operations
	$\{x\in \Omega :f(x)<c/\alpha \}\in \mathsf{\Sigma }$	Measurable by proposition 2.
	$h^{-1}\left((-\infty ,c)\right)\in \mathsf{\Sigma }$	$h^{-1}\left((-\infty ,c)\right)=\{x\in \Omega :\alpha \ast f\left(x\right)<c\}.$

   □

3.2.2. Method Addition

Proposition 3.

Let $(\Omega ,\mathsf{\Sigma })$ be a measurable space and $f:\Omega \to \mathbb{R}$ and $g:\Omega \to \mathbb{R}$ be $\mathsf{\Sigma }-\mathcal{B}\left(\mathbb{R}\right)$ measurable functions, then $h:\Omega \times \Omega \to \mathbb{R}$ defined as $h(x,y):f\left(y\right)+g\left(x\right)$ is $\mathsf{\Sigma }\otimes \mathsf{\Sigma }-\mathcal{B}\left(\mathbb{R}\right)$ measurable.

Proof. 

We want to show that $h^{-1}\left((-\infty ,c)\right)\in \mathsf{\Sigma }\otimes \mathsf{\Sigma }$ for all $c\in \mathbb{R}$ according to proposition 2.

Now, note that

$$\{(x,y):f\left(y\right)+g\left(x\right)<c\}=h^{-1}\left((-\infty ,c)\right)$$

And using Lemma 4 we establish that:

$$\{(x,y)\in \Omega \times \Omega :g\left(x\right)+f\left(y\right)<c\}=\bigcup _{q\in \mathbb{Q}}{L}_{q,c}\times R_{q,c}$$

where

$$L_{q,c}=\{x\in \Omega :g\left(x\right)<c-q\}$$

and

$$R_{q,c}=\{y\in \Omega :f\left(y\right)<q\}$$

So, if we show that $\bigcup _{q\in \mathbb{Q}}{L}_{q,c}\times R_{q,c}\in \mathsf{\Sigma }\otimes \mathsf{\Sigma }$, then h is measurable.

$\{x\in \Omega :g(x)<c-q\}\in \mathsf{\Sigma }$	measurable by proposition 2
$\{y\in \Omega :f(y)<q\}\in \mathsf{\Sigma }$	measurable by proposition 2
$L_{q,c}\times R_{q,c}\in \mathsf{\Sigma }\times \mathsf{\Sigma }$	family product definition in [11]
$L_{q,c}\times R_{q,c}\in \mathsf{\Sigma }\otimes \mathsf{\Sigma }$	$\mathsf{\Sigma }\times \mathsf{\Sigma }\subseteq \mathsf{\Sigma }\otimes \mathsf{\Sigma }$.
${\displaystyle \bigcup _{q\in \mathbb{Q}}^{}}{L}_{q,c}\times R_{q,c}\in \mathsf{\Sigma }\otimes \mathsf{\Sigma }$	$\mathsf{\Sigma }\otimes \mathsf{\Sigma }$ is $\overline{\mathrm{C}\mathrm{U}}$
So $h^{-1}\left((-\infty ,c)\right)\in \mathsf{\Sigma }\otimes \mathsf{\Sigma }$	$h^{-1}\left((-\infty ,c)\right)=\bigcup _{q\in \mathbb{Q}}{L}_{q,c}\times R_{q,c}.$

   □

3.2.3. Method Product

Lemma 5.

Let $(\Omega ,\mathsf{\Sigma })$ be a measurable space and $f:\Omega \to \mathbb{R}$ be $\mathsf{\Sigma }-\mathcal{B}\left(\mathbb{R}\right)$ measurable function, then $h:\Omega \to \mathbb{R}$ defined as $h\left(x\right):f^2\left(x\right)$ is $\mathsf{\Sigma }-\mathcal{B}\left(\mathbb{R}\right)$ measurable.

Proof. 

We need to show that $h\left(x\right)$ is measurable. i.e., we need to show that $h^{-1}\left((c,\infty )\right)\in \mathsf{\Sigma }$, for all $c\in \mathbb{R}$, note that $h^{-1}\left((c,\infty )\right)=\{x\in \Omega :f^2\left(x\right)>c\}$, We prove by cases.

$[c\ge 0]$

$\{x\in \Omega :f^2\left(x\right)>c\}=$	.
$\{x\in \Omega :f\left(x\right)>\sqrt{c}\}\cup \{x\in \Omega :f\left(x\right)<-\sqrt{c}\}$	Inequality
$\{x\in \Omega :f\left(x\right)>\sqrt{c}\}\in \mathsf{\Sigma }$	Measurable by proposition 2
$\{x\in \Omega :f\left(x\right)<-\sqrt{c}\}\in \mathsf{\Sigma }$	Measurable by proposition 2
$\{x\in \Omega :f\left(x\right)>\sqrt{c}\}\cup \{x\in \Omega :f\left(x\right)<-\sqrt{c}\}\in \mathsf{\Sigma }$	$\mathsf{\Sigma }$ is $\overline{\mathrm{C}\mathrm{U}}$

$[c<0]$

$\{x\in \Omega :f^2\left(x\right)>c\}=\Omega$	All values of $f^2\left(x\right)$ are positive.
$\Omega \in \mathsf{\Sigma }$	Definition of $\sigma$-algebra
$h^{-1}(c,\infty )\in \mathsf{\Sigma }$	$\{x\in \Omega :f^2\left(x\right)>c\}=h^{-1}(c,\infty ).$

   □

Proposition 4.

Let $(\Omega ,\mathsf{\Sigma })$ be a measurable space and $f:\Omega \to \mathbb{R}$ and $g:\Omega \to \mathbb{R}$ be $\mathsf{\Sigma }-\mathcal{B}\left(\mathbb{R}\right)$ measurable functions, then $h:\Omega \times \Omega \to \mathbb{R}$ defined as $h(x,y):f\left(y\right)\ast g\left(x\right)$ is $\mathsf{\Sigma }\otimes \mathsf{\Sigma }-\mathcal{B}\left(\mathbb{R}\right)$ measurable.

Proof. 

We can observe that the product of two functions can be expressed as follows:

$f\left(y\right)\ast g\left(x\right)=\frac{1}{2}[{(f\left(y\right)+g\left(x\right))}^2-f{\left(y\right)}^2-g{\left(x\right)}^2]$

This is measurable because it is in terms of addition, square of measurable functions, and multiplication with a scalar. Follows from Propositions 2 and 3, and Lemma 5.    □

3.2.4. Arithmetic Kernels

Now we proceed to characterize the arithmetic methods using deterministic kernels as defined in Section 2.1.3 that establishes as condition to work on a measurable function.

Theorem 4. 

(Addition kernel) Let $h(x,y)=f\left(x\right)+g\left(y\right)$ be a measurable function as defined in Proposition 3. The addition function ${\mathbf{1}}_{+}:(\Omega \times \Omega )\times \mathcal{B}\left(\mathbb{R}\right)\to [1,0]$ defined as ${\mathbf{1}}_h((x,y),A)=1_{h(x,y)}\left(A\right)$ is a kernel.

Proof. 

This is a deterministic kernel as defined in in Section 2.1.3, it is sufficient to prove that $h(x,y)$ is measurable, which is done in Proposition 3.    □

Theorem 5. 

(Product-Scalar kernel) Let $h\left(x\right)=\alpha \ast f\left(x\right)$ be a measurable function as defined in Proposition 2. The product-scalar function ${\mathbf{1}}_{\alpha }:\Omega \times \mathcal{B}\left(\mathbb{R}\right)\to [1,0]$ defined as ${\mathbf{1}}_h(x,A)=1_h\left(A\right)$ is a kernel.

Proof. 

This is a deterministic kernel as defined in in Section 2.1.3, it is sufficient to prove that $h\left(x\right)$ is measurable, which is done in Proposition 2.    □

3.2.5. Product Kernel

Theorem 6. 

(Product kernel) Let $h(x,y)=f\left(x\right)\ast g\left(y\right)$ be a measurable function as defined in Proposition 4. The product function ${\mathbf{1}}_{*}:(\Omega \times \Omega )\times \mathcal{B}\left(\mathbb{R}\right)\to [1,0]$ defined as ${\mathbf{1}}_h((x,y),A)=1_{h(x,y)}\left(A\right)$ is a kernel.

Proof. 

This is a deterministic kernel as defined in Section 2.1.3, it is sufficient to prove that $h(x,y)$ is measurable, which is done in Proposition 4.    □

Remark 1.

For the sake of writing simplicity, in the rest of this paper, whenever we refer to an arithmetic kernel or a combination of these, we will use the symbol ${\mathbf{1}}_{+}$.

4. Results

4.1. Selection Scheme Formalization

A Selection Scheme, is a method of selecting a group of individuals from a population [28]. Some studies of these schemes can be found in [29,30,31]. Many schemes define an individual selection mechanism $\mathrm{S}1:{\Omega }^{\lambda }\to \Omega$, and selects a group of individuals by repeatedly applying S1. In this paper, we study the uniform, fitness proportional, tournament ([32]), roulette, and ranking selection schemes:

1.

A uniform scheme (${\mathrm{U}}_{\mathrm{NIFORM}}1:{\Omega }^{\lambda }\to \Omega$) gives to each candidate solution $i=1,2,\dots ,\lambda$, the same selection probability $p\left(x_i\right)=\frac{1}{\lambda }$.

2.

A fitness proportional scheme (${\mathrm{P}}_{\mathrm{ROPORTIONAL}}1:{\Omega }^{\lambda }\to \Omega$) gives to each candidate solution $i=1,2,\dots ,\lambda$, a selection’s probability $p\left(x_i\right)$ such that $p\left(x_i\right)<p\left(x_j\right)$ if $f\left(x_j\right)⊲f\left(x_i\right)$ and $p\left(x_i\right)=p\left(x_j\right)$ if $f\left(x_i\right)=f\left(x_j\right)$.

3.

A tournament scheme (${\mathrm{T}}_{\mathrm{OURNAMENT}}{1}^m:{\Omega }^{\lambda }\to \Omega$) of size m chooses m individuals using a Uniform scheme and selects an individual from these using a Proportional1 scheme, $Tournament{1}^m=Proportional1\circ Unifor{m}^m$.

4.

A roulette scheme (${\mathrm{R}}_{\mathrm{OULETTE}}1:{\Omega }^{\lambda }\to \Omega$) is a fitness proportional one where $p\left(x_i\right)=\frac{rate\left(x_i\right)}{{\sum }_{i=1}^{\lambda }rate\left(x_i\right)}$ with $rate\left(x_i\right)<rate\left(x_j\right)$ if $f\left(x_j\right)⊲f\left(x_i\right)$ and $rate\left(x_i\right)=rate\left(x_j\right)$ if $f\left(x_i\right)=f\left(x_j\right)$. If $f\left(x_i\right)\ge 0$ for all $i=1,2,\dots ,\lambda$ and maximizing then $rate\left(x_i\right)$ can be set to $f\left(x_i\right)$.

5.

A ranking scheme (${\mathrm{R}}_{\mathrm{ANKING}}1:{\Omega }^{\lambda }\to \Omega$) is a roulette one with

$rate\left(x_i\right)=1+\left|\right\{x_k:f\left(x_i\right)⊲f\left(x_k\right)\left\}\right|$.

6.

A stud scheme (${\mathrm{S}}_{\mathrm{TUD}}:{\Omega }^{\lambda }\to \Omega$) chooses the best candidate and can be characterized for the next kernel $K_{R_{\mu ,\mu +\lambda }}={\pi }_{\left\{1\right\}}\circ s_{\lambda ,\lambda -1}$

(7)

An Over Selection scheme as defined in [33] (${\mathrm{OS}}_{\mathrm{ELECTION}}1:{\Omega }^{\lambda }\to \Omega$) is a roulette one with $p\left(x_i\right)=0.8/(\lambda \ast 0.68)$ if $ranking\left(x_i\right)⊲\lambda \ast 0.68$ and $p\left(x_i\right)=0.2/(\lambda \ast 0.32)$ if $ranking\left(x_i\right)⊳\lambda \ast 0.32$ where $ranking\left(x_i\right)$ is defined as

$ranking\left(x_i\right)=1+\left|\right\{x_k:f\left(x_i\right)⊲f\left(x_k\right)\left\}\right|$.

Proposition 5.

If $\mathrm{s}1:{\Omega }^{\lambda }\to \Omega$ is a selection scheme with kernel $K_{\mathrm{s}1}$ then $\mathrm{s}:{\Omega }^{\lambda }\to {\Omega }^{\mu }$ has kernel $K_{\mathrm{s}}={⊛}_{i=1}^{\mu }{K}_{\mathrm{s}1}$.

Corollary 1.

If $\mathrm{s}1$ is based on a probability function then $K_{\mathrm{s}}$ is a kernel.

Corollary 2.

TheUniform, Proportional, Tournament, Roulette,Ranking,StudandOSelectionselection schemes have Markov kernels.

4.2. Recombination Scheme Formalization

Recombination schemes use information from one or more parents and generate offspring that share information with their parents. Details for each scheme can be found in [25,34,35,36].

In the following characterizations each individual from a population belongs to ${\Omega }^n$ (set of elementary events), where n is the dimension. Keep in mind that all theory developed by [11] is applicable, hence it is generalized from tuples of tuples to a single tuple. Review Proposition 32 and Corollary 33 in [11].

1.

A Single-Point Crossover method (${\mathrm{SPC}1}_d$$:{\Omega }^n\times {\Omega }^n\to {\Omega }^n\times {\Omega }^n$) is described in Algorithm 2 and can be characterized by the next kernel:

$$K_{SP{C}_{Q_l}}={\pi }_{\{1\dots d\}}\left(A\right)⊛{\pi }_{\{d+1\dots n\}}\left(B\right)$$

$$K_{SP{C}_{Q_r}}={\pi }_{\{1\dots d\}}\left(B\right)⊛{\pi }_{\{d+1\dots n\}}\left(A\right)$$

$$K_{SPC}=K_{SP{C}_{Q_l}}⊛K_{SP{C}_{Q_r}}.$$

(17)

Proof. 

$K_{SPC}$ is defined in terms of projection kernel and join-kernels.    □

Algorithm 2. Single Point Crossover-Spc1

Spc1${}_d$($A,B$) 1: $A_l$ = ${\pi }_{\{1,\dots ,d\}}$ (A) 2: $A_r$ = ${\pi }_{\{d+1,\dots ,n\}}$ (A) 3: $B_l$ = ${\pi }_{\{1,\dots ,d\}}$ (B) 4: $B_r$ = ${\pi }_{\{d+1,\dots ,n\}}$ (B) 5: $Q_l$ = $Join(A_l,B_r)$ 6: $Q_r$ = $Join(B_l,A_r)$ 7: return ($Q_l,Q_r$)

2.

A Multiple-Point Crossover scheme ($\mathrm{M}\mathrm{PC}{1}_D$$:{\Omega }^n\times {\Omega }^n\to {\Omega }^n\times {\Omega }^n$). Let $D=\{1,d_1,d_2,\dots d_m,n\}$ be an ordered list of $\{m+2\in {\mathbb{N}}^{+}\}$ integers that indicate the m positions of crossover plus the first and last position. This formalization just considers when m is an odd number. We can see in Algorithm 3 the description of the algorithm and can be characterized by the next kernel. where l is the length of D.

$$K_{MP{C}_{Q_l}}={⊛}_{i=1}^{l/2}({\pi }_{\{D_{i*2-1}\dots D_{i*2}\}}\left(A\right)⊛{\pi }_{\{D_{i*2}+1\dots D_{i*2+1}\}}\left(B\right))$$

$$K_{MP{C}_{Q_r}}={⊛}_{i=1}^{l/2}({\pi }_{\{D_{i*2-1}\dots D_{i*2}\}}\left(B\right)⊛{\pi }_{\{D_{i*2}+1\dots D_{i*2+1}\}}\left(A\right))$$

$$K_{MPC}=K_{MP{C}_{Q_l}}⊛K_{MP{C}_{Q_r}}.$$

(18)

Proof. 

$K_{MPC}$ is defined in terms of projection kernel and join-kernels.    □

Algorithm 3. Multiple Point Crossover-MultiplePoint1

1: $D=[1,d_1,d_2,\dots d_m,n]$ MultiplePoint1${}_D$($A,B$) 1: $Q_l=\left\{\right\}$ 2: $Q_r=\left\{\right\}$ 3: for$i=\mathit{1}$ to $length\left(D\right)/2$ do 4:     $Q_{l_i}$ = ${\mathrm{J}}_{\mathrm{OIN}}({\pi }_{\{D_{i\ast 2-1},\dots ,D_{i\ast 2}\}}\left(A\right),{\pi }_{\{D_{i\ast 2}+1,\dots ,D_{i\ast 2+1}\}}\left(B\right))$ 5: for$i=\mathit{2}$ to $length\left(D\right)/2$ do 6:     $Q_{r_i}$ = ${\mathrm{J}}_{\mathrm{OIN}}({\pi }_{\{D_{i\ast 2-1},\dots ,D_{i\ast 2}\}}\left(B\right),{\pi }_{\{D_{i\ast 2}+1,\dots ,D_{i\ast 2+1}\}}\left(A\right))$ 7: return ($Q_l,Q_r$)

3.

A Multi-Parent Crossover scheme (${\mathrm{M}}_{\mathrm{ULTI}}{\mathrm{P}}_{\mathrm{ARENT}}\mathrm{C}1:{\Omega }^{n\ast b}\to {\Omega }^n$) can be considered as a generalization of Uniform Crossover, where the definition is given in Algorithm 4. There, the method (${\mathrm{S}}_{\mathrm{IZE}}:{\Omega }^{n\ast b}\to \mathbb{N}\times \mathbb{N}$) in line 1 calculates the number of features of each individual and the amount of parents ($n,b$) respectively. The method ${\mathrm{G}}_{\mathrm{ENERATE}}{\mathrm{L}}_{\mathrm{IST}}{I}_{\mathrm{NDEX}}1:\mathbb{N}\times \mathbb{N}\to {\mathbb{N}}^n$ creates a list of length n where each position has an integer that indicates some parent. This assignation is done following some rule defined in the design of the algorithm. Finally, the method ${\mathrm{C}}_{\mathrm{ROSSOVER}}{1}_D:{\Omega }^{n*b}\to {\Omega }^n$ assigns each element from the parents to a new individual according to values of D. In this characterization we can see that $P\in {\Omega }^{n\ast b}$, we are using this representation in order to use all theory created in [11] Section 3 that allow us to move from tuples of tuples to a single tuple.

The method ${\mathrm{C}}_{\mathrm{ROSSOVER}}1(D,n)$ as defined in Algorithm 4 can be characterized by a kernel $K_{Crossover}:{\Omega }^{n\ast b}\times {\mathsf{\Sigma }}^{\otimes n}\to [0,1]$ defined as:

$$K_{Crossover1}={⊛}_{i=1}^n{\pi }_{\{n\ast (D_i-1)+i\}}.$$

(19)

Proof. 

$K_{Crossover1}$ is defined in terms of projection kernel and join kernels.    □

Algorithm 4. MultiParent Crossover-MultiParentC1

Crossover1${}_{D,n}$(P) 1: $Q=\left\{\right\}$ 2: for$i=\mathit{1}$ to n do 3:     $Q_i$ = ${\pi }_{\{(n\ast (D_i-1))+i\}}\left(P\right)$ 4: return (Q) MultiParentC1(P) 1: n, b= size(P) 2: D = GenerateListIndex (n, b) 3: Q = Crossover1${}_{D,n}$(P) 4: return (Q)

4.

A Shuffle Crossover scheme (${\mathrm{S}}_{\mathrm{HUFFLE}}C1:{\Omega }^n\times {\Omega }^n\to {\Omega }^n\times {\Omega }^n$). We start by permuting each parent. Next, we use some scheme that we have studied above to obtain children. Finally, we undo the permutation that we did to the beginning of the method. The definition can be seen in Algorithm 5. Where method ${\mathrm{R}}_{\mathrm{ANDON}}{\mathrm{P}}_{\mathrm{ERMUTATION}}:\mathbb{N}\to {\mathbb{N}}^n$ generates a permutation of a set of indexes corresponding to the length of features of each parent; ${\mathrm{C}}_{\mathrm{ONV}}{{\mathrm{P}}_{\mathrm{ERMUTATION}}}_{L_{per}}:{\Omega }^n\to {\Omega }^n$ sorts the features of the parents according to the set of indexes obtained in ${\mathrm{R}}_{\mathrm{ANDON}}{\mathrm{P}}_{\mathrm{ERMUTATION}}$; ${\mathrm{S}}_{\mathrm{EGMENTED}}{1}_p:{\Omega }^n\times {\Omega }^n\to {\Omega }^n\times {\Omega }^n$ is the same as definition above; and ${\mathrm{C}}_{\mathrm{ONV}}{\mathrm{P}}_{\mathrm{ERMUTATION}}{{\mathrm{I}}_{\mathrm{NV}}}_{L_{per}}:{\Omega }^n\to {\Omega }^n$ undo the permutation obtained after obtain the child. This method can be characterized by the next kernels:

$$K_{RandomPermutation}=K_{\mathcal{P}}$$

$$K_{ConvPermutation}={⊛}_{i=1}^n{\pi }_{L_{pe{r}_i}}$$

$$K_{Segmented{1}_p}=K_{SC}$$

$$K_{ConvPermutationInv}={⊛}_{i=1}^n{\pi }_i$$

$$K_{\mathbf{ShuffleC}}=[{⊛}_{i=1}^n{\pi }_i⊛{⊛}_{i=1}^n{\pi }_i]\circ K_{SC}\circ [{⊛}_{i=1}^n{\pi }_{L_{pe{r}_i}}⊛{⊛}_{i=1}^n{\pi }_{L_{pe{r}_i}}]\circ K_{\mathcal{P}}.$$

(20)

Proof. 

$K_{\mathbf{ShuffleC}}$ is defined in terms of projection kernel, join-kernels and kernel composition.    □

Algorithm 5. Shuffle Crossover-ShuffleC1

ConvPermutation1${}_{L_{per}}$(P) 1: $Q=\left\{\right\}$ 2: for$i=\mathit{1}$ to n do 3:     $Q_i$ = ${\pi }_{\left\{L_{pe{r}_i}\right\}}\left(P\right)$ 4: return (Q) ConvPermutationInv1${}_{L_{per}}$(P) 1: $Q=\left\{\right\}$ 2: for$i=\mathit{1}$ to n do 3:     $Q_{\left\{L_{pe{r}_i}\right\}}$ = ${\pi }_i\left(P\right)$ 4: return (Q) ShuffleC1($A,B$) 1: $L_{per}={\mathrm{R}}_{\mathrm{ANDOM}}{\mathrm{P}}_{\mathrm{ERMUTATION}}\left(n\right)$ 2: $A_{per}={\mathrm{C}}_{\mathrm{ONV}}{\mathrm{P}}_{\mathrm{ERMUTATION}}{}_{L_{per}}\left(A\right)$ 3: $B_{per}={\mathrm{C}}_{\mathrm{ONV}}{\mathrm{P}}_{\mathrm{ERMUTATION}}{}_{L_{per}}\left(B\right)$ 4: $Q_{1_{per}},Q_{2_{per}}$ = Segmented1${}_p$($A_{per},B_{per}$) 5: $Q_1={\mathrm{C}}_{\mathrm{ONV}}{\mathrm{P}}_{\mathrm{ERMUTATION}}{\mathrm{I}}_{\mathrm{NV}}{}_{L_{per}}\left(Q_{1_{per}}\right)$ 6: $Q_2={\mathrm{C}}_{\mathrm{ONV}}{\mathrm{P}}_{\mathrm{ERMUTATION}}{\mathrm{I}}_{\mathrm{NV}}{}_{L_{per}}\left(Q_{2_{per}}\right)$ 7: return ($Q_1,Q_2$)

5.

Flat Crossover or Arithmetic Crossover schemes, we can use them when the features are defined in the real numbers. (${\mathrm{F}}_{\mathrm{LAT}}\mathrm{C}1:{\Omega }^n\times {\Omega }^n\to {\Omega }^n$), (${\mathrm{A}}_{\mathrm{RITHMETIC}}\mathrm{C}1:{\Omega }^n\times {\Omega }^n\to {\Omega }^n\times {\Omega }^n$).The definitions can be seen in Algorithm 6. These methods can be characterized by the kernels $K_{FlatC1}:({\Omega }^n\times {\Omega }^n)\times {\mathsf{\Sigma }}^{\otimes n}\to [1,0]$, $K_{ArithmeticC1}:({\Omega }^n\times {\Omega }^n)\times ({\mathsf{\Sigma }}^{\otimes n}\times {\mathsf{\Sigma }}^{\otimes n})\to [1,0]$ where the definitions are:

$$K_{FlatC1}={⊛}_{i=1}^n[{\mathbf{1}}_{{+}_1}\circ [{\pi }_i\left(A\right)⊛{\pi }_i\left(B\right)]]$$

$$K_{ArithmeticC1}=[{⊛}_{i=1}^n[{\mathbf{1}}_{{+}_1}\circ [{\pi }_i\left(A\right)⊛{\pi }_i\left(B\right)]]⊛{⊛}_{i=1}^n[{\mathbf{1}}_{{+}_2}\circ [{\pi }_i\left(A\right)⊛{\pi }_i\left(B\right)]]].$$

(21)

Proof. 

$K_{ArithmeticC1}$ is defined in terms of projection kernel, join-kernels and kernel composition.    □

6.

A Blended Crossover scheme, can be seen as a generalization of ${\mathrm{F}}_{\mathrm{LAT}}\mathrm{C}1$. The scheme is represented by the function (${\mathrm{B}}_{\mathrm{LENDED}}\mathrm{C}{1}_{\alpha }:{\Omega }^n\times {\Omega }^n\to {\Omega }^n$).The definitions can be seen in Algorithm 6. These methods can be characterized by the kernels $K_{BlendedC1}:({\Omega }^n\times {\Omega }^n)\times {\mathsf{\Sigma }}^{\otimes n}\to [1,0]$, defined by:

$$K_{mi{n}_i}={\pi }_1\circ s_{2_{⊲}}\circ [{\pi }_i\left(A\right)⊛{\pi }_i\left(B\right)]$$

$$K_{ma{x}_i}={\pi }_1\circ s_{2_{⊳}}\circ [{\pi }_i\left(A\right)⊛{\pi }_i\left(B\right)]$$

$$K_{BlendedC1}={⊛}_{i=1}^n[{\mathbf{1}}_{{+}_i}\circ [[K_{mi{n}_i}⊛K_{ma{x}_i}]\circ U[0,1]]].$$

(22)

Proof. 

$K_{BlendedC1}$ is defined in terms of projection kernel, join-kernels, kernel composition, arithmetic kernel and sorts kernel.    □

Algorithm 6. Flat and Arithmetic Crossover-ShuffleC1, ArithmeticC1

FlatC1 ($A,B$) 1: $Q=\left\{\right\}$ 2: for$i=\mathit{1}$ to n do 3:     $\alpha \sim$$U[0,1]$ 4:     $Q_i$ = $\alpha \ast {\pi }_i\left(A\right)+(1-\alpha )\ast {\pi }_i\left(B\right)$ 5: return (Q) ArithmeticC ($A,B$) 1: $Q_l=\left\{\right\}$ 2: $Q_r=\left\{\right\}$ 3: for$i=\mathit{1}$ to n do 4:     $\alpha \sim$$U[0,1]$ 5:     $Q_{l_i}$ = $\alpha \ast {\pi }_i\left(A\right)+(1-\alpha )\ast {\pi }_i\left(B\right)$ 6:     $Q_{r_i}$ = $(\alpha -1)\ast {\pi }_i\left(A\right)+\left(\alpha \right)\ast {\pi }_i\left(B\right)$ 7: return ($Q_l,Q_r$)

Algorithm 7. Blended Crossover-BlendedC1

BlendedC1${}_{\alpha }$($A,B$) 1: $Q=\left\{\right\}$ 2: for$i=\mathit{1}$ to n do 3:     $x_{min}=min({\pi }_i\left(A\right),{\pi }_i\left(B\right))$ 4:     $x_{max}=max({\pi }_i\left(A\right),{\pi }_i\left(B\right))$ 5:     $dx=x_{max}-x_{min}$ 6:     $\beta \sim [0,1]$ 7:     $Q_i=\beta \ast (x_{min}-\alpha \ast dx)+(1-\beta )\ast (x_{max}+\alpha \ast dx)$ 8: return (Q)

7

A Linear Crossover scheme, (${\mathrm{L}}_{\mathrm{INEAR}}\mathrm{C}1:{\Omega }^n\times {\Omega }^n\to {\Omega }^n\times {\Omega }^n\times {\Omega }^n$). The definitions can be seen in Algorithm 6. These methods can be characterized by the kernels $K_{LinearC1}:({\Omega }^n\times {\Omega }^n)\times ({\mathsf{\Sigma }}^{\otimes n}\times {\mathsf{\Sigma }}^{\otimes n}\times {\mathsf{\Sigma }}^{\otimes n})\to [1,0]$, defined by:

$$K_{LinearC1}={⊛}_{i=1}^n[[{\mathbf{1}}_{{+}_{1_i}}\circ [{\pi }_i\left(A\right)⊛{\pi }_i\left(B\right)]]⊛{\mathbf{1}}_{{+}_{2_i}}\circ [{\pi }_i\left(A\right)⊛{\pi }_i\left(B\right)]⊛{\mathbf{1}}_{{+}_{3_i}}\circ [{\pi }_i\left(A\right)⊛{\pi }_i\left(B\right)]].$$

(23)

Proof. 

$K_{LinearC1}$ is defined in terms of kernel, join-kernels, kernel composition and addition kernel.    □

Algorithm 8. Linear Crossover-LinearC1

LinearC1($A,B$) 1: $Q_1=\left\{\right\}$ 2: $Q_2=\left\{\right\}$ 3: $Q_3=\left\{\right\}$ 4: for$i=\mathit{1}$ to n do 5:     $Q_{1_i}=(1/2)\ast {\pi }_i\left(A\right)+(1/2)\ast {\pi }_i\left(B\right)$ 6:     $Q_{2_i}=(3/2)\ast {\pi }_i\left(A\right)-(1/2)\ast {\pi }_i\left(B\right)$ 7:     $Q_{3_i}=(-1/2)\ast {\pi }_i\left(A\right)+(3/2)\ast {\pi }_i\left(B\right)$ 8: return ($Q_1,Q_2,Q_3$)

4.3. Simulated Annealing (sa)

4.3.1. Concept

The Simulated Annealing algorithm (sa) considers the idea behind the process of heating and cooling a material to recrystallize it, see Algorithm 9. When the temperature decreases, the material settles into a more ordered state, and the state into which they settle is not always the same. This state tends to have low energy compared when the material is in the presence of high temperature ([25]). If we consider energy as a cost function, we can use this approach to minimize cost functions. Therefore, SA is a stochastic algorithm that works with a single-individual that generates a single candidate-solution x (parent) and sets a high temperature to explore the search space. Then, a variation mechanism generates a new candidate-solution y (child) and measures its cost. A replacement policy, that fitness function and the temperature, picks one individual between the father and the child. Finally, a process decreases the temperature looking for each new solution having less energy.

Clearly, the replacement policy in Algorithm 9 (lines 6, …, 11) is not elitist. This allows sa to expand the search but can lead to the loss of some good candidate-solutions. In practice, it is normal to keep track of the best solution found so far [25]. If this is done, the replacement policy is an elitist one.

Algorithm 9. Simulated Annealing [25]

Simulated annealing 1: $T=$ initial temperature > 0 2: $\alpha \left(T\right)=$ cooling function: $\alpha \left(T\right)\in [0,T]$ for all T 3: Initialize a candidate solution $x_0$ to minimization problem $f\left(x\right)$ 4: while¬ TerminationCondition() do 5:     Generate a candidate solution x 6:     if $f\left(x\right)<f\left(x_0\right)$ 7:     $x_0=x$ 8:     else 9:     $r=U[0,1]$ 10:     if$r<exp[(f\left(x_0\right)-f\left(x\right))/T]$ 11:     $x_0=x$ 12:     $T=\alpha \left(T\right)$

4.3.2. Formalization

To formalize and characterize (sa), we use the approach proposed by [11]. We rewrite Algorithm 9 in terms of individual non-stationary stochastic methods, see Algorithm 10. This new Algorithm is in terms of Variation-Replacement methods. Observe that Algorithms 9 and 10 are equivalents. Line 5 of Algorithm 9 is the method Variate${}_{SA}$ (line 1) of Algorithm 10; lines 6 to 11 of Algorithm 9 is the method Replace${}_{SA}$ (line 2) of Algorithm 10. Finally, line 12 of Algorithm 9 and method UpdateParameters (line 3) perform the same task.

Algorithm 10. Simulated Annealing in terms of VR methods

NextPop${}_{SA}$(x) 1: y=Variate${}_{SA}$(x) 2: y=Replace${}_{S{A}_T}$ (y, x) 3: UpdateParameters (T) 4: return$x^{\prime }$

Now, we focus on characterizing (sa) as a VR stochastic method and analyzing its convergence through non-stationary Markov kernels.

Proposition 6.

IfReplace${}_{SA}$($x,x$) is an elitist method, then it can be characterized by the Markov Kernel $R_{SA}:{\Omega }^2\times \mathsf{\Sigma }⟶[1,0]$ defined as:

$$K_{R_{SA}}={\pi }_1\circ s_2.$$

(24)

Proof. 

$K_{R_{SA}}$ is defined in the same way that the method of R${}_{HC}$ in [11]. So the proof uses the same argument that Lemma 75 in [11].    □

Proposition 7.

If the stochastic method Variate${}_{A{S}_T}$ can be characterized by a non-stationary Markov kernel $V_{S{A}_T}^{\left(t\right)}:\Omega \times \mathsf{\Sigma }⟶[1,0]$ and condition of Proposition 6 are fulfilled then method theNextPop${}_{S}A$(x) can be described as a VR non-stationary Markov Kernel defined as

$$K_{SA}^{\left(t\right)}=K_R\circ K_{V_{S{A}_T}}^{\left(t\right)}.$$

(25)

Proof. 

$K_{SA}^{\left(t\right)}$ is a kernel composition under the given conditions.    □

Proposition 8.

IfReplace${}_{SA}$is an elitist method, then NextPop${}_{SA}$ can be characterized by an elitist non-stationary Markov kernel.

Proof. 

This proof uses the same argument as Proposition 77 in [11].    □

4.3.3. Convergence

Corollary 3.

If the conditions of Propositions 6, 7 and 8, are fulfilled and methodVariate${}_{A{S}_T}$ is optimal strictly bounded from zero thenNextPop${}_{SA}$ is optimal strictly bounded from zero.

Proof. 

Follows from Definition 67, Lemma 68, and Definition 69 in [11] and Proposition 8 that state that NextPop${}_{SA}$ can be characterized by an elitist kernel, and this is optimal strictly bounded from zero.    □

Theorem 7.

sawill converge to the global optimum ifReplace${}_{S}A$ is elitist and ifVariate${}_{A{S}_T}$is optimal strictly bounded from zero.

Proof. 

Follows from Corollary 3, and Propositions 6–8.    □

4.4. Evolutionary Strategies (es)

4.4.1. Concept

Evolutionary Strategies ($\mu /\rho \underset{,}{+}\lambda$)-es are a type of Evolutionary Algorithms that apply mutation, recombination, and selection operators to a population of individuals [22], see Algorithm 11. Every individual has two parts: the candidate solution (x) and the set of endogenous strategy parameters (s) used to control the mutation operator ([22]). An es randomly initializes the population, (Line 2), and evolves both parts of the individual (Lines 5–9) up to certain ending-condition is fulfilled (Line 3). The set of endogenous parameters are exposed to evolution (Lines 6 and 8) before producing a child candidate solution (Line 7 and 9) to introduce variety. The new individual is a composition of a set of selected candidate solutions (Line 5). es generates a new population of $\lambda$ new individuals each generation (Line 4). Finally, es selects a final population using two possible approaches. The ($\mu$ + $\lambda$)-es approach that selects the best $\mu$ individuals among the $\mu$ parents and $\lambda$ children or the ($\mu$,$\lambda$)-es that selects the best $\mu$ individuals from the $\lambda$ children (notice that $\lambda \ge \mu$ in this case). In this work, we study both of them.

Algorithm 11. Evolutionary strategies described by [22]

ES$\mu /\rho \underset{,}{+}\lambda$ 1: $g=0$ 2: initialize($P_q^{\left(0\right)}:=\{(y_m^{\left(0\right)},s_m^{\left(0\right)},F\left(y_m^{\left(0\right)}\right))$, $m=1,\dots ,\mu$}) 3: while¬ TerminationCondition() do 4:     for $l=\mathit{1}$ to $\lambda$ do 5:         $a_l$ = Marriage($P_q^g,\rho$) 6:         $s_l$ = Recombination${}_s$($a_l$) 7:         $y_l$ = Recombination${}_y$($a_l$) 8:         $s_l^{\prime }$ = Mutation${}_s$($s_l$) 9:         $y_l^{\prime }$ = Mutation${}_s$($y_l$, $s_l^{\prime }$) 10:         $F_l^{\prime }$ = $F\left(y_l^{\prime }\right)$ 11:     $P_0^g=\{(y_l^{\prime },s_l^{\prime },F_l^{\prime }),l=1,\dots ,\lambda \}$ 12:     if ($\mu ,\lambda$) then 13:         $P_q^{g+1}=Selection(P_0^g,\mu )$ 14:     else ($\mu +\lambda$) 15:         $P_q^{g+1}=Selection(P_0^g,P_q^g,\mu )$ 16:     g = g+1

4.4.2. Formalization

To formalize and characterize ($\mu /\rho 0.35ex\underset{0.5ex,}{+}\lambda$)-es, we rewrite Algorithm 11 in terms of individual non-stationary stochastic methods, see Algorithm 12. This follows the approach in [11] that express the algorithms in terms of Variation-Replacement methods to study their convergence properties.

Notice that Algorithms 11 and 12 are equivalents: Lines 4–11 in Algorithm 11 is method Variate(P) (Line 1) in the NextPop method of Algorithm 12. Also, Lines 12–15 in Algorithm 11 are Line 2 in the NextPop method of Algorithm 12. Using this characterization, we proceed to characterize each method of Algorithm 12 through non-stationary Markov kernels.

With the object of characterizing ($\mu /\rho \underset{,}{+}\lambda$)-ES we need to establish some non-stationary Markov kernels. First, we study the Variate method (Line 1, method NextPop, Algorithm 12).

Following Definition 55 in [11], we can express the variation method ${\mathrm{V}}_{\mathrm{ARIATE}}:{\Omega }^{\mu }⟶{\Omega }^{\lambda }$ as a joined stochastic method.

$$\begin{array}{c}\hfill {\mathrm{V}}_{\mathrm{ARIATE}}\left(P\right)=\prod _{i=1}^{\lambda }{\mathrm{N}}_{\mathrm{EXT}}{\mathrm{S}}_{\mathrm{UB}}{\mathrm{P}}_{\mathrm{OP}}{}_i\left(P\right)\end{array}$$

(26)

where ${\mathrm{N}}_{\mathrm{EXT}}{\mathrm{S}}_{\mathrm{UB}}{\mathrm{P}}_{\mathrm{OP}}:{\Omega }^{\mu }⟶\Omega$ chooses $\rho$ individuals from the population, combines the $\rho$ individuals, generates a child and finally mutates the strategy and the child.

Algorithm 12. Evolutionary strategies algorithm-NextPop method described in terms of VR methods

NextSubPop${}_i$(P) 1: $a=$PickParents(P) 2: $q=$Xover${}_a$(P) 3: UpdateStrategies${}_a$(s, i) 4: $d=$Variate${}_s$(q) 5: returnd UpdateStrategies${}_a$(s, i) 1: z = XoverStrategie${}_a$(s) 2: $s_i$ = VariateStrategie(z) Variate(P) 1: for$i=\mathit{1}$ to $\lambda$ do 2:     $Q_i$ = NextSubPop${}_i$(P) 3: return Q NextPop${}_{\mathsf{\Psi }}$(P) 1: Y = Variate(P) 2: Q = Replace${}_{\Psi }$(P, Y) 3: returnQ

Proposition 9.

If Lines 8 and 9 of methodUpdateStrategiesof Algorithm 5 can be characterized by non-stationary kernels $X:{\mathbb{R}}^{\rho }\times \mathcal{B}{\left(\mathbb{R}\right)}^{\otimes \rho }⟶[0,1]$ and $V{S}^{\left(t\right)}:\mathbb{R}\times \mathcal{B}\left(\mathbb{R}\right)⟶[0,1]$ respectively.UpdateStrategiescan be characterized by a non-stationary kernel $U{S}^{\left(t\right)}:{\mathbb{R}}^{\rho }\times \mathcal{B}\left(\mathbb{R}\right)⟶[0,1]$ defined as:

$$K_{US}^{\left(t\right)}=K_{VS}^{\left(t\right)}\circ K_{XS}.$$

(27)

Proof. 

$K_{US}^{\left(t\right)}$ is in terms of kernel composition, follows from Definition 25 in [11]. □

Proposition 10.

If Lines 2 and 4 of Algorithm 12 can be characterized by non-stationary Markov kernels ${\mathrm{X}}_{\mathrm{OVER}}{}_a:({\Omega }^{\rho }\times \mathsf{\Sigma })⟶[0,1]$ and ${\mathrm{V}}_{\mathrm{ARIATE}}{}_s:(\Omega \times \mathsf{\Sigma })⟶[0,1]$ respectively, then the method ${\mathrm{N}}_{\mathrm{EXT}}{\mathrm{S}}_{\mathrm{UB}}{\mathrm{P}}_{\mathrm{OP}}$ can be characterized by the kernel ${\mathrm{N}}_{\mathrm{EXT}}{\mathrm{S}}_{\mathrm{UB}}{\mathrm{P}}_{\mathrm{OP}}:({\Omega }^{\rho }\times \mathsf{\Sigma })⟶[0,1]$ defined as the non-stationary kernel:

$$K_{{\mathrm{N}}_{\mathrm{EXT}}{\mathrm{S}}_{\mathrm{UB}}{\mathrm{P}}_{\mathrm{OP}}}=K_{{\mathrm{V}}_{\mathrm{ARIATE}}{}_s}^{\left(t\right)}\circ K_{XOVER}\circ {\pi }_{\left\{1,\dots ,\rho \right\}}\circ K_{\mathcal{P}}.$$

(28)

Proof. 

$K_{{\mathrm{N}}_{\mathrm{EXT}}{\mathrm{S}}_{\mathrm{UB}}{\mathrm{P}}_{\mathrm{OP}}}$ is in terms of kernel composition, follows from Definition 25 in [11]. □

Proposition 11.

If ${\mathrm{N}}_{\mathrm{EXT}}{\mathrm{S}}_{\mathrm{UB}}{\mathrm{P}}_{\mathrm{OP}}$ can be characterized by a non-stationary Markov kernel, the stochastic method ${\mathrm{V}}_{\mathrm{ARIATE}}{}^{\left(t\right)}$ can be characterized by a kernel $V:{\Omega }^{\mu }\times {\mathsf{\Sigma }}^{\otimes \mu }⟶[0,1]$ defined as

$$K_{{\mathrm{V}}_{\mathrm{ARIATE}}}^{\left(t\right)}=\left[{⊛}_{i=1}^{\lambda }\left[K_{{\mathrm{N}}_{\mathrm{EXT}}{\mathrm{S}}_{\mathrm{UB}}{\mathrm{P}}_{{\mathrm{OP}}_i}}\right]\right].$$

(29)

Proof. 

$K_{{\mathrm{V}}_{\mathrm{ARIATE}}}^{\left(t\right)}$ is a join stochastic method, follows from Definition 55 and Proposition 56 in [11]. □

Proposition 12.

The stochastic methodReplace${}_{(\mu +\lambda )}$ used in Line 2 of methodNextPop, can be characterized by the kernel $R_{\mu ,\mu +\lambda }:{\Omega }^{\mu +\lambda }\times {\mathsf{\Sigma }}^{\otimes \mu }⟶[0,1]$ defined as $K_{R_{\mu ,\mu +\lambda }}={\pi }_{\left\{1,\dots ,\mu \right\}}\circ s_{\mu +\lambda ,\mu +\lambda -1}$ and the stochastic method ${\mathrm{R}}_{\mathrm{EPLACE}}{}_{(\mu ,\lambda )}$, can be characterized by the kernel $R_{\mu ,\lambda }:{\Omega }^{\lambda }\times {\mathsf{\Sigma }}^{\otimes \mu }⟶[0,1$ defined as $K_{R_{\mu ,\lambda }}={\pi }_{\left\{1,\dots ,\mu \right\}}\circ s_{\lambda ,\lambda -1}$.

Proof. 

$K_{R_{\mu ,\lambda }}$ and $K_{R_{\mu +\lambda }}$ are kernels composition. Follows from Definition 25 in [11]. □

Corollary 4.

If methodsPickParents,XOver${}_a$,XoverStrategie${}_a$,VariateStrategieand,Variate${}_s$can be described by Markov kernels fulfilling the conditions of Propositions 9 and 10, evolutionary Strategies can be described by a VR kernel.

$$K_{ES}=K_R\circ K_V$$

where:

$$K_V=K_{Variate}$$

$$K_R=K_{R_{\mu ,\lambda }}\mathrm{or}{K}_R=K_{R_{\mu +\lambda }}.$$

(30)

Proof. 

Follows from Propositions 9–12. □

4.4.3. Convergence

Proposition 13.

TheNextPop${}_{\left(\mu /\rho +\lambda \right)-ES}$is an elitist stochastic method that can be characterized by an elitist stochastic kernel.

Proof. 

Let $k\in [1,\mu ]$ be the index of the best individual in population P, then $f\left(BEST\left(P\right)\right)=f\left(P_k\right)$. Since $P\subseteq \{P\cup {\mathrm{V}}_{\mathrm{ARIATE}}(P\left)\right\}$ and the method Replace is elitist. It is clear that $f\left(BEST(P\cup {\mathrm{V}}_{\mathrm{ARIATE}}\left(P\right))\right)\underset{-}{◃}f\left(P_k\right)$. □

Corollary 5.

If conditions of Proposition 9 and 10 are satisfied andVariate${}_s$is optimal strictly bounded from zero then the methodNextPop${}_{\mu +\lambda }$ is optimal strictly from zero.

Proof. 

Follows from Definition 67, Lemma 68, and Definition 69 of [11] and Proposition 13 that establish that an elitist kernel is optimal strictly bounded from zero. □

Theorem 8. 

($\mu /\rho +\lambda$)-ES will converge to the global optimum if methodsPickParentsandVariate${}_s^{\left(t\right)}$ can be characterized by stationary or non-stationary Markov kernels andVariate${}_s$is optimal strictly bounded from zero.

Proof. 

Follows from Theorem 3 and Corollary 5. □

5. Discussion

We have generalized the conditions of convergence to the global optimum from stationary to non-stationary Markov process that are presented in the work of stochastic global optimization algorithms: a systematic approach proposed in [11]. We study the necessary theory to describe with kernels some arithmetic methods. For doing so, it was necessary to use the concepts studied in real analysis, such as arithmetic between measurable functions. However, the literature found only studied the case of operating two functions on the same variable but not on two different variables. Hence, the concepts of product sigma algebra presented by Gomez in [11], were used to prove that arithmetic operations between measurable functions on two different variables are also measurable.

We formalized some selection and recombination schemes to generalize the theory to cover as many variations of each algorithm as possible. For that, we have found that most of these methods could be characterized using the kernels studied in [11] and the new kernels studied in this paper. This makes us think that other schemes in the literature could be easily adapted to the concepts developed in this paper.

In this paper, we have formalized and characterized the simulated annealing algorithm and evolutionary strategies using the developed theory (both have been formalized in terms of Variation-Replacement kernels). A wide variety of non-stationary algorithms described algorithmically can be found in the literature. However, the theory described in this work cannot be used directly. For that reason, the first step to take is to write the algorithms in terms of Variation-Replacement as shown in Section 4.3 and Section 4.4. This approach can also be studied in [1]. There, the class of hybrid adaptive evolutionary algorithms is characterized.

Also, we formulated a set of conditions that sa and es algorithms should fulfill to achieve a global convergence. After characterizing these algorithms by a Variation-Replacement Kernel, it has been proven that these can converge to the global optimum if the particular implementation of the Variational method is strictly bounded from zero, which depends of the way each algorithm is implemented.

Our future work will focus on using the proposed approach to formalize as many stationary and non-stationary SGoals as possible, and extending and developing the theory for several particular methods (Mutation, recombination and selection) that can be considered in SGoals. Moreover, we will study new convergence conditions and not only for the global optimum.