An Overview of Few Nature Inspired Optimization Techniques and Its Reliability Applications

Optimization has been a hot topic due to its inevitably in the development of new algorithms in almost every applied branch of Mathematics. Despite the broadness of optimization techniques in research fields, there is always an open scope of further refinement. We present here an overview of nature-inspired optimization with a subtle background of fundamentals and classification and their reliability applications. An attempt has been made to exhibit the contrast nature of multi objective optimization as compared to single objective optimization. Though there are various techniques to achieve the optimality in optimization problems but nature inspired algorithms have proved to be very efficient and gained special attention in modern research problems. The purpose of this article is to furnish the foundation of few nature inspired optimization techniques and their reliability applications to an interested researcher. KeywordsMetaheuristics, Grey wolf optimizer, Multi-objective optimization, Reliability optimization


Introduction
As the term suggests, optimization is an act of moving towards the optimum. In other words, it is a pursuit to the best in the local sense and the method facilitating this, comes under the category of an optimization method. Mathematically, one seeks the extremum (called optimum) of a function ( 1 , 2 , … … , ) of several variables in the domain of its definition with simultaneously taking care of the restrictions (called constraints) on the variables , 1 ≤ ≤ .
The inevitability of optimization can be understood in the natural learning processes of a child who optimizes the speech from lisping to fluency, writing skills from scribbling to perfection and walking skills from crawling to feet. This act of betterment is widespread in the areas of science, engineering, economics, management, etc. where the numerical data is processed. For instance, a civil engineer is aimed to the best design of an anti-earthquake bridge subject to the approved budget and time constraints. In contrast, a hardware engineer working on computer chips is always keen to minimize the area of the resulting layout accommodating the optimum = ( ) = 0, 1 ≤ ≤ = (1) that optimizes (minimize or maximize) the function ( ), i.e., ( * ) ≤ ( ) (minimization) or ( * ) ≥ ( ) (maximization) ∀ = ( 1 , 2 , … , ) ∈ } where, is the dimension of the problem which is to optimize. ( ) is termed as objective function and its domain = 1 × 2 × … × is called the decision variable space. , either continuous or discrete, is the search space of , the ℎ optimization parameter. and are not necessarily identical, in the sense of either type or size. An n-dimensional vector of optimization variables = ( 1 , 2, … , ) is called the feasible solution if it satisfies all the constraints. search space can be viewed as the set of all feasible solutions. A feasible solution * that minimizes (maximizes) the objective function is called an optimal solution. The symbol = is the ℎ equality constraint function, = is the number of equality constraint functions. + is the ℎ positive constraint function, + is the number of positive constraint functions. − is the ℎ negative constraint function, − is the number of negative constraint functions.
In the light of qualitative analysis for the calculus of functions, an optimal solution * s.t. ∀ ∈ , ( * ) − ( ) ≥ 0 ≤ 0 is called the global optimum solution. Such a solution is absolutely the best set of parameters ( 1 * , 2 * , … , * ) ∈ that optimizes . Hoping straightaway to get such a global winner solution is generally a cumbersome task and this happens in the case of non-linear programming (NLP) (Boyd and Vandenberghe, 2004;Kumar et al., 2017d;. Nonetheless, expecting a local winner * * in some open proper subset ⊂ needs to qualify fairly relaxed criteria i.e. ∀ ∈ , ( * * ) − ( ) ≥ 0 ≤ 0. Such a solution is termed as a local optimum solution. A lot to depends upon the choice of the initial point 0 ∈ while working with a local optimization algorithm but a good converging algorithm is the one, which always converges regardless of this particular choice. Such an algorithm is said to be globally convergent in the sense of freeness for the choice of 0 inside (Bergh, 2001).

Classification
On the basis of decision variable space, constraints and objective function an optimization problem is classified as follows: For the given decision variable space ⊆ ℝ (where ℝ is the set of reals) of real valued programming problem, if in addition ⊆ ℤ (where ℤ is the set of integers) holds then the problem is called as Integer Programming Problem. In contrast, if ⊆ ℤ doesn't hold and ∩ ℤ ≠ ∅ then the problem is called as Mixed Integer Programming Problem. Furthermore, if each , 1 ≤ ≤ is finite then the problem is called as Combinatorial Programming Problem. The term combinatorial has been derived from the fact that the solution is one of all permutations or combinations of the elements of finite sets .
An optimization problem is constrained or unconstrained depending on the presence or absence of the constraints. As long as all the constraints and objective function involves linear polynomials, the problem is called Linear Programming Problem (LPP) else it is Nonlinear Programming Problem (NLPP). Further, the appearance of posynomials in the constraints and objective function categorizes the problem as Geometric Programming Problem (GPP). A polynomial is a function of the form ( 1 , 2 , … , ) = ∑ where all the coordinates and the coefficients are positive real numbers, and the exponents are real numbers.
Based on the number of objective functions involved, an optimization problem can be classified as Single /Multi Objective Optimization Problem (SOOP/MOOP) depending on whether it contains a single or more objective function.
A special class of optimization called Convex Optimization deals with the convex nature of objective function to be minimized and related constraints. In such problems due to the convexity of search space and the objective function , local minima is sufficient to guarantee the global minima. It is worth mentioning here the definitions of a convex set and convex function which are as follows: Convex set: A set ⊆ ℝ is convex if the line segment joining any two points of it lies inside the set itself. Mathematically, ∀ , ∈ ; . (Boyd and Vandenberghe, 2004). However unbounding this parameter by letting it free on the set of reals, gives rise to an affine set where the whole infinite line through any two points of lies completely inside . Thus for an affine set , the linear combination of and i.e. ∀ , ∈ , 1 + 2 ∈ where 1 + 2 = 1. Hence an affine set is a space in its own regard.
Convex function: A real-valued function : ⟶ ℝ defined is convex if the domain is a convex set and the line segment joining any two points on the graph of lies above or on the graph i.e.
If the symbol ′ ≤ ′ is replaced by the symbol of equality ′ = ′ then is an affine function.

Multi-Objective Optimization
In view of the multi criteria nature of most of the real world problems, one certainly faces an objective function addressing more than one attributes to be optimized in the related optimization problem. For instance, a person wants to buy a cooling device which can serve his two objectives of maximum comfort and least cost (Figure 1). Among the large range of such devices with an obvious direct proportion between comfort and cost, he will eventually settle down to something which on average fulfill both objectives. He cannot maximize comfort without compromising his penny pincher attitude. It is evident that another customer may select a device based on the same attributes but having values quite different from . This triggers the existence of a number (possibly infinite) of Pareto optimal solutions for a MOOP. Applications involving simultaneous optimization of several incommensurable problems like curve fitting (Ahonen et al., 1997), proteins atomic structure determination (Bush et al., 1995), pattern recognition of X-ray diffraction (Paszkowicz, 1996), potential function parameter optimization (Skinner and Broughton, 1995), production scheduling (Swinehart et al., 1996) and design of complex hardware/software systems (Zitzler, 1999) have been posed beautifully. The general MOOP problem can be defined as: Given ∶ → ℝ , ( ≥ 2) and = 1 × 2 × … × ∋ = ( 1 , 2 , … , ) where, the objective to be maximized (if any) is converted into an objective to be minimized by taking its negative. Other symbols are same as earlier we defined in section 1. An n-dimensional vector of optimization variables = ( 1 , 2, … , ) which satisfies all the constraints is called feasible solution and the subset ⊆ containing all of them is feasible decision space or feasible region or search space. The image of under i.e. ( ) ⊆ ℝ is feasible criterion space.
Apparently, in multi-objective optimization, a single solution is scarcely the finest for all the objectives simultaneously. This triggers the emergence of paying attention to a refined set ⊆ of solutions that may be improved further for some ( ), but only at the cost of degrading at least other component ≠ ( ) of the objective vector function ( ). Nevertheless each member of * ∈ dominates (or is non dominated by) each member of ∈ \ in the sense of functional values.
Mathematically, ∀ * ∈ , ∀ ∈ \ ; ( * ) ≤ ( ) holds for each ∈ {1,2, … , } and ( * ) < ( ) for at least one ∈ {1,2, … , }. This refined set is called as Pareto optimal solution set and its range ( ) is called as Pareto front or Pareto boundary (Figure 2). It is notable that any two solutions * , * ∈ are incomparable as neither of them can dominate the other in all objectives. The size and shape of Pareto optimal front generally depend upon the interaction among objective functions and their quantity (Deb, 1999). The conflicting nature of objectives result in a Pareto front with a larger span as compared to the case of cooperating objectives

Nature Inspired Optimization Techniques
There are different methods capable of finding solutions to optimization problems but the class which gained popularity is of Meta Heuristics methods. We are reviewing some popular nature inspired Meta Heuristics techniques useful in reliability optimization of complex systems and which in due course of time further laid the foundation to different subsidiary methods.

Ant Colony Optimization (ACO)
The Ant Colony Algorithm was proposed by Dorigo (1992). His source of inspiration for this was the foraging behavior of some ant species. Nature has given them the ability to discover the shortest path between their nest and food source. He found that ants living in a group are well versed to cooperate and find easily the shortest path in search of food, but single ant cannot (Dorigo, 1994). Initially, an ant randomly explores the area surrounding their nest and on locating the food spot, it analyzes food quantity and during its return journey brings some food on it back to the nest. During the return journey, a chemical named pheromone dropped by the ant on the ground. The quantity of pheromone deposited, which may depend on the quantity and quality of the food, will guide other ants to the food source. The other ants can sense this chemical in their way of searching food. The rest of the ants choose the way which has higher pheromone. As a result, pheromone accumulates faster in the shorter path and eventually all the ants converge to it. The basic idea of a real ant system can be depicted in Figure 3. Numerous ACO algorithms have been proposed by the researchers, of which the first one, Ant System (AS) (Dorigo, 2006) can be understood in the context of Travelling Salesman Problem. In a network of cities, where ( , ) represents the edge joining city to city . Suppose is the pheromone associated with the edge ( , ), which is updated by all the ants involved in building the solution as follows: where, is the evaporation rate, ∆ is the amount of pheromone laid on edge ( , ) by ant : where, is a constant, and is the length of the tour constructed by ant .
A stochastic mechanism involving the probability is followed by an ant at city to move towards city , which is given by: where, is the set of all adjacent vertices to vertex which are not visited yet. is the desirability of transition from to for ant and the parameter ≥ 1 is to control its influence. Typically, = 1 where is the distance between and .

Particle Swarm Optimization (PSO)
Particle Swarm Optimization (PSO) is a population-based search algorithm, first proposed by Eberhart and Kennedy (1995). The algorithm mirrors the behavior of animals' societies that don't have any permanent leader among them but they follow temporary leaders in small intervals of time during the quest of food (Pant et al., 2015b). Few classical examples like a flock of birds and school of fish, where the members of the group follow a specific member for a while which has the closest reach to the food source. Thus, the group tends to reach the food source optimally after the finite number of switches at the temporary leader position.
The PSO operates on a population of individuals ( ) called particles, where each particle ( ) represents a potential solution, flying through the search space. The position and velocity i.e. and respectively, are updated according to the relationship between the particles' parameters and the best location, the particle, and the population have found so far. The search is biased toward better regions of space, with the result being a sort of "flocking" toward the best solutions. Let and respectively denote the personal best position of particle and the fitness (objective) function. Then the personal best of particle at time step is updated as The personal best position is the best position (i.e. best fitness value) of the particle achieved so far. When smaller neighborhoods are used, the algorithm is generally referred to as an lbest PSO.
In case the neighborhood is the entire swarm, the best position in the neighborhood is referred to as the global best particle, and the resulting algorithm is referred to as a gbest PSO Pant et al., 2017b). For the gbest model, the best particle is determined from the entire swarm by selecting the best 'personal best position'. If the position of the global best particle is denoted by the vector , then ̂∈ { 0 ( ), 1 ( ), … , ( )} s.t. (( ) = min{ ( 0 ( )), ( 1 ( )), … , ( ( ))} For the stochastic purpose, the algorithm makes use of two independent random real sequences < 1 > and < 2 > in the interval (0,1). The values of 1 and 2 are scaled by two constants 1 and 2 respectively, called acceleration coefficients. These acceleration coefficients weight the stochastic terms 1 and 2 that pull each particle towards pbest and gbest positions. Thus the velocity and position of particle are updated using the following equations: ( + 1) = ( ) + ( + 1) Stopping criteria depends on the number of iteration or the process may stop if the velocity updates are close to zero. One can measure the quality of the particles using a fitness function and it reflects the optimality of a particular solution (Pant and Singh, 2011;Pant et al., 2015a).

Grey Wolf Optimizer (GWO)
Grey Wolf optimization algorithm (GWO) proposed by Mirjalili et al. (2014) is inspired by the socio-hunting behaviour of a wild animal named Grey wolves. They present its multi-objective version in the year 2016 (Mirjalili et al., 2016). This algorithm is mathematically modeled on the basis of hunting and encircling behavior of grey wolves during their hunting of prey ( Figure 4). This algorithm is governed by the following equations: Here, the position vector of the pray is denoted by

Reliability Optimization
For releasing the need for industry, a product designer must satisfy different criteria associated with product development. These are specific to maximizing product reliability and minimizing product cost. At the same time, he/she has to ensure the comfort of consumers and enhancing the functional safety of the product (Kumar and Singh, 2008;Kumar et al., 2017c).
Various researchers of the field of reliability engineering have applied the concepts of nature inspired optimization techniques to their reliability optimization problems (Table 1). Recently, the reliability-cost optimization of the life support system in space capsule by using a very recent metaheuristic named Multi-Objective Grey Wolf Optimizer (MOGWO) approach has been done by Kumar et al. (2019b). The efficiency of MOGWO in optimizing the reliability-cost of life support system has also been demonstrated by comparing its results with a very popular swarm based optimization technique named multi-objective particle swarm optimization. Kumar et al. (2019a) also presented a framework based on Grey Wolf Optimizer for technical specifications optimization of residual heat removal system of a nuclear power plant safety system.  (2007) Simulated Annealing (SA) Atiqullah and Rao (1993); Wattanapongsakorn and Lavitan (2004) Genetic Algorithm (GA) Coit and Smith (1996); Deeter and Smith (1997) Tabu Ramirez-Rosado and Bernal-Agustín (2001) Genetic Algorithm (GA) Huang et al. (2006)

Conclusion
In the last two decades, nature inspired optimization algorithms have witnessed an increasing interest amongst the community of reliability researches. In this article, mathematical background related to optimization and various aspects associated with nature inspired optimization algorithms in the context of reliability optimization are discussed which is beneficial for the researcher in the field of reliability optimization.