A Hybrid Algorithm Based on PSO and GA for Feature Selection

: One of the main problems of machine learning and data mining is to develop a basic model with a few features, to reduce the algorithms involved in classification’s computational complexity. In this paper, the collection of features has an essential importance in the classification process to be able minimize computational time, which decreases data size and increases the precision and effectiveness of specific machine learning activities. Due to its superiority to conventional optimization methods, several metaheuristics have been used to resolve FS issues. This is why hybrid metaheuristics help increase the search and convergence rate of the critical algorithms. A modern hybrid selection algorithm combining the two algorithms; the genetic algorithm (GA) and the Particle Swarm Optimization (PSO) to enhance search capabilities is developed in this paper. The efficacy of our proposed method is illustrated in a series of simulation phases, using the UCI learning array as a benchmark dataset.


Introduction
The selection of features is a complicated work because challenging functional interactions are possible. When interacting with other functions, an individually appropriate element can be redundant. Due to the vast search area, the feature selection process is complex. The search area is increasingly increasing in size with regard to the amount of data set features [1]. Consequently, a complete search is almost futile in most situations. Many methods for active filtering, such as Sequential Backward Selection (SBS) [2], have been applied in order to solve this problem. However, these methods to select features also face a range of challenges such as local optima inflation and high computational costs. An effective global search technique is essential in order to help address feature selection issues. Methods of Evolutionary Computing (EC) are famous for their global search capability. PSO is a relatively new EC technique focused on swarm intelligence [3,4].
The technology is relatively modern. PSO is computationally cheaper and can converge more rapidly than other EC algorithms like Gas. Reasons why PSO was brought into for this field, including a collection of features as an essential technique [5][6][7]. This technique was introduced by Eberhart et al. [8] as a population-based stochastic optimizer technique. PSO simulates a swarm of insects, an assembly of birds, or a fish school's social behavior. PSO shares certain parallels with evolutionary computer technologies such as genetic algorithms and artificial neural networks. It comes in-between with a random population and upgrades generations to scan for optima. If one particle finds a decent way to eat, it is possible to immediately obey the rest of the colony, such as the Swarm of Ants.
In a multidimensional research area, the swarm behavior is modeled on particles with positions and velocity as features. In hyperspace, these particles feed by adapting their location and speed.
Inspired by PSO and GA, a new evolutionary hybrid algorithm (HEA) is developed by integrating the advantages of both algorithms to determine the most likely features in the chosen subset of features. The technique of the profit ratio is used to achieve this.
The swarm conduct is modeled on particles with locations and velocity in a multidimensional research environment. These particles feed through hyperspace by adjusting their own position and direction.
It is how the majority of the document is arranged. We provide history material in Chapter 2. The related works are discussed in Chapter 3. The new algorithm, the strategy bundle creation methods, is defined in Chapter 4 while Chapter 5 discusses the nature of the experiments, and Chapter 6 discusses the findings. The results and prospective research work are presented in Chapter 7.

Literature Survey
Numerous optimization algorithms to solve the complicated optimization problems have been recently proposed, as this vast number of algorithms have different behavior. We classify it into three major categories: evolutionary algorithm, dependent on Swarm and physics. The evolutionary algorithm is based on biological evolution that integrates Darwinian evolutionary principles. They are most famous (GA). The solutions are initially created by random means, and those are modified with each algorithm iteration. To effectively look for the ideal solution GA uses the crossover and mutation operators. As the perfect situation is more likely to take part in new offspring production, the new approach is predicted to be better [8]. Secondly, the Swarm is founded on the action of swarms, stoves, schools or animal flocks in nature. The search officers' activities are navigated through collective, social and intelligence simulation. The best-known swarm-based metaheuristic is PSO. It is inspired by birds flocking social behavior. It uses the best global approach in order to ensure research and the best local place for exploitation. ACO is another standard algorithm for this class. Thirdly, physics focused on physical law motivations. Annealing (SA) and HS algorithms simulation are the oldest algorithms concentrate on physics. SA is inspired by the metallurgical annealing process, in which metals are heated up to a certain degree and the impurity removal is accompanied by regulated cooling. The HS algorithm is influenced by concerts where musicians strive to produce the most delicate harmonic harmony. This phenomenon has been optimized for optimization in the HS algorithm. The GSA that is an imitation of Newton's gravitational law is another standard algorithm of this family. The search agents are called masses interacting with each other to find the optimum solution in accordance with this rule. Charged machine research (CSS), complies with the Electrostatics Law of Coulomb and the Mechanicals Law of Newton, aiming to find the best solutions effectively. Equilibrium Optimizer is the algorithm proposed recently, inspired by a wellmixed dynamic equilibrium on the volume of control. Mass balance equation is used to characterize nonreactive component concentration in the volume of control.

Feature Selection
The selection of features seeks to choose a range of features from the primary data collection, which can efficiently clarify input data while reducing the influence of noise and misfits.
1. Identification of a subset: the detection stage for candidate subsets' development. This stage is the stage of searching, which is possible to start with all variables or a subset of random variables without any function. The search methods at this point include traditional methodologies and EC techniques for producing subsets of features.
2. Subset assessment: an evaluation criterion used to calculate the desirability level of the subsets generated.
3. State of hold: the algorithm's execution starts with realizing the stop condition and ends until the stop situation has been realized. The separating condition may be specified based on an analysis function. 4. Validity: the accreditation of the chosen subset is the subject of this point. This accreditation is done using simulated databases (or real-world data sets), by experimenting and analyzing the effects of the proposed algorithm using former methods.

Particle Swarm Optimization (PSO)
This approach of the Particle Swarm Optimization (PSO) [9], it simulates a swarm of flies, an assembly of birds, or a fish school's social behavior. PSO has many parallels with evolving calculation technologies, such as genetic algorithms and artificial neural networks. It is initialized by a random solution population and updates generations to look for optima. If one particle finds a decent way to eat, it is possible to immediately obey the rest of the colony, such as the Swarm of ants or even the fish school. In multidimensional research space [10], the swarming behavior is modeled on particles with characteristically location and speed. These particles drink their own position and velocity by changing the hyperspace.
Although the PSO has proven itself to be an efficient way to find optimum final sub-sets of functions, it has some deficiencies. One of the problems of PSO-based filtering processes is that the number of search features was abandoned [11] while only optimizing the precision of the classification. Related functions in the final function subset are more limited. Furthermore, the current PSO-based methods do not use features' correlation knowledge to direct the likelihood of the search process to be chosen in good functions that improve the classification accuracy.

Genetic Algorithm (GA)
In 1960 Dr. Richenberg, whose thesis focused on evolutionary planning, first proposed evolutionary calculations. Later on, several scholars investigated his hypothesis in 1975, John Holland at the University of Michigan invented the genetic algorithm. This algorithm begins with an appropriate "k" response called the initial population, then calculates the value of each response using the fitness function, and two solutions are chosen from the available responses with the desired parent utility [11]; new children then come into being by mixing parent answers, added by the predominant population into the new population [11].

Related Work
Existing feature selection algorithms may be categorized into two categories: approaches wrapper and filter methods. Wrapper approaches use an algorithm for learning/classification in the assessment process, while filter approaches do not. The solution of filters is argued to be less costly and generally computerized [12], while wrappers will usually produce improved results [13]. There have been several proposals for algorithms for selecting features over the last few years. This section reviews a typical function selection algorithm. The comparative output of the two networks demonstrates that the Wavelet network is statistically more critical than MLP. In identifying common mental disorders causes Ludermir et al. [14] used PSO techniques to identify them. Nasir et al. [15] present a hybrid algorithm that combines the Regrouping Particle Swarm Optimization (RegPSO) with the RFWN that is used to track, classify and characterize acoustic signals as a result of the surface drag operation. In order to deal with problems of nonlinear mixed-integer optimization in series, parallel series and bridge schemes, Shoo et al. [16] have created a hybrid approach. Naik et al. have suggested an effective classification system based on hybrid PSO and GA ANN [17] and are found in comparison to other alternatives to achieve relative improvement in efficiency. PSO has been widely used in the field of FS. A sigmoid function was used to translate location values into probability to include features in binary PSO [18]. In certain instances, both continuous and discrete PSO were used in conjunction to optimize SVM and function parameters [19]. A distributed framework was used, with the server performing PSO calculations and the client performing SVM training and testing. PSO was changed to form geometric PSO in [20]. New agents were created using crossover operations on the current agents, the local best, and the best in the world, before mutating the agent created after crossover. For gene selection, the algorithm was used. The use of a rough set to perform FS was another slight change to PSO. To perform text clustering, Abualigah et al. proposed a hybrid algorithm combining PSO and genetic operators in [21]. By choosing a better collection of insightful features, the proposed hybrid model enhanced the efficiency of the k-means clustering algorithm. Kanan et al. [22] suggested another method for text document clustering that used adaptive PSO to find a more informative subset of features while also reducing the time required.
The best result of the two was taken in each iteration [23]. For protein function prediction, the hybrid algorithm used FS. A similar approach was taken, but this time the application domain was text classification. Using the SVM classifier, the researchers contrasted the use of ACO, GA, and PSO on siRNA data.
The fact that both GA and PSO outperformed ACO was a key finding. Ghosh et al. [12] proposed an embedded ACO called wrapper filter ACOFS, which uses a filter approach to test function subsets and reduces the overall model's time requirement. The writers have used memory to keep track of the best outcomes through generations. Abualigah et al. [21] proposed a novel text clustering approach based on Krill herd (KH).
MMKHA was developed for eight text datasets and used as a hybrid-enhanced KH algorithm. Chuang et al. [24] proposed approach for the combined objective and hybrid KH algorithms of the text record clustering. Abualigah et al. [21] an improved KH algorithm in the cluster of text papers.

Proposed Method
Following the GA, our system is applied, and every step has pushed the population towards new, better answers. The cost function estimates the expense of a member of the population, thus retaining the speed of the population, and then changes the Gbest and Pbest of each member concerning new costs. Pbest refers to the best reaction each population has achieved up until now.
It must indeed be remembered that GA does not have the potential to exploit and the only root of GA manipulation is a mutation that causes very little chromosome disturbance. However, utilizing crossover operations, GA can do a remarkable exploration of the quest space. When considering PSO, it must be noted that PSO has strong local search capability but that it cannot achieve adequate exploration. PSO is always trapped in a local optima that obstructs the ability to explore. These complementary GA and PSO commercialization for exploitation and discovery encourage one to combine its findings to obtain efficient and usable results.
GA and PSO are two separate types of FS algorithms, namely evolutionary and swarm intelligence. GA is helpful for transferring useful functions from generation to generation. PSO has the benefit of thoroughly searching the search space with particles related to each other's function data. These advantages of PSO and GA are combined into exploiting and exploring in a balanced way.
This section will introduce our algorithm in detail as follows: Step 1 Determine how many features Step 2 Population Initialization.
Step 3 Update the particle positions.
Step 4 Cover ideological principles of the present method Step 5 Gbest mutation.
Step 6 k-Nearest Neighbor (kNN) At first, all the features of the dataset are provisional analyzed. The objective is to classify features more likely to exist in the chosen function subset. The technique of the profit ratio is used to achieve this. The algorithm aims to select features that make the most significant difference in each class as a split function for the decision tree branches. The profit ratio is a change in the knowledge gain, which decreases the difference between the high branch attributes. When selecting an attribute, the benefit ratio takes account of number and scale of branches.

Datasets and Parameter Setting
In our simulation experiments, seven data sets are followed to verify the efficiency of our suggested algorithm compared with the methods. These data sets are taken from the UCI learning device shown in Tab. 1. The data sets contain various variables, groups, and examples that prepare a comprehensive survey of the proposed approach [29] and the approaches used. One of the most common KNN algorithms was used to calculate the classification efficiency. k is calibrated for 5 in these experiments (5NN). The 7 data sets used in the trials selected from the UCI machine learning repository are seen above from Tab. 1. The function numbers of the PSO-based multi-target selection algorithms were chosen (from 13 to 166), groups (from 2 to 3), and instances (from 32 to 1 000).
In each sample of data, both cases are split randomly into two groups during the experiments: 70 percent as the testing set and 30 percent as the evaluation set. Each particle (single) constitutes a sub-set of functions during the training phase.

Experimental Results and Analysis
The maximum number of iterations and also the population play an essential role when talking about the multi-agent evolutionary algorithm in characterizing the actions of one agent from other agents' knowledge and the progressive progression of the agents in each other.

Comparison
The power of the proposed FS model will be determined by the application to different datasets. The model's efficiency is based on the performance of LFFS, LFS and our approach suggested in Tab. 2. This section provides the related material for the experiments.

Conclusion
We also suggested a hybrid approach for FS in this paper based on a popular meta-heuristic called the algorithm PSO and GA. The proposed algorithm has been compared to ten state-of-the-art metaheuristic and hybrid metaheuristic FS methods using seven regular UCI datasets. The results obtained show that the algorithm proposed is superior to other approaches. We may then conclude that our way of solving FS problems is a competitive process. If we observe the findings, our algorithm manages to solve the PSO discovery and exploitation limitations. But in some situations, global optima may not be found according to the problem's need. We need to do a thorough experiment simultaneously to find the optimal value of the parameters used for various problems in this algorithm. The proposed algorithm can be applied to other common and interesting problems in research as a potential scope of work, such as facial recognition.

Conflicts of Interest:
The authors whose names are listed immediately below report the following details of affiliation or involvement in an organization or entity with a financial or non-financial interest in the subject matter or materials discussed in this manuscript.