Swarm intelligence based classifiers☆
Introduction
Particle swarm optimization (PSO) is a swarm intelligence technique developed by Kennedy and Eberhart in 1995 [1]. In fact, natural flocking and swarm behavior of birds and insects inspired him to PSO. This technique has been used in several optimization and engineering problems (e.g. [2], [3]). In pattern recognition problems some particle swarm clustering techniques have been proposed (e.g. [4], [5]), but a swarm intelligence based classifier using PSO directly to obtain the decision functions in the feature space has not been implemented in the recent researches. In this paper an intelligent particle swarm classifier (IPS-classifier) is developed, integrating the concept of intelligently controlling the search process of PSO with a proposed particle swarm classifier (PS-classifier), which is designed to find the decision hyperplanes in the feature space.
An IPS-classifier has an additional intelligent controller to adapt the important parameters of PSO to increase its efficiency and means steering the swarm to an appropriate trajectory to find a better solution. Simulation results indicate that convergence to better hyperplanes with a lower number of iterations can be achieved using this technique. In fact, the IPS-classifier searches the solution space and chooses hyperplanes in such manner that the misclassified training points are minimized.
PSO is a simple and powerful search technique in high dimensional spaces, therefore IPS-classifier has the potentiality to successfully classify different classes in high dimensional feature spaces by achieving the separate hyperplanes, with a little priori information.
Any intelligent controller may be utilized to increase the efficiency of the classifier. In this article a fuzzy structure has been chosen and the IPS-classifier using this controller is called fuzzy controlled particle swarm classifier (FCPS-classifier).
The fuzzy controller rules have been extracted from a linguistic description from previous researches on study the effects of the PSO parameters on its search process (e.g. [6], [7], [8]).
Two common benchmark problems and a special problem in pattern recognition are considered to compare the proposed and the other methods with each other. The Iris data and the Wine data classification are common problems in pattern recognition researches with low and medium feature space dimensions and automatic target recognition for continuous wave radars is a special pattern recognition problem with high feature space dimensions. The performance of an IPS-classifier is compared with k-NN and MLP classifiers to show that the average recognition scores of designed IPS-classifier are better than or comparable to the traditional classifiers. To show the effectiveness of the intelligent fuzzy controller in the search process and correctly steering the swarm toward the solution, some meaningful figures are included.
In this paper, Section 2 explains a particle swarm classifier (PS-classifier). Intelligent particle swarm classifier is described in the next Section. Section 4 considers implementation of the classifier and experimental results on the three aforesaid pattern recognition problems. Finally, conclusion and discussion is presented in Section 5.
Section snippets
PSO algorithm
In the basic PSO technique proposed by Kennedy and Eberhart [1], great number of particles moves around in a multi-dimensional space and each particle memorizes its position vector and velocity vector as well as the time at which the particle has acquired the best fitness. Furthermore, related particles can share data at the best-fitness time. The velocity of each particle is updated with the best positions acquired for all particles over iterations and the best positions are acquired by the
Intelligent particle swarm classifier
It is known that the parameters of PSO play major roles in its search process (e.g. premature convergence, convergence rate, local capturing, exploitation, exploration, etc.). In the PSO defined in Section 2.1 inertia weight, cognitive parameter and social parameter are these three important factors. In the most of reported researches on the applications of PSO, these parameters have been achieved by running the PSO for several times with different sets of parameter to find a proper set. On the
Implementation and experimental results
Three pattern recognition problems with different augmented feature vectors dimensions (5,14,129) are used to show the performance of the IPS-classifier. A description of the data sets is given here.
Conclusion and discussion
Powerfulness and effectiveness of PSO algorithm, specially in high dimensional spaces, are motivations to design a swarm intelligence based classifier, using the basic PSO to obtain the decision hyperplanes in the feature space. Thus, the proposed classifier is a basic one. Now using the idea, anyone may construct the modified versions of the proposed classifier based on the many other kinds of PSO (maybe MPSO-classifier, QPSO-classifier, and so on).
Since inertia weight, cognitive parameter and
Acknowledgement
This research has been supported by the Iranian Telecommunication Research Center (ITRC) through Grant 500-6618.
References (26)
- J. Kennedy, R.C. Eberhart, Particle swarm optimization, Proc. IEEE Int. Conf. Neural Networks IV (1995)...
- Z. He, C. Wie, L. Yang, X. Gao, S. Yao, R.C. Eberhart, Y. Shi, Extracting rules from fuzzy neural network by particle...
- et al.
A particle swarm optimization for reactive power and voltage control considering voltage security assessment
IEEE Trans. Power Syst.
(2000) - D.W. Van der Merwe, A.P. Engelbrecht, Data clustering using particle swarm optimization, Proceedings of the 2003...
- Yan Yang, M. Kamel, Clustering ensemble using swarm intelligence, Proceedings of the 2003 IEEE Swarm Intelligence...
- Y. Shi, R.C. Eberhart, Fuzzy adaptive particle swarm optimization, Proceedings of the 2001 Congress on Evolutionary...
- W. Zhang, Y. Liu, M. Clerc, An adaptive PSO algorithm for reactive power optimization, Proc. Int. Conf. Adv. Power...
- et al.
The particle swarm-explosion, stability, and convergence in a multidimensional complex space
IEEE Trans. Evol. Comput.
(2002) - Y. Shi, R.C. Eberhart, A modified particle swarm optimizer, Proc. IEEE Int. Conf. Evol. Comput. (1998)...
- M. Clerc, The swarm and the queen: towards a deterministic and adaptive particle swarm optimization, Proceedings of the...
Cited by (65)
Optimal fuzzy controller parameters using PSO for speed control of Quasi-Z Source DC/DC converter fed drive
2015, Applied Soft Computing JournalCitation Excerpt :In PSO, every particle remembers its own previous best values as well as neighborhood best and therefore PSO has more effective memory capability than that of GA. In addition PSO is more efficient in maintaining the diversity of the swarm [57–61], since all the particles use the information related to the most successful particle in order to improve them, whereas in GA, the worse solutions are discarded and only the good ones are saved. Sedaghati and Babei [64] presented the different states of operation of double input Z-Source DC–DC converters, the input DC voltage can be boosted and deliver the power to load individually or the input DC sources may be combination of new energy sources.
Presenting a new multiclass classifier based on learning automata
2013, NeurocomputingCitation Excerpt :In testing phase, the trained classifier, assigns the unknown input pattern to one of the class based on the extracted feature vector. Until now, many methods have been proposed for pattern recognition; Well known methods such as K-Nearest Neighbor (K-NN) [1], Neural Network based classifier such as Multilayer Perceptron (MLP) [2], Genetic Algorithm (GA) classifier [3], Particle Swarm (PS) classifier [4], Support Vector Machine (SVM) [5], Decision Trees [6] and Fuzzy Methods [7] are some of them. The Learning Automata (LA) based pattern recognition methods [8–10] have also been the focus of attention in recent years.
Facing the classification of binary problems with a GSA-SVM hybrid system
2013, Mathematical and Computer ModellingClustering of Fuzzy Data Sets Based on Particle Swarm Optimization with Fuzzy Cluster Centers
2022, International Journal of Industrial Engineering and Production ResearchFuzzy Grasshopper Optimization Algorithm: A Hybrid Technique for Tuning the Control Parameters of GOA Using Fuzzy System for Big Data Sonar Classification
2022, Iranian Journal of Electrical and Electronic EngineeringSupport vector machine classification using semi-parametric model
2022, Soft Computing
- ☆
This paper is an extended version of the paper entitled “Intelligent Particle Swarm Classifiers”, which has been presented in the First International Conference on Modeling, Simulation and Applied Optimization (ICMSAO/05) and has been selected for the special issue of Journal of the Franklin Institute.