Swarm intelligence based classifiers

https://doi.org/10.1016/j.jfranklin.2005.12.006Get rights and content

Abstract

A proposed particle swarm classifier has been integrated with the concept of intelligently controlling the search process of PSO to develop an efficient swarm intelligence based classifier, which is called intelligent particle swarm classifier (IPS-classifier). This classifier is described to find the decision hyperplanes to classify patterns of different classes in the feature space. An intelligent fuzzy controller is designed to improve the performance and efficiency of the proposed classifier by adapting three important parameters of PSO (inertia weight, cognitive parameter and social parameter). Three pattern recognition problems with different feature vector dimensions are used to demonstrate the effectiveness of the introduced classifier: Iris data classification, Wine data classification and radar targets classification from backscattered signals. The experimental results show that the performance of the IPS-classifier is comparable to or better than the k-nearest neighbor (k-NN) and multi-layer perceptron (MLP) classifiers, which are two conventional classifiers.

Introduction

Particle swarm optimization (PSO) is a swarm intelligence technique developed by Kennedy and Eberhart in 1995 [1]. In fact, natural flocking and swarm behavior of birds and insects inspired him to PSO. This technique has been used in several optimization and engineering problems (e.g. [2], [3]). In pattern recognition problems some particle swarm clustering techniques have been proposed (e.g. [4], [5]), but a swarm intelligence based classifier using PSO directly to obtain the decision functions in the feature space has not been implemented in the recent researches. In this paper an intelligent particle swarm classifier (IPS-classifier) is developed, integrating the concept of intelligently controlling the search process of PSO with a proposed particle swarm classifier (PS-classifier), which is designed to find the decision hyperplanes in the feature space.

An IPS-classifier has an additional intelligent controller to adapt the important parameters of PSO to increase its efficiency and means steering the swarm to an appropriate trajectory to find a better solution. Simulation results indicate that convergence to better hyperplanes with a lower number of iterations can be achieved using this technique. In fact, the IPS-classifier searches the solution space and chooses hyperplanes in such manner that the misclassified training points are minimized.

PSO is a simple and powerful search technique in high dimensional spaces, therefore IPS-classifier has the potentiality to successfully classify different classes in high dimensional feature spaces by achieving the separate hyperplanes, with a little priori information.

Any intelligent controller may be utilized to increase the efficiency of the classifier. In this article a fuzzy structure has been chosen and the IPS-classifier using this controller is called fuzzy controlled particle swarm classifier (FCPS-classifier).

The fuzzy controller rules have been extracted from a linguistic description from previous researches on study the effects of the PSO parameters on its search process (e.g. [6], [7], [8]).

Two common benchmark problems and a special problem in pattern recognition are considered to compare the proposed and the other methods with each other. The Iris data and the Wine data classification are common problems in pattern recognition researches with low and medium feature space dimensions and automatic target recognition for continuous wave radars is a special pattern recognition problem with high feature space dimensions. The performance of an IPS-classifier is compared with k-NN and MLP classifiers to show that the average recognition scores of designed IPS-classifier are better than or comparable to the traditional classifiers. To show the effectiveness of the intelligent fuzzy controller in the search process and correctly steering the swarm toward the solution, some meaningful figures are included.

In this paper, Section 2 explains a particle swarm classifier (PS-classifier). Intelligent particle swarm classifier is described in the next Section. Section 4 considers implementation of the classifier and experimental results on the three aforesaid pattern recognition problems. Finally, conclusion and discussion is presented in Section 5.

Section snippets

PSO algorithm

In the basic PSO technique proposed by Kennedy and Eberhart [1], great number of particles moves around in a multi-dimensional space and each particle memorizes its position vector and velocity vector as well as the time at which the particle has acquired the best fitness. Furthermore, related particles can share data at the best-fitness time. The velocity of each particle is updated with the best positions acquired for all particles over iterations and the best positions are acquired by the

Intelligent particle swarm classifier

It is known that the parameters of PSO play major roles in its search process (e.g. premature convergence, convergence rate, local capturing, exploitation, exploration, etc.). In the PSO defined in Section 2.1 inertia weight, cognitive parameter and social parameter are these three important factors. In the most of reported researches on the applications of PSO, these parameters have been achieved by running the PSO for several times with different sets of parameter to find a proper set. On the

Implementation and experimental results

Three pattern recognition problems with different augmented feature vectors dimensions (5,14,129) are used to show the performance of the IPS-classifier. A description of the data sets is given here.

Conclusion and discussion

Powerfulness and effectiveness of PSO algorithm, specially in high dimensional spaces, are motivations to design a swarm intelligence based classifier, using the basic PSO to obtain the decision hyperplanes in the feature space. Thus, the proposed classifier is a basic one. Now using the idea, anyone may construct the modified versions of the proposed classifier based on the many other kinds of PSO (maybe MPSO-classifier, QPSO-classifier, and so on).

Since inertia weight, cognitive parameter and

Acknowledgement

This research has been supported by the Iranian Telecommunication Research Center (ITRC) through Grant 500-6618.

References (26)

  • J. Kennedy, R.C. Eberhart, Particle swarm optimization, Proc. IEEE Int. Conf. Neural Networks IV (1995)...
  • Z. He, C. Wie, L. Yang, X. Gao, S. Yao, R.C. Eberhart, Y. Shi, Extracting rules from fuzzy neural network by particle...
  • H. Yoshida et al.

    A particle swarm optimization for reactive power and voltage control considering voltage security assessment

    IEEE Trans. Power Syst.

    (2000)
  • D.W. Van der Merwe, A.P. Engelbrecht, Data clustering using particle swarm optimization, Proceedings of the 2003...
  • Yan Yang, M. Kamel, Clustering ensemble using swarm intelligence, Proceedings of the 2003 IEEE Swarm Intelligence...
  • Y. Shi, R.C. Eberhart, Fuzzy adaptive particle swarm optimization, Proceedings of the 2001 Congress on Evolutionary...
  • W. Zhang, Y. Liu, M. Clerc, An adaptive PSO algorithm for reactive power optimization, Proc. Int. Conf. Adv. Power...
  • M. Clerc et al.

    The particle swarm-explosion, stability, and convergence in a multidimensional complex space

    IEEE Trans. Evol. Comput.

    (2002)
  • Y. Shi, R.C. Eberhart, A modified particle swarm optimizer, Proc. IEEE Int. Conf. Evol. Comput. (1998)...
  • M. Clerc, The swarm and the queen: towards a deterministic and adaptive particle swarm optimization, Proceedings of the...
  • B. Al-kazemi, C.K. Mohan, Multi-phase generalization of the particle swarm optimization algorithm, Proceedings of the...
  • N. Higashi, H. Iba, Particle swarm optimization with Gaussian mutation, Proceedings of the 2003 IEEE Swarm Intelligence...
  • S. Yang, M. Wang, L. Jiao, A quantum particle swarm optimization, Proceedings of the 2004 Congress on Evolutionary...
  • Cited by (65)

    • Optimal fuzzy controller parameters using PSO for speed control of Quasi-Z Source DC/DC converter fed drive

      2015, Applied Soft Computing Journal
      Citation Excerpt :

      In PSO, every particle remembers its own previous best values as well as neighborhood best and therefore PSO has more effective memory capability than that of GA. In addition PSO is more efficient in maintaining the diversity of the swarm [57–61], since all the particles use the information related to the most successful particle in order to improve them, whereas in GA, the worse solutions are discarded and only the good ones are saved. Sedaghati and Babei [64] presented the different states of operation of double input Z-Source DC–DC converters, the input DC voltage can be boosted and deliver the power to load individually or the input DC sources may be combination of new energy sources.

    • Presenting a new multiclass classifier based on learning automata

      2013, Neurocomputing
      Citation Excerpt :

      In testing phase, the trained classifier, assigns the unknown input pattern to one of the class based on the extracted feature vector. Until now, many methods have been proposed for pattern recognition; Well known methods such as K-Nearest Neighbor (K-NN) [1], Neural Network based classifier such as Multilayer Perceptron (MLP) [2], Genetic Algorithm (GA) classifier [3], Particle Swarm (PS) classifier [4], Support Vector Machine (SVM) [5], Decision Trees [6] and Fuzzy Methods [7] are some of them. The Learning Automata (LA) based pattern recognition methods [8–10] have also been the focus of attention in recent years.

    • Clustering of Fuzzy Data Sets Based on Particle Swarm Optimization with Fuzzy Cluster Centers

      2022, International Journal of Industrial Engineering and Production Research
    View all citing articles on Scopus

    This paper is an extended version of the paper entitled “Intelligent Particle Swarm Classifiers”, which has been presented in the First International Conference on Modeling, Simulation and Applied Optimization (ICMSAO/05) and has been selected for the special issue of Journal of the Franklin Institute.

    View full text