Abstract
Particle Swarm Optimization (PSO) is a popular and bionic algorithm based on the social behavior associated with bird flocking for optimization problems. To maintain the diversity of swarms, a few studies of multi-swarm strategy have been reported. However, the competition among swarms, reservation or destruction of a swarm, has not been considered further. In this paper, we formulate four rules by introducing the mechanism for survival of the fittest, which simulates the competition among the swarms. Based on the mechanism, we design a modified Multi-Swarm PSO (MSPSO) to solve discrete problems, which consists of a number of sub-swarms and a multi-swarm scheduler that can monitor and control each sub-swarm using the rules. To further settle the feature selection problems, we propose an Improved Feature Selection (IFS) method by integrating MSPSO, Support Vector Machines (SVM) with F-score method. The IFS method aims to achieve higher generalization capability through performing kernel parameter optimization and feature selection simultaneously. The performance of the proposed method is compared with that of the standard PSO based, Genetic Algorithm (GA) based and the grid search based methods on 10 benchmark datasets, taken from UCI machine learning and StatLog databases. The numerical results and statistical analysis show that the proposed IFS method performs significantly better than the other three methods in terms of prediction accuracy with smaller subset of features.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Guyon I, Elisseeff A. An introduction to variable and feature selection. Journal of Machine Learning Research, 2003, 3, 1157–1182.
Kennedy J, Eberhart R. Particle swarm optimization. Proceedings of the IEEE International Conference on Neural Network, Perth, Australia, 1995, 1942–1948.
Eberhart R, Kennedy J. A new optimizer using particle swarm theory. Proceedings of the Sixth International Symposium on Micro Machine and Human Science, Nagoya, Japan, 1995, 39–43.
Lin S W, Ying K C, Chen S C, Lee Z J. Particle swarm optimization for parameter determination and feature selection of support vector machines. Expert Systems with Applications, 2008, 35, 1817–1824.
Huang C L, Dun J F. A distributed PSO-SVM hybrid system with feature selection and parameter optimization. Applied Soft Computing, 2008, 8, 1381–1391.
Blackwell T. Particle swarms and population diversity. Soft Computing, 2005, 9, 793–802.
Blackwell T, Branke J. Multiswarms, exclusion, and anti-convergence in dynamic environments. IEEE Transactions on Evolutionary Computation, 2006, 10, 459–472.
Parrott D, Li X D. Locating and tracking multiple dynamic optima by a particle swarm model using speciation. IEEE Transactions on Evolutionary Computation, 2006, 10, 440–458.
Niu B, Zhu Y L, He X X, Wu H. MCPSO: A multi-swarm cooperative particle swarm optimizer. Applied Mathematics and Computation, 2007, 185, 1050–1062.
Chen Y W, Lin C J. Combination of feature selection approaches with SVM in credit scoring. Expert Systems with Applications, 2006, 37, 315–324.
Kennedy J, Eberhart R. A discrete binary version of the particle swarm algorithm. Proceedings of the IEEE International Conference on Systems, Man and Cybernetics, Orlando, USA, 1997, 4104–4108.
Vapnik V N. The Nature of Statistical Learning Theory, 2nd ed, Springer, New York, 1999.
Boser B E, Guyon I M, Vapnik V N. A training algorithm for optimal margin classifiers. Proceedings of the fifth Annual Workshop on Computational Learning Theory, Pittsburgh, USA, 1992, 144–152.
Clerc M, Kennedy J. The particle swarm-explosion, stability, and convergence in a multidimensional complex space. IEEE Transactions on Evolutionary Computation, 2002, 6, 58–73.
Trelea I C. The particle swarm optimization algorithm: convergence analysis and parameter selection. Information Processing Letters, 2003, 85, 317–325.
Kadirkamanathan V, Selvarajah K, Fleming P J. Stability analysis of the particle dynamics in particle swarm optimizer. IEEE Transactions on Evolutionary Computation, 2006, 10, 245–255.
Jiang M, Luo Y P, Yang S Y. Stochastic convergence analysis and parameter selection of the standard particle swarm optimization algorithm. Information Processing Letters, 2007, 102, 8–16.
Shi Y, Eberhart R. Modified particle swarm optimizer. Proceedings of IEEE International Conference on Evolutionary Computation, Anchorage, USA, 1998, 69–73.
Zhan Z H, Zhang J, Li Y. Adaptive Particle Swarm Optimization. IEEE Transactions on Systems Man and Cybernetics Part B-Cybernetics, 2009, 39, 1362–1381.
Xie J Y, Wang C X. Using support vector machines with a novel hybrid feature selection method for diagnosis of ery- themato-squamous diseases. Expert Systems with Applications, 2011, 38, 5809–5815.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Liu, Y., Wang, G., Chen, H. et al. An improved particle swarm optimization for feature selection. J Bionic Eng 8, 191–200 (2011). https://doi.org/10.1016/S1672-6529(11)60020-6
Published:
Issue Date:
DOI: https://doi.org/10.1016/S1672-6529(11)60020-6