Abstract
It has been shown by several researchers that multi-classifier systems can result in effective solutions to difficult tasks. In this work, we propose a multi-classifier system based on both supervised and unsupervised learning. According to the principle of “divide-and-conquer”, the input space is partitioned into overlapping subspaces and Support Vector Machines (SVMs) are subsequently used to solve the respective classification subtasks. Finally, the decisions of the individual SVMs are appropriately combined to obtain the final classification decision. We used the Fuzzy c-means (FCM) method for input space partitioning and we considered a scheme for combining the decisions of the SVMs based on a probabilistic interpretation. Compared to single SVMs, the multi-SVM classification system exhibits promising accuracy performance on well-known data sets.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
R. Maclin and D. Opitz. An empirical evaluation of bagging and boosting. In Proceedings of the Fourteenth National Conference on Artificial Intelligence, pages 546–551, 1997.
Y. Freund and R.E. Schapire. Experiments with a new boosting algorithm. In Proceedings of the Thirteenth International Conference on Machine Learning, pages 148–156, 1996.
R. Avnimelech and N. Intrator. Boosted mixture of experts: an ensemble learning scheme. Neural computation, 11(2):483–497, 1999.
D. Pavlov, J. Mao, and B. Dom. Scaling-up support vector machines using boosting algorithm. In Proceedings of the 15th International Conference on Pattern Recognition (ICPR2000), volume 2, pages 219–222, Barcelona, Spain, September 2000.
L. Kuncheva. Clustering-and-selection model for classifier combination. In Proceedings of the 4th International Conference on Knowledge-based Intelligent Engineering Systems (KES’2000), Brighton, UK, 2000.
E. Alpaydin. Techniques for combining multiple learners. In Proceedings of Engineering of Intelligent Systems, volume 2, pages 6–12. ICSC Press, 1998.
A. Vericas, A. Lipnickas, K. Malmqvist, M. Bacauskiene, and A. Gelzinis. Soft combination of neural classifiers: A comparative study. Pattern Recognition Letters, (20):430–441, 1999.
K. Tumer and J. Ghosh. Order statistics combiners for neural classifiers. In Proceedings of the World Congress on Neural Networks, pages I:31–34, Washington D.C., 1995. INNS Press.
A.J.C. Sharkey. Combining Artificial Neural Nets: Ensemble and Modular Multi-Net Systems. Springer Press, 1999.
M.I. Jordan and R.A. Jacobs. Hierarchical mixtures of experts and the em algorithm. Neural Computation, (6):181–214, 1994.
E. Alpaydin. Voting over multiple condensed nearest neighbor subsets. Artificial Intelligence Review, 11:115–132, 1997.
L. Kuncheva. Combining classifiers by clustering, selection and decision templates. Technical report, University of Wales, UK, 2000.
L. Breiman. Bagging predictors. Technical Report 421, Department of Statistics, University of California, Berkeley, 1994.
P.K. Chan and S.J. Stolfo. A comparative evaluation of voting and meta-learning on partitioned data. In Proceedings of the Twelfth International Machine Learning Conference, San Mateo,CA, 1995. Morgan Kaufmann.
J.C. Bezdek. Pattern Recognition with Fuzzy Objective Function Algorithms. Plenum Press, New York, 1981.
V. Vapnik. The Nature of Statistical Learning Theory. Springer-Verlag, New York, 1995.
Y. Cherkassky and F. Mulier. Learning from Data–Concepts, Theory and Methods. John Wiley & Sons. New York, 1998.
C. J. C. Burges. A tutorial on support vector machines for pattern recognition. Data Mining and Knowledge Discovery, 2(2):955–974, 1998.
B. Schölkopf, C.J.C. Burges, and V. Vapnik. Extracting support data for a given task. In U.M. Fayyad and R. Uthurusamy, editors, Proceedings, First International Conference on Knowledge Discovery & Data Mining. AAAI Press, Menlo Park, 1995.
OSU SVM Toolbox 2.00. [http://eewww.eng.ohio-state.edu/~maj/osu-svm/].
ESPRIT Basic Research Project ELENA (no. 6891). [ftp://ftp.dice.ucl.ac.be/pub/neural-nets/ELENA/databases], 1995.
R. Herbrich and J. Weston. Adaptive margin support vector machines for classification learning. In Proceedings of the Ninth International Conference on Artificial Neural Networks, pages 880–885, 1999.
Y.-J. Lee and O.L. Mangasarian. SSVM: A smooth support vector machine for classification. Technical Report 99-02, Data Mining Institute, 1999.
UCI Machine Learning Databases Repository, University of California-Irvine, Department of Information and Computer Science. [ftp://ftp.ics.edu/pub/machine-learning-databases].
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2001 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Frossyniotis, D.S., Stafylopatis, A. (2001). A Multi-SVM Classification System. In: Kittler, J., Roli, F. (eds) Multiple Classifier Systems. MCS 2001. Lecture Notes in Computer Science, vol 2096. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-48219-9_20
Download citation
DOI: https://doi.org/10.1007/3-540-48219-9_20
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-42284-6
Online ISBN: 978-3-540-48219-2
eBook Packages: Springer Book Archive