Volkan Vural
ABSTRACT
We introduce a framework, which we call Divide-by-2 (DB2), for extending support vector machines (SVM) to multi-class problems. DB2 offers an alternative to the standard one-against-one and one-against-rest algorithms. For an N class problem, DB2 produces an N − 1 node binary decision tree where nodes represent decision boundaries formed by N − 1 SVM binary classifiers. This tree structure allows us to present a generalization and a time complexity analysis of DB2. Our analysis and related experiments show that, DB2 is faster than one-against-one and one-against-rest algorithms in terms of testing time, significantly faster than one-against-rest in terms of training time, and that the cross-validation accuracy of DB2 is comparable to these two methods.
- Bennett, K. P., Cristianini, N., Shawe-Taylor, J., & Wu, D. (2000). Enlarging the margins in perceptron decision trees. Machine Learning, 41, 295--313. Google Scholar
Digital Library
- Blake, C., & Merz, C. (1998). UCI repository of machine learning databases.Google Scholar
- Crammer, K., & Singer, Y. (2000). On the learnability and design of output codes for multiclass problems. Computational Learing Theory (pp. 35--46). Google ScholarDigital Library
- Dumais, S. T., & Chen, H. (2000). Hierarchical classification of Web content. Proceedings of SIGIR-00, 23rd ACM International Conference on Research and Development in Information Retrieval (pp. 256--263). Athens, GR: ACM Press, New York, US. Google ScholarDigital Library
- Dy, J. G., & Brodley, C. E. (2000). Visualization and interactive feature selection for unsupervised data. Knowledge Discovery and Data Mining (pp. 360--364). Google ScholarDigital Library
- Forgy, E. (1965). Cluster analysis of multivariate data: Efficiency vs interpretability of classifications. Biometrics, 21, 768--780.Google Scholar
- Friedman, J. (1996). Another approach to polychotomous classification (Technical Report). Stanford University, Department of Statistics.Google Scholar
- Fukunaga, k. (1990). Introduction to statistical pattern recognition. Boston, MA: Academic Press. 2 edition. Google ScholarDigital Library
- Hsu, C., & Lin, C. A. comparison of methods for multiclass support vector machines. Technical report, Department of Computer Science and Information Engineering, National Taiwan University, Taipei, Taiwan, 2001, 19.Google Scholar
- Knerr, S., Personnaz, L., & Dreyfus, G. (1990). Single-layer learning revisited: A stepwise procedure for building and training a neural network. Neurocomputing: Algorithms, Architectures and Applications, NATO ASI Series. Springer.Google Scholar
- Platt, J. (1998). Sequential minimal optimization: A fast algorithm for training support vector machines. Technical Report 98-14, Microsoft Research, Redmond, Washington, April 1998. http://www.research.microsoft.com/jplatt/smo.html.Google Scholar
- Platt, J., Cristianini, N., & Shawe-Taylor, J. (2000). Large margin dags for multiclass classification. Advances in Neural Information Processing Systems 12 (pp. 547--553).Google Scholar
- Vapnik, V. (1995). The nature of statistical learning theory. New York: Springer. Google ScholarDigital Library
- Vapnik, V. (1998). Statistical learning theory. New York: Wiley. Google ScholarDigital Library
- Weston, J., & Watkins, C. (1999). Support vector machines for multiclass pattern recognition. Proceedings of the Seventh European Symposium On Artificial Neural Networks.Google Scholar
- A hierarchical method for multi-class support vector machines
Recommendations
A decision based one-against-one method for multi-class support vector machine
The support vector machine (SVM) has a high generalisation ability to solve binary classification problems, but its extension to multi-class problems is still an ongoing research issue. Among the existing multi-class SVM methods, the one-against-one ...
Wavelet twin support vector machines based on glowworm swarm optimization
Twin support vector machine is a machine learning algorithm developing from standard support vector machine. The performance of twin support vector machine is always better than support vector machine on datasets that have cross regions. Recently ...
Multi-view Laplacian twin support vector machines
Twin support vector machines are a recently proposed learning method for pattern classification. They learn two hyperplanes rather than one as in usual support vector machines and often bring performance improvements. Semi-supervised learning has ...
Comments