Abstract
This paper is concerned with the theoretical foundation of support vector machines (SVMs). The purpose is to develop further an exact relationship between SVMs and the statistical learning theory (SLT). As a representative, the standard C-support vector classification (C-SVC) is considered here. More precisely, we show that the decision function obtained by C-SVC is just one of the decision functions obtained by solving the optimization problem derived directly from the structural risk minimization principle. In addition, an interesting meaning of the parameter C in C-SVC is given by showing that C corresponds to the size of the decision function candidate set in the structural risk minimization principle.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Bartlett P L, Jordan M I, McAuliffe J D. Convexity, classification, and risk bounds. J Amer Statist Assoc, 2006, 101: 138–156
Bin J B, Vapnik V N. Learning with rigorous support vector machines. In: Proceedings of the 16th Annual Conference on Learning Theory (COLT’03). New York: Springer, 2003, 243–257
Blanchard G, Bousquet O, Massart P. Statistical performance of support vector machines. Ann Statist, 2008, 36: 489–531
Bonnans J F, Shapiro A. Perturbation Analysis of Optimization Problem. New York: Springer-Verlag, 2000
Boyd S, Vandenberghe L. Convex Programming. Cambridge: Cambridge University Press, 2004
Cortes C, Vapnik V N. Support-vector networks. Machine Learning, 1995, 20: 273–297
Cristianini N, Shawe-Taylor J. An introduction to Support Vector Machines and Other Kernel-Based Learning Methods. Cambridge: Cambridge University Press, 2000
Herbrich R. Learning Kernel Classifiers Theory and Algorithms. Cambridge: The MIT Press, 2002
Lin Y. A note on margin-based loss functions in classification. Statist Probab Lett, 2002, 68: 73–82
Scovel J C, Steiwart I. Fast rates for support vector machines using gaussian kernels. Ann Statist, 2007, 35: 575–607
Steinwart I. Consistency of support vector machines and other regularized kernel machines. IEEE Trans Inform Theory, 2005, 51: 128–142
Vapnik V N. Statistical Learning Theory. New York: John Wiley and Sons, 1998
Zhang T. Statistical behavior and consistency of classification methods based on convex risk minimization. Ann Statist, 2004, 32: 56–84
Author information
Authors and Affiliations
Corresponding author
Additional information
This work was partially supported by National Natural Science Foundation of China (Grant No. 10971223, 10601064), Key Project of National Natural Science Foundation of China (Grant No. 10631070, 70531040) and the Science Foundation of Renmin University of China (Grant No. 06XNB055)
Rights and permissions
About this article
Cite this article
Zhang, C., Tian, Y. & Deng, N. The new interpretation of support vector machines on statistical learning theory. Sci. China Ser. A-Math. 53, 151–164 (2010). https://doi.org/10.1007/s11425-010-0018-6
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11425-010-0018-6