Abstract
A fast SVM training algorithm for multi-classes consisting of parallel and sequential optimizations is presented. The main advantage of the parallel optimization step is to remove most non-support vectors quickly, which dramatically reduces the training time at the stage of sequential optimization. In addition, some strategies such as kernel caching, shrinking and calling BLAS functions are effectively integrated into the algorithm to speed up the training. Experiments on MNIST handwritten digit database have shown that, without sacrificing the generalization performance, the proposed algorithm has achieved a speed-up factor of 110, when compared with Keerthi et al.’s modified SMO. Moreover, for the first time ever we investigated the training performance of SVM on handwritten Chinese database ETL9B with more than 3000 categories and about 500,000 training samples. The total training time is just 5.1 hours. The raw error rate of 1.1% on ETL9B has been achieved.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Schölkopf, B., Burges, C.J.C., Vapnik, V.: Extracting support data for a given task. In: Proceedings of the First International Conference on Knowledge Discovery and Data Mining, Menlo Park, CA (1995) 252–257
DeCoste, D., Schölkopf, B.: Training invariant support vector machines. Machine Learning. 46(1–3) (2002) 161–190
Joachims, T.: Text categorization with support vector machine: learning with many relevant features. In: Proceedings of 10th European Conference on Machine Learning (ECML) (1998) 137–142
Osuna, E., Freund, R., Girosi, F.: Training support vector machines: An application to face detection. In: Proceedings of the 1997 conference on Computer Vision and Pattern Recognition(CVPR’97), Puerto Rico (1997) 130–136
Platt, J.C.: Fast training of support vector machines using sequential minimal optimization. In: Schölkopf, B., Burges, C.J.C., Smola, A. (eds.): Advances in kernel methods: Support Vector Machines, MIT Press, Cambridge, MA (1998) 185–208
Joachims T.: Making large-scale support vector machine learning practical. In: Schölkopf, B., Burges, C.J.C., Smola, A. (eds.): Advances in kernel methods: Support Vector Machines, MIT Press, Cambridge, MA (1998) 169–184
Dongarra, J.J., Croz, J.D., Duff, I.S., Hammarling, S.: A set of level 3 basic linear algebra subprograms. ACM Trans. Math. Soft. 16 (1990) 1–17
Kuhn, H., Tucker, A.: Nonlinear programming. In: Proceedings of 2nd Berkeley Symposium on Mathematical Statistics and Probabilistics. University of California Press (1951) 481–492
Dong, J.X., Suen, C.Y., Krzyżak, A.: A fast svm training algorithm. In: Lee, S.-W., Verri, A. (eds.): Pattern Recognition with Support Vector Machines. Springer Lecture Notes in Computer Science LNCS 2388, Niagara Falls, Canada (2002) 481–492
Keerthi, S.S., Shevade, S.K., Bhattachayya, C., Murth K.R.K.: Improvements to Platt’s SMO algorithm for SVM classifier design. Neural Computation, 13 (2001) 637–649
Dong, J.X., Krzyżak, A., Suen, C.Y.: A fast svm training algorithm. International Journal of Pattern Recognition and Artificial Intelligence, 17(3) (2003) 1–18
Saito, T., Yamada, H., Yamamoto, K.: An analysis of handprinted character database VIII: An estimation of the database ETL9 of handprinted characters in JIS Chinese characters by directional pattern matching approach. Bul. Electrotech 49(7) (1985) 487–525
Dong, J.X., Suen, C.Y., Krzyżak, A.: High accuracy handwritten Chinese character recognition using support vector machine. Tech. Rep. CENPARMI, Concordia University, Canada, (2003)
Yamada, H., Yamamoto, K., Saito, T.: A nonlinear normalization method for handprinted kanji character recognition-line density equalization. Pattern Recognition, 23(9) (1990) 1023–1029
Duda, R.O., Hart, P.E.: Pattern Classification and Scene Analysis. John Wiley & Sons. Inc, New York (1973)
Kimura, F., Wakabayashi T., Tsuruoka, S., Miyake, Y.: Improvement of handwritten Japanese character recognition using weighted direction code histogram. Pattern Recognition, 30(8) (1997) 1329–1337
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2003 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Dong, Jx., Krzyżak, A., Suen, C. (2003). A Fast Parallel Optimization for Training Support Vector Machine. In: Perner, P., Rosenfeld, A. (eds) Machine Learning and Data Mining in Pattern Recognition. MLDM 2003. Lecture Notes in Computer Science, vol 2734. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-45065-3_9
Download citation
DOI: https://doi.org/10.1007/3-540-45065-3_9
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-40504-7
Online ISBN: 978-3-540-45065-8
eBook Packages: Springer Book Archive