Skip to main content
SpringerLink
Log in

Robust and Sparse Linear Programming Twin Support Vector Machines

  • Published:
Cognitive Computation Aims and scope Submit manuscript

Abstract

In this paper, we propose a new linear programming formulation of exact 1-norm twin support vector machine (TWSVM) for classification whose solution is obtained by solving a pair of dual exterior penalty problems as unconstrained minimization problems using Newton–Armijo algorithm. The idea of our formulation is to reformulate TWSVM as a strongly convex problem by incorporated regularization techniques and then derive an exact 1-norm linear programming formulation for TWSVM to improve robustness and sparsity. The solution of two modified unconstrained minimization problems reduces to solving just two systems of linear equations as opposed to solving two quadratic programming problems in TWSVM and TBSVM, which leads to extremely simple and fast algorithm. One significant advantage of our proposed method is the implementation of structural risk minimization principle. However, only empirical risk is considered in the primal problems of TWSVM due to its complex structure and thus may incur overfitting and suboptimal in some cases. Our approach has the advantage that a pair of matrix equation of order equals to the number of input examples is solved at each iteration of the algorithm. The algorithm converges from any starting point that can be easily implemented in MATLAB without using any optimization packages. Computational comparisons of our proposed method against original TWSVM, GEPSVM and SVM have been made on both synthetic and benchmark datasets. Experimental results show that our method is better or comparable in both computation time and classification accuracy.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  1. Balasundaram S, Tanveer M. On Lagrangian twin support vector regression. Neural Comput Appl. 2013;22(1):257–67.

    Article  Google Scholar 

  2. Bradley PS, Mangasarian OL. Feature selection via concave minimization and support vector machines. In: Machine learning proceedings of the fifteenth international conference (ICML ’98). 1998; p. 82–90.

  3. Brown MPS, Grundy WN, Lin D. Knowledge-based analysis of microarray gene expression data using support vector machine. In: Proceedings of the national academy of sciences of USA. 2000; 97(1):262–7.

  4. Burges C. A tutorial on support vector machines for pattern recognition. Data Min Knowl Discov. 1998;2:1–43.

    Article  Google Scholar 

  5. Cambria E, Hussain A. Sentic album: content-, concept-, and context-based online personal photo management system. Cogn Comput. 2012;4(4):477–96.

    Article  Google Scholar 

  6. Cambria E, Hussain A. Sentic computing: techniques, tools, and applications. In: Springer briefs in Cogn Comput; 2012.

  7. Cao FL, Xie TF, Xu ZB. The estimate for approximation error of neural networks: a constructive approach. Neurocomputing. 2009;71(4):626–30.

    Google Scholar 

  8. Chang CC, Lin CJ. A library for support vector machines. LIBSVM: ACM Trans Intell Syst Technol. 2011;2(3):27:1– 27.

    Google Scholar 

  9. Cortes C, Vapnik VN. Support vector networks. Mach Learn. 1995;20:273–97.

    Google Scholar 

  10. Cristianini N, Shawe-Taylor J. An introduction to support vector machines and other kernel based learning method. Cambridge: Cambridge University Press; 2000.

    Book  Google Scholar 

  11. Duda RO, Hart PR, Stork DG. Pattern classification. 2nd ed. London: Wiley; 2001.

    Google Scholar 

  12. Fiacco AV, McCormick GP. Nonlinear programming: sequential unconstrained minimization techniques. London: Wiley; 1968.

    Google Scholar 

  13. Fung G, Mangasarian OL. Finite Newton method for Lagrangian support vector machine classification. Neurocomputing. 2003;55(1–2):39–55.

    Article  Google Scholar 

  14. Gao S, Ye Q, Ye N. 1-norm least squares twin support vector machines. Neurocomputing. 2011;74:3590–7.

    Article  Google Scholar 

  15. Grassi M, Cambria E, Hussain A, Piazza F. Sentic web: a new paradigm for managing social media affective information. Cogn Comput. 2011;3(3):480–9.

    Article  Google Scholar 

  16. Hiriart-Urruty J-B, Strodiot JJ, Nguyen VH. Generalized hessian matrix and second order optimality conditions for problems with CL1 data. Appl Math Optim. 1984;11:4356.

    Article  Google Scholar 

  17. Jayadeva, Khemchandani R, Chandra S. Twin support vector machines for pattern classification. IEEE Trans Pattern Anal Mach Intell. 2007;29(5):905–910.

  18. Joachims T, Ndellec C, Rouveriol C. Text categorization with support vector machines: learning with many relevant features. In: European conference on machine learning. 1998; (10): 137–142.

  19. Joachims T. Making large-scale support vector machine learning practical. In: Advances in Kernel methods: support vector learning. MIT Press, Cambridge; 1999.

  20. Kumar MA, Gopal M. Application of smoothing technique on twin support vector machines. Pattern Recognit Lett. 2008;29:1842–8.

    Article  Google Scholar 

  21. Kumar MA, Gopal M. Least squares twin support vector machines for pattern classification. Expert Syst Appl. 2009;36:7535–43.

    Article  Google Scholar 

  22. Kumar MA, Khemchandani R, Gopal M, Chandra S. Knowledge based least squares twin support vector machines. Inf Sci. 2010;180(23):4606–18.

    Article  Google Scholar 

  23. Mangasarian OL. A finite Newton method for classification. Optim Methods Softw. 2002;17:913–29.

    Article  Google Scholar 

  24. Mangasarian OL. Exact 1-norm support vector machines via unconstrained convex differentiable minimization. J Mach Learn Res. 2006;7:1517–30.

    Google Scholar 

  25. Mangasarian OL, Wild EW. Multisurface proximal support vector classification via generalized eigenvalues. IEEE Trans Pattern Anal Mach Intell. 2006;28(1):69–74.

    Article  PubMed  Google Scholar 

  26. Murphy PM, Aha DW. UCI repository of machine learning databases. University of California Irvine. http://www.ics.uci.edu/mlearn; 1992.

  27. Musicant DR. NDC: Normally distributed clustered datasets. http://www.cs.wisc.edu/musicant/data/dc; 1998.

  28. Osuna E, Freund R, Girosi F. Training support vector machines: an application to face detection. In: Proceedings of computer vision and pattern recognition. 1997; p. 130–136.

  29. Peng X. TSVR: an efficient twin support vector machine for regression. Neural Netw. 2010;23(3):365–72.

    Article  PubMed  Google Scholar 

  30. Peng X. TPMSVM: a novel twin parametric-margin support vector machine for pattern recognition. Pattern Recognit. 2011;44:2678–92.

    Article  Google Scholar 

  31. Peng X. Building sparse twin support vector machine classifiers in primal space. Inf Sci. 2011;181:3967–80.

    Article  Google Scholar 

  32. Platt J. Fast training of support vector machines using sequential minimal optimization. In: Scholkopf B, Burges CJC, Smola AJ, editors. Advances in Kernel methods-support vector learning. Cambridge: MIT Press; 1999. p. 185–208.

    Google Scholar 

  33. Qi Z, Tian Y, Shi Y. A nonparallel support vector machine for a classification problem with universum learning. J Comput Appl Math. 2014;263:288–98.

    Article  Google Scholar 

  34. Shao YH, Chen WJ, Deng NY. Nonparallel hyperplane support vector machine for binary classification problems. Inf Sci. 2014;263:22–35.

    Article  Google Scholar 

  35. Shao YH, Zhang CH, Wang XB, Deng NY. Improvements on twin support vector machines. IEEE Trans Neural Netw. 2011;22(6):962–8.

    Article  PubMed  Google Scholar 

  36. Shao YH, Zhang CH, Yang ZM, Jing L, Deng NY. An \(\epsilon \)-twin support vector machine for regression. Neural Comput Appl. 2013;23(1):175–85.

    Article  Google Scholar 

  37. Tanveer M. Smooth linear programming twin support vector machines. Int J Mach Learn Comput. 2013;3(2):240–4.

    Article  Google Scholar 

  38. Tian Y, Ping Y. Large-scale linear nonparallel support vector machine solver. Neural Netw. 2014;50:166–74.

    Article  PubMed  Google Scholar 

  39. Tikhonov AN, Arsen VY. Solutions of ill-posed problems. New York: Wiley; 1977.

    Google Scholar 

  40. Vapnik VN. Statistical learning theory. New York: Wiley; 1998.

    Google Scholar 

  41. Vapnik VN. The nature of statistical learning theory. 2nd ed. New York: Springer; 2000.

    Book  Google Scholar 

  42. Wang QF, Cambria E, Liu CL, Hussain A. Common sense knowledge for handwritten Chinese recognition. Cogn Comput. 2013;5(2):234–42.

    Article  Google Scholar 

  43. Wu JD, Liu CT. Finger-vein pattern identification using svm and neural network technique. Expert Syst Appl. 2011;38(11):14284–9.

    Google Scholar 

  44. Xu Y, Guo R, Wang L. A twin multi-class classification support vector machine. Cogn Comput. 2013;5(4):580–8.

    Article  Google Scholar 

  45. Xu Y, Wang L. A weighted twin support vector regression. Knowl-Based Syst. 2012;33:92–101.

    Article  CAS  Google Scholar 

  46. Xu Y, Wang L, Zhong P. A rough margin-based \(\nu \)-twin support vector machine. Neural Comput Appl. 2012;21(6):1307–17.

    Article  Google Scholar 

  47. Zhong P, Xu Y, Zhao Y. Training twin support vector regression via linear programming. Neural Comput Appl. 2012;21(2):399–407.

    Article  Google Scholar 

Download references

Acknowledgments

The author acknowledges the valuable comments of the anonymous reviewers and the Editor of Cognitive Computation whose enthusiasm is gladly appreciated. Also, the author would like to express his sincere gratitude to Professor S. Balasundaram for his help during the preparation of this manuscript.

Author information

Authors and Affiliations

  1. Department of Computer Science and Engineering, The LNM Institute of Information Technology, Jaipur, 302 031, India

    M. Tanveer

Authors
  1. M. Tanveer
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to M. Tanveer.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Tanveer, M. Robust and Sparse Linear Programming Twin Support Vector Machines. Cogn Comput 7, 137–149 (2015). https://doi.org/10.1007/s12559-014-9278-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12559-014-9278-8

Keywords

Search

Navigation