Skip to main content
SpringerLink
Log in

A Review and New Contribution on Conic Multivariate Adaptive Regression Splines (CMARS): A Powerful Tool for Predictive Data Mining

  • Conference paper
  • First Online:
Modeling, Dynamics, Optimization and Bioeconomics I

Part of the book series: Springer Proceedings in Mathematics & Statistics ((PROMS,volume 73))

Abstract

This study aims at critically reviewing the research that has been conducted to improve the backward part of the Multivariate Adaptive Regression Splines (MARS) method that leads to the development of Conic MARS (CMARS) method. Several studies have been carried out to apply the method to various fields of study including life, finance, industry, business, energy, and environment. Moreover, the performance of CMARS method is empirically compared to that of Classification and Regression Trees, Generalized Additive Models and Infinite Kernal Learning for classification, and to that of Multiple Linear Regression and MARS for prediction. For this purpose, real-life and simulation data sets with different characteristics such as size, the number and type of predictors, application areas are used. Comparative studies indicate that the CMARS method is a very promising tool for establishing computational models between variables of complex data sets. Yet, in other studies, CMARS have been improved and extended further. The improvement leads to the development of Bootstrapping CMARS method to build less complex CMARS models while the extension robustifies CMARS to enable modeling random input and output variables more precisely. To sum up, this review indicates that currently the CMARS method is a powerful alternative to the MARS algorithm as well as the other predictive data mining tools, and thus, deserves more attention to evaluate and develop it even further.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
USD 169.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Aster, R.-C., Borchers, B., Thurber, C.: Parameter Estimation and Inverse Problems. Academic Press, Burlington (2004)

    Google Scholar 

  2. Bakır, B.: Defect Cause Modelling with Decision Tree and Regression Analysis: A Case Study in Casting Industry. MSc., Middle East Technical University (2006)

    Google Scholar 

  3. Batmaz, İ., Köksal, G.: Overview of knowledge discovery in databases process and data mining for surveillance technologies and EWS. In: Koyuncugil, A.S., Özgülba \(\vartheta\), N. (eds.) Surveillance Technologies and Early Warning Systems: Data Mining Applications for Risk Detection. IGI Global Publisher (Idea Group Publisher), Hershey, PA (2011)

    Google Scholar 

  4. Batmaz, İ., Yazıcı, C., Yerlikaya-Özkurt, F.: Bootstrapping conic multivariate adaptive regression splines. Technical Report, 4, (2012)

    Google Scholar 

  5. Batmaz, İ., Yerlikaya-Özkurt, F., Kartal-Koç, E., Köksal, G., Weber, G.-W.: Evaluating the CMARS performance for modeling nonlinearities. In: Proceedings of the 3rd Global Conference on Power Control and Optimization, Gold Coast (Australia), 1239, pp. 351–357 (2010)

    Google Scholar 

  6. Breiman, L., Friedman, J.-H., Olshen, R., Stone, C.: Classification and regression trees. Wadswort International Group, Belmont, California (1984)

    MATH  Google Scholar 

  7. Craven, P., Wahba, G.: Smoothing noisy data with spline functions: estimating the correct degree of smoothing by the method of generalized cross-validation. Numerische Mathematik 31, 377–403 (1979)

    Article  MATH  MathSciNet  Google Scholar 

  8. Çelik, G.: Parameter Estimation in Generalized Partial Linear Models with Conic Quadratic Programming. MSc., Middle East Technical University (2010)

    Google Scholar 

  9. Davis, C.-S.: Statistical Methods for the Analysis of Repeated Measures. Springer, New York (2003)

    Google Scholar 

  10. Efron, B., Tibshirani, R.-J.: An Introduction to the Bootstrap. Chapman & Hall, New York (1993)

    Google Scholar 

  11. Friedman, J.-H.: Multivariate adaptive regression splines. Ann. Stat. 19, 1–141 (1991)

    Article  MATH  Google Scholar 

  12. Hastie, T., Tibshirani, R.: Generalized additive models: some applications. J. Am. Stat. Assoc. 82, 371–386 (1987)

    Article  MATH  MathSciNet  Google Scholar 

  13. Hastie, T., Tibshirani, R., Friedman, J.-H.: The Elements of Statistical Learning, Data Mining, Inference and Prediction. Springer, New York (2001)

    Book  MATH  Google Scholar 

  14. Hock, W., Schittkowski, K.: Test Examples for Nonlinear Programming Codes. Springer, New York (1981)

    Book  MATH  Google Scholar 

  15. Kartal, E.: Metamodelling Complex Systems Using Liner and Nonlinear Regression Methods. MSc., Middle East Technical University (2007)

    Google Scholar 

  16. Kartal-Koç, E., İyigün, C.: Restructuring forward stepwise algorithm of multivariate adaptive regression splines (MARS) by using self organizing maps (SOM). In: INFORMS 2011 Annual Meeting. Charlotte, North Carolina, USA, November 13–16 (2011)

    Google Scholar 

  17. Köksal, G., Batmaz, İ., Testik, M.-C.: A review of data mining applications for quality improvement in manufacturing industry. Expert Syst. Appl. (2011). Doi.10.1016/j.eswa.2011.04.063

    Google Scholar 

  18. MARS from Salford Systems. Available at http://www.salfordsystems.com/mars/phb

  19. Martinez, W.-L., Martinez, A.-R.: Computational Statistics Handbook with MATLAB. Chapman and Hall, CRC, London (2002)

    Google Scholar 

  20. MATLAB Version 7.5 (R2007b)

    Google Scholar 

  21. Milborrow, S.: Earth: Multivariate Adaptive Regression Spline Models, R Software Package (2009). Available at http://cran.r-project.org/web/packages/earth/index.html

    Google Scholar 

  22. MINITAB 15 Statistical Software. Available at www.minitab.com

  23. MOSEK, A very powerful commercial software for CQP. Available at http://www.mosek.com

  24. Nemirovski, A.: A Lectures on Modern Convex Optimisation, Israel Institute of Technology (2002). Available at http://iew3.technion.ac.il/Labs/Opt/opt/LN/Final.pdf

    Google Scholar 

  25. Nesterov, Y.-E., Nemirovski, A.-S.: Interior Point Polynomial Algorithms in Convex Programming. SIAM, Philadelphia (1994)

    Book  MATH  Google Scholar 

  26. Neter, J., Kutner, M., Wasserman, W., Nachtsheim, C.: Applied Liear Statistical Models. WCB/McGrawHill, Boston (1996)

    Google Scholar 

  27. Osei-Bryson, K.-M.: Evaluation of decision trees: A multi-criteria approach. Comput. Oper. Res. 31, 1933–1945 (2004)

    Article  MATH  Google Scholar 

  28. Özmen, A.: Robust Conic Quadratic Programming Applied to Quality Improvement - A Robustification of CMARS. MSc., Middle East Technical University (2010)

    Google Scholar 

  29. Özmen, A., Yıldırım, M.-H., Türker-Bayrak, Ö., Weber, G.-W.: Electricity Price Modelling for Turkey, OR 2011 Zurich, August 30 September 2, 2011, Zurich, Switzerland

    Google Scholar 

  30. Özmen, A., Weber, G.-W., Batmaz, İ.: The new robust CMARS (RCMARS) method. In: 24th Mini EURO Conference - On Continuous Optimization and Information-Based Technologies in the Financial Sector, MEC EurOPT Selected Papers, ISI Proceedings, pp. 362–368 (2010 )

    Google Scholar 

  31. Özmen, A., Weber, G.-W., Batmaz, İ., Kropat, E.: RCMARS: Robustification of CMARS with different scenarios under polyhedral uncertainty set. Communications in Nonlinear Science and Numerical Simulation (CNSNS): Nonlinear, Fractional and Complex. DOI: 10.1016/j.cnsns.2011.04.001

    Google Scholar 

  32. Özöğür-Akyüz, S., Weber, G.-W.: Infinite kernel learning via infinite and semi-infinite programming. Optim. Method Software 6, 937–970 (2010)

    Article  Google Scholar 

  33. Sezgin-Alp, Ö., Büyükbebeci, E., Işcanoğlu-Çekiç, A., Yerlikaya-Özkurt, F., Taylan, P., Weber, G.-W.: -CMARS and GAM and CQP- modern optimization methods applied to international credit default prediction. J. Comput. Appl. Math. 235, 4639–4651 (2011)

    Article  MathSciNet  Google Scholar 

  34. StatLib: Datasets Archive. Available at http://lib.stat.cmu.edu/datasets/

  35. Taylan, P., Weber, G.-W., Beck, A.: New approaches to regression by generalized additive models and continuoues optimisation for modern applications in finance, science and technology. J. Optim. 56, 675–698 (2007)

    Article  MATH  MathSciNet  Google Scholar 

  36. Taylan, P., Weber, G.-W., Yerlikaya, F.: Continuous optimisation applied in MARS for modern applications in finance, science and technology. In: ISI Proceedings of 20th Mini-EURO Conference Continuous Optimisation and Knowledge-Based Technologies, pp. 317–322 (2008)

    Google Scholar 

  37. Taylan, P., Weber, G.-W., Yerlikaya-Özkurt, F.: A new approach to multivariate adaptive regression spline by using Tikhonov regularizati on and continuous optimization. TOP (Oper. Res. J. SEIO (Spanish Stat. Oper. Res. Soc.) 18, 377–395 (2010)

    Google Scholar 

  38. UCI: Machine Learning Repository. Available at http://archive.ics.uci.edu/ml/

  39. Weber, G.-W., Batmaz, İ., Köksal, G., Taylan, P., Yerlikaya-Özkurt, F.: CMARS: A new contribution to nonparametric regression with multivariate adaptive regression splines supported by continuous optimisation. Inverse Probl. Sci. Eng. 20(3), 371–400 (2011)

    Article  Google Scholar 

  40. Weber, G.-W., Özmen, A., Batmaz, İ.: RCMARS: The new robust CMARS method. In: VI Moscow International Conference on Operations Research (ORM 2010), Moscow, Russia, October 20–25 (2010)

    Google Scholar 

  41. Yazıcı, C.: A Computational Approach to Nonparametric Regression: Bootstrapping the CMARS Method. MSc., Middle East Technical University (2011)

    Google Scholar 

  42. Yazıcı, C., Yerlikaya-Özkurt, F., Batmaz, İ.: A computational approach to nonparametric regression: bootstrapping the CMARS method. In: CFE-ERCIM 2011: 4th International Conference of the ERCIM (European Research Consortium for Informatics and Mathematics) Working Group on Computing and Statistics. London, UK. December 17–19 (2011). Book of Abstracts, 115

    Google Scholar 

  43. Yerlikaya, F.: A New Contribution to Nonlinear Robust Regression and Classification with MARS and Its Application to Data Mining for Quality Control in Manufacturing. MSc., Middle East Technical University (2008)

    Google Scholar 

  44. Yerlikaya Özkurt, F., Taylan, P., Batmaz, İ., Köksal, G., Weber, G.-W.: A modification of MARS by Tikhonov regularization and conic quadratic programming for modeling quality data. In: EURO XXIII 2009, Bonn, Germany, July 5–8 (2009)

    Google Scholar 

Download references

Acknowledgements

One of the authors, Fatma Yerlikaya-Özkurt, has been supported by the TUBITAK Domestic Doctoral Scholarship Program.

Author information

Authors and Affiliations

  1. Institute of Applied Mathematics, Middle East Technical University, 06800, Ankara, Turkey

    Fatma Yerlikaya-Özkurt & Gerhard-Wilhelm Weber

  2. Department of Statistics, Middle East Technical University, 06800, Ankara, Turkey

    İnci Batmaz

Authors
  1. Fatma Yerlikaya-Özkurt
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. İnci Batmaz
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Gerhard-Wilhelm Weber
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Fatma Yerlikaya-Özkurt .

Editor information

Editors and Affiliations

  1. University of Porto, Department of Mathematics, Porto, Portugal

    Alberto Adrego Pinto

  2. Univ. of California, Dept. of Agricultural and Resource Economics, Berkeley, California, USA

    David Zilberman

Rights and permissions

Reprints and permissions

Copyright information

© 2014 Springer International Publishing Switzerland

About this paper

Cite this paper

Yerlikaya-Özkurt, F., Batmaz, İ., Weber, GW. (2014). A Review and New Contribution on Conic Multivariate Adaptive Regression Splines (CMARS): A Powerful Tool for Predictive Data Mining. In: Pinto, A., Zilberman, D. (eds) Modeling, Dynamics, Optimization and Bioeconomics I. Springer Proceedings in Mathematics & Statistics, vol 73. Springer, Cham. https://doi.org/10.1007/978-3-319-04849-9_40

Download citation

Publish with us

Policies and ethics

Search

Navigation