Skip to main content

Advertisement

SpringerLink
Log in

An enhanced XCS rule discovery module using feature ranking

  • Original Article
  • Published:
International Journal of Machine Learning and Cybernetics Aims and scope Submit manuscript

Abstract

XCS is a genetics-based machine learning model that combines reinforcement learning with evolutionary algorithms to evolve a population of classifiers in the form of condition-action rules. Like many other machine learning algorithms, XCS is less effective on high-dimensional data sets. In this paper, we describe a new guided rule discovery mechanisms for XCS, inspired by feature selection techniques commonly used in machine learning. In our approach, feature quality information is used to bias the evolutionary operators. A comprehensive set of experiments is used to investigate how the number of features used to bias the evolutionary operators, population size, and feature ranking technique, affect model performance. Numerical simulations have shown that our guided rule discovery mechanism improves the performance of XCS in terms of accuracy, execution time and more generally in terms of classifier diversity in the population, especially for high-dimensional classification problems. We present a detailed discussion of the effects of model parameters and recommend settings for large scale problems.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Notes

  1. GRD-XCS was introduced in [5]. This paper is a revised and a substantially extended version of that paper.

  2. Victorian Partnership for Advanced Computing: http://www.vpac.org.

References

  1. The XCS source code in C is freely available on Illinois Genetic Algorithms Laboratory (IlliGAL) web site: http://illigal.org/category/source-code.

  2. UCI Machine Learning Repository: The Center for Machine Learning and Intelligent Systems at the University of California, Irvine. http://archive.ics.uci.edu/ml

  3. Weka 3, is an open source data mining tool (in java), with a collection of machine learning algorithms developed by Machine Learning Group at University of Waikato. http://www.cs.waikato.ac.nz/ml/weka

  4. Abedini M, Kirley M (2010) A multiple population XCS: evolving condition-action rules based on feature space partitions. In: 2010 IEEE Congress on Evolutionary computation (CEC), pp 1–8, July 2010

  5. Abedini M, Kirley M (2011) Guided rule discovery in XCS for high-dimensional classification problems. In: Proceedings of 24th Australasian artificial intelligence conference. Lecture notes in artificial intelligence, vol 7106

  6. Alon U, Barkai N, Notterman DA, Gishdagger K, Ybarradagger S, Mackdagger D, Levine AJ (1999) Broad patterns of gene expression revealed by clustering analysis of tumor and normal colon tissues probed by oligonucleotide arrays. Proc Natl Acad Sci USA 96:6745–6750

    Article  Google Scholar 

  7. Bacardit J, Krasnogor N (2006) Smart crossover operator with multiple parents for a Pittsburgh learning classifier system. In: Proceedings of the 8th conference on GECCO. ACM, New York, pp 1441–1448

  8. Bacardit J, Stout M, Hirst JD, Sastry K, Llorà à X, Krasnogor N (2007) Automated alphabet reduction method with evolutionary algorithms for protein structure prediction. In: Thierens D, Beyer H-G, Bongard J, Branke J, Clark JA, Cliff D, Congdon CB, Deb K, Doerr B, Kovacs T, Kumar S, Miller JF, Moore J, Neumann F, Pelikan M, Poli R, Sastry K, Stanley KO, Stutzle T, Watson RA, Wegener I (eds) GECCO ’07: Proceedings of the 9th annual conference on Genetic and evolutionary computation, vol 1, London, 7–11 July 2007. ACM Press, New York, pp 346–353

  9. Bacardit J, Stout M, Krasnogor N, Hirst JD, Blazewicz J (2006) Coordination number prediction using learning classifier systems: performance and interpretability. In: Cattolico M (ed) Genetic and evolutionary computation conference, GECCO 2006, proceedings, Seattle, Washington, USA, July 8–12, 2006. ACM, New York, pp 247–254

  10. Bonilla Huerta E, Hernandez Hernandez J, Hernandez Montiel L (2010) A new combined filter-wrapper framework for gene subset selection with specialized genetic operators. In: Advances in pattern recognition. Lecture notes in computer science, vol 6256. Springer, Berlin/Heidelberg, pp 250–259

  11. Butz MV, Goldberg DE, Tharakunnel K (2003) Analysis and improvement of fitness exploitation in XCS: bounding models, tournament selection, and bilateral accuracy. Evol Comput 11:239–277

    Article  Google Scholar 

  12. Butz MV, Pelikan M, Llorà à X, Goldberg DE (2006) Automated global structure extraction for effective local building block processing in XCS. Evol Comput 14:345–380

    Article  Google Scholar 

  13. Butz MV, Wilson SW (2001) An Algorithmic description of XCS. In: Advances in learning classifier systems. Lecture notes in computer science, vol 1996/2001. Springer, Berlin/Heidelberg, pp 267–274

  14. Fernandndez A, Garcianda S, Luengo J, Bernado-Mansilla E, Herrera F (2010) Genetics-based machine learning for rule induction: state of the art, taxonomy, and comparative study. IEEE Trans Evol Comput 14(6):913–941

    Article  Google Scholar 

  15. Golub TR, Slonim DK, Tamayo P, Huard C, Gaasenbeek M, Mesirov JP, Coller H, Loh ML, Downing JR, Caligiuri MA, Bloomfield CD (1999) Molecular classification of cancer: class discovery and class prediction by gene expression monitoring. Science 286:531–537

    Article  Google Scholar 

  16. Guyon I, Weston J, Barnhill S, Vapnik V (2002) Gene selection for cancer classification using support vector machines. Mach Learn 46:389–422

    Article  MATH  Google Scholar 

  17. Hall MA (1998) Correlation-based feature subset selection for machine learning. PhD thesis, University of Waikato, Hamilton, New Zealand

  18. Hedenfalk I, Duggan D, Chen Y, Radmacher M, Bittner M, Simon R, Meltzer P, Gusterson B, Esteller M, Kallioniemi OP, Wilfond B, Borg A, Trent J (2001) Gene-expression profiles in hereditary breast cancer. N Engl J Med 344(8):539–548

    Article  Google Scholar 

  19. Ian EF, Witten H (2005) Data mining: practical machine learning tools and techniques. Morgan Kaufmann series in data management systems, 2 edn. Morgan Kaufmann, Menlo Park

  20. Isabelle Guyon MN, Steve Gunn, Zadeh L (eds) (2006) Feature extraction, foundations and applications. Springer, Berlin

  21. José-Revuelta LMS (2008) A hybrid GA-TS technique with dynamic operators and its application to channel equalization and fiber tracking. In: Jaziri W (ed) Local search techniques: focus on tabu search, I-Tech, Vienna. ISBN 978-3-902613-34-9

  22. Kononenko I (1994) Estimating attributes: analysis and extensions of relief. In: Bergadano F, Raedt LD (eds) Machine learning: Proceedings of the ECML-94, european conference on machine learning, Catania, Italy, April 6–8, 1994. Lecture notes in computer science, vol 784. Springer, Berlin, pp 171–182

  23. Lanzi PL (1997) A study of the generalization capabilities of XCS. In: Bäck T (ed) Proceedings of the 7th international conference on genetic algorithms. Morgan Kaufmann, Menlo Park, pp 418–425

  24. Moore JH, White BC (2006) Exploiting expert knowledge in genetic programming for genome-wide genetic analysis. In: PPSN. Lecture notes in computer science, vol 4193. Springer, Berlin, pp 969–977

  25. Morales-Ortigosa S, Orriols-Puig A, Bernadó-Mansilla E (2008) New crossover operator for evolutionary rule discovery in XCS. In: 8th international conference on hybrid intelligent systems. IEEE Computer Society, pp 867–872

  26. Morales-Ortigosa S, Orriols-Puig A, Bernadó-Mansilla E (2009) Analysis and improvement of the genetic discovery component of XCS. Int Jt Conf Hybrid Intell Syst 6:81–95

    MATH  Google Scholar 

  27. Orriols-Puig A, Casillas J, Bernadó-Mansilla E (2008) Genetic-based machine learning systems are competitive for pattern recognition. Evol Intell 1:209–232. doi:10.1007/s12065-008-0013-9

    Article  Google Scholar 

  28. Pang-Ning Tan MSVK (2006) Introduction to data mining. Addison-Wesley Longman Publishing Co., Inc., Chicago

  29. Platt J (1998) Fast training of support vector machines using sequential minimal optimization. In: Schoelkopf B, Burges C, Smola A (eds) Advances in kernel methods—support vector learning. MIT Press, Cambridge

  30. Singh D, Febbo PG, Ross K, Jackson DG, Manola J, Ladd C, Tamayo P, Renshaw AA (2002) Gene expression correlates of clinical prostate cancer behavior. Cancer Cell 1:203–209

    Article  Google Scholar 

  31. Stalph PO, Butz MV, Goldberg DE, Llorà à X (2009) On the scalability of xcs(f). In: Rothlauf F (ed) GECCO. ACM, New York, pp 1315–1322

  32. Sumathi S, Sivanandam SN (2006) Introduction to data mining and its applications. Studies in computational intelligence, vol 29. Springer, Berlin

  33. Wang P, Weise T, Chiong R (2011) Novel evolutionary algorithms for supervised classification problems: an experimental study. Evol Intell 4(1):3–16

    Article  Google Scholar 

  34. Wilson SW (1995) Classifier fitness based on accuracy. Evol Comput 3(2):149–175. http://prediction-dynamics.com/

    Google Scholar 

  35. Wilson SW (1999) Get real! XCS with continuous-valued inputs. In: Lanzi PL, Stolzmann W, Wilson SW (eds) Learning classifier systems, from foundations to applications. Lecture notes in computer science, vol 1813. MIT Press, Cambridge, pp 209–222

  36. Wu F-X, Zhang W, Kusalik A (2006) On Determination of minimum sample size for discovery of temporal gene expression patterns. In: First international multi-symposiums on computer and computational sciences, pp 96–103

  37. Zhang Y-Q, Rajapakse JC (eds) (2008) Machine learning in bioinformatics. Wiley book series on bioinformatics: computational techniques and engineering. 1st edn. John Wiley & Sons, New Jersey

Download references

Author information

Authors and Affiliations

  1. Department of Computing and Information Systems, The University of Melbourne, Victoria, Australia

    Mani Abedini & Michael Kirley

Authors
  1. Mani Abedini
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Michael Kirley
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mani Abedini.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Abedini, M., Kirley, M. An enhanced XCS rule discovery module using feature ranking. Int. J. Mach. Learn. & Cyber. 4, 173–187 (2013). https://doi.org/10.1007/s13042-012-0085-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13042-012-0085-9

Keywords

Search

Navigation