Abstract
Training neural networks is a complex task of great importance in the supervised learning field of research. In this work we tackle this problem with five algorithms, and try to offer a set of results that could hopefully foster future comparisons by following a kind of standard evaluation of the results (the Prechelt approach). To achieve our goal of studying in the same paper population based, local search, and hybrid algorithms, we have selected two gradient descent algorithms: Backpropagation and Levenberg-Marquardt, one population based heuristic such as a Genetic Algorithm, and two hybrid algorithms combining this last with the former local search ones. Our benchmark is composed of problems arising in Medicine, and our conclusions clearly establish the advantages of the proposed hybrids over the pure algorithms.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Alander, J.T.: Indexed Bibliography of Genetic Algorithms and Neural Networks. Technical Report 94-1-NN, University of Vaasa, Department of Information Technology and Production Economics (1994)
Rumelhart, D., Hinton, G., Williams, R.: Learning Representations by Backpropagation Errors. Nature 323, 533–536 (1986)
Cotta, C., Alba, E., Sagarna, R., Larrañaga, P.: Adjusting Weights in Artificial Neural Networks using Evolutionary Algorithms. In: Larrañaga, P., Lozano, J. (eds.) Estimation of Distribution Algorithms. A New Tool for Evolutionary Computation, pp. 357–373. Kluwer Academic Publishers, Dordrecht (2001)
Cantú-Paz, E.: Pruning Neural Networks with Distribution Estimation Algorithms. In: Cantú-Paz, E., Foster, J.A., Deb, K., Davis, L., Roy, R., O’Reilly, U.-M., Beyer, H.-G., Kendall, G., Wilson, S.W., Harman, M., Wegener, J., Dasgupta, D., Potter, M.A., Schultz, A., Dowsland, K.A., Jonoska, N., Miller, J., Standish, R.K. (eds.) GECCO 2003. LNCS, vol. 2723, pp. 790–800. Springer, Heidelberg (2003)
Alba, E., Aldana, J.F., Troya, J.M.: Full Automatic ANN Design: A Genetic Approach. In: Mira, J., Cabestany, J., Prieto, A. (eds.) New Trends in Neural Computation, pp. 399–404. Springer, Heidelberg (1993)
Yao, X., Liu, Y.: A New Evolutionary System for Evolving Artificial Neural Networks. IEEE Transactions on Neural Networks 8, 694–713 (1997)
Yao, X.: Evolving Artificial Neural Networks. Proceedings of the IEEE 87, 1423–1447 (1999)
Prechelt, L.: Proben1 — A Set of Neural Network Benchmark Problems and Benchmarking Rules. Technical Report 21, Fakultät für Informatik Universität Karlsruhe, 76128 Karlsruhe, Germany (1994)
Hagan, M.T., Menhaj, M.B.: Training Feedforward Networks with the Marquardt Algorithm. IEEE Transactions on Neural Networks 5 (1994)
McClelland, J.L., Rumelhart, D.E.: Parallel Distributed Processing: Explorations in the Microstructure of Cognition. MIT Press, Cambridge (1986)
Rosenblatt, F.: Principles of Neurodynamics. Spartan Books, New York (1962)
Holland, J.H.: Adaptation in Natural and Artificial Systems. The University of Michigan Press, Ann Arbor (1975)
Davis, L. (ed.): Handbook of Genetic Algorithms. Van Nostrand Reinhold, New York (1991)
Cotta, C., Troya, J.M.: On Decision-Making in Strong Hybrid Evolutionary Algorithms. In: Moonis, A., Mira, J., de Pobil, A.P. (eds.) IEA/AIE 1998. LNCS, vol. 1415, pp. 418–427. Springer, Heidelberg (1998)
Bennett, K.P., Mangasarian, O.L.: Robust Linear Programming Discrimination of Two Linearly Inseparable Sets. Optimization Methods and Software 1, 23–34 (1992)
Mangasarian, O.L., Setiono, R., Wolberg, W.H.: Pattern Recognition via Linear Programming: Theory and Application to Medical Diagnosis. In: Coleman, T.F., Li, Y. (eds.) Large-Scale Numerical Optimization, pp. 22–31. SIAM Publications, Philadelphia (1990)
Wolberg, W.H.: Cancer Diagnosis via Linear Programming. SIAM News 23, 1–18 (1990)
Wolberg, W.H., Mangasarian, O.L.: Multisurface Method of Pattern Separation for Medical Diagnosis Applied to Breast Cytology. Proceedings of the National Academy of Sciences 87, 9193–9196 (1990)
Smith, J.W., Everhart, J.E., Dickson, W.C., Knowler, W.C., Johannes, R.S.: Using the ADAP Learning Algorithm to Forecast the Onset of Diabetes Mellitus. In: Proceedings of the Twelfth Symposium on Computer Applications in Medical Care, pp. 261–265. IEEE Computer Society Press, Los Alamitos (1988)
Detrano, R., Janosi, A., Steinbrunn, W., Pfisterer, M., Schmid, J., Sandhu, S., Guppy, K., Lee, S., Froelicher, V.: International Application of a New Probability Algorithm for the Diagnosis of Coronary Artery Disease. American Journal of Cardiology, 304–310 (1989)
Gennari, J.H., Langley, P., Fisher, D.: Models of Incremental Concept Formation. Artificial Intelligence 40, 11–61 (1989)
Ragg, T., Gutjahr, S., Sa, H.: Automatic Determination of Optimal Network Topologies Based on Information Theory and Evolution. In: Proceedings of the 23rd EUROMICRO Conference, Budapest, Hungary (1997)
Land, W.H., Albertelli, L.E.: Breast Cancer Screening Using Evolved Neural Networks. In: IEEE International Conference on Systems, Man, and Cybernetics, vol. 2, pp. 1619–1624. IEEE Computer Society Press, Los Alamitos (1998)
Erhard, W., Fink, T., Gutzmann, M.M., Rahn, C., Doering, A., Galicki, M.: The Improvement and Comparison of Different Algorithms for Optimizing Neural Networks on the MasPar MP-2. In: Heiss, M. (ed.) Neural Computation – NC 1998, pp. 617–623. ICSC Academic Press, London (1998)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2004 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Alba, E., Chicano, J.F. (2004). Training Neural Networks with GA Hybrid Algorithms. In: Deb, K. (eds) Genetic and Evolutionary Computation – GECCO 2004. GECCO 2004. Lecture Notes in Computer Science, vol 3102. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-24854-5_87
Download citation
DOI: https://doi.org/10.1007/978-3-540-24854-5_87
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-22344-3
Online ISBN: 978-3-540-24854-5
eBook Packages: Springer Book Archive