Skip to main content

Advertisement

SpringerLink
Log in

Training Feed-Forward Multi-Layer Perceptron Artificial Neural Networks with a Tree-Seed Algorithm

  • Research Article-Computer Engineering and Computer Science
  • Published:
Arabian Journal for Science and Engineering Aims and scope Submit manuscript

Abstract

The artificial neural network (ANN) is the most popular research area in neural computing. A multi-layer perceptron (MLP) is an ANN that has hidden layers. Feed-forward (FF) ANN is used for classification and regression commonly. Training of FF MLP ANN is performed by backpropagation (BP) algorithm generally. The main disadvantage of BP is trapping into local minima. Nature-inspired optimizers have some mechanisms escaping from the local minima. Tree-seed algorithm (TSA) is an effective population-based swarm intelligence algorithm. TSA mimics the relationship between trees and their seeds. The exploration and exploitation are controlled by search tendency which is a peculiar parameter of TSA. In this work, we train FF MLP ANN for the first time. TSA is compared with particle swarm optimization, gray wolf optimizer, genetic algorithm, ant colony optimization, evolution strategy, population-based incremental learning, artificial bee colony, and biogeography-based optimization. The experimental results show that TSA is the best in terms of mean classification rates and outperformed the opponents on 18 problems.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  1. Bassett, D.S.; Gazzaniga, M.S.: Understanding complexity in the human brain. Trends Cognit. Sci. 15(5), 200–209 (2011)

    Google Scholar 

  2. Haykin, S.: Neural Networks: A Comprehensive Foundation. Prentice Hall PTR, New York (1994)

    MATH  Google Scholar 

  3. Yao, L.; Li, T.; Li, Y.; Long, W.; Yi, J.: An improved feed-forward neural network based on UKF and strong tracking filtering to establish energy consumption model for aluminum electrolysis process. Neural Comput. Appl. 31(8), 4271–4285 (2019)

    Google Scholar 

  4. Zhang, Y.; Gendeel, M.A.A.; Peng, H.; Qian, X.: Xu H Supervised Kohonen network with heterogeneous value difference metric for both numeric and categorical inputs. Soft Comput.ss 24(3), 1763–1774 (2020)

    Google Scholar 

  5. Mirjalili, S.: Evolutionary radial basis function networks. In: Evolutionary Algorithms and Neural Networks: Theory and Applications. Springer International Publishing, Cham, pp 105–139 (2019). https://doi.org/10.1007/978-3-319-93025-1-8

  6. Shojaeifard, A.; Amroudi, A.N.; Mansoori, A.; Erfanian, M.: Projection recurrent neural network model: a new strategy to solve weapon-target assignment problem. Neural Process. Lett. 30(8), 2538–2547 (2019)

    Google Scholar 

  7. Tavanaei, A.; Ghodrati, M.; Kheradpisheh, S.R.; Masquelier, T.; Maida, A.: Deep learning in spiking neural networks. Neural Netw. 111, 47–63 (2019)

    Google Scholar 

  8. Mirjalili, S.: How effective is the Grey Wolf optimizer in training multi-layer perceptrons. Appl. Intell. 43(1), 150–161 (2015)

    Google Scholar 

  9. Lee, S.-J.; Tseng, C.-H.; Lin, G.R.; Yang, Y.; Yang, P.; Muhammad, K.; Pandey, H.M.: A dimension-reduction based multilayer perception method for supporting the medical decision making. Pattern Recogn. Lett. 131, 15–22 (2020)

    Google Scholar 

  10. Hertz, J.A.: Introduction to the Theory of Neural Computation. CRC Press, Amsterdam (2018)

    Google Scholar 

  11. Mitchell, M.; Holland, J.H.; Forrest, S.: When will a genetic algorithm outperform hill climbing. In: Advances in Neural Information Processing Systems, pp. 51–58 (1994)

  12. Sonuc, E.; Sen, B.; Bayir, S.: A cooperative GPU-based parallel multistart simulated annealing algorithm for quadratic assignment problem. Eng. Sci. Technol. Int. J. 21(5), 843–849 (2018). https://doi.org/10.1016/j.jestch.2018.08.002

    Article  Google Scholar 

  13. Pandey, H.M.; Rajput, M.; Mishra, V.: Performance comparison of pattern search, simulated annealing, genetic algorithm and jaya algorithm. In: Data Engineering and Intelligent Computing. Springer, Berlin, pp 377–384 (2018)

  14. Şahman, M.A.; Altun, A.A.; Dündar, A.O.: A new MILP model proposal in feed formulation and using a hybrid-linear binary PSO (H-LBP) approach for alternative solutions. Neural Comput. Appl. 29(2), 537–552 (2018)

    Google Scholar 

  15. Cinar, A.C.; Korkmaz, S.; Kiran, M.S.: A discrete tree-seed algorithm for solving symmetric traveling salesman problem. Eng. Sci. Technol. Int. J. (2019)

  16. Tongur, V.; Hacibeyoglu, M.; Ulker, E.: Solving a big-scaled hospital facility layout problem with meta-heuristics algorithms. Eng. Sci. Technol. Int. J. (2019)

  17. Xu, X.; Rong, H.; Trovati, M.; Liptrott, M.; Bessis, N.: CS-PSO: chaotic particle swarm optimization algorithm for solving combinatorial optimization problems. Soft. Comput. 22(3), 783–795 (2018)

    Google Scholar 

  18. Egrioglu, E.; Yolcu, U.; Bas, E.; Dalar, A.Z.: Median-Pi artificial neural network for forecasting. Neural Comput. Appl. 31(1), 307–316 (2019)

    Google Scholar 

  19. Yasar, A.; Saritas, I.; Sahman, M.A.; Dundar, A.O.: Classification of leaf type using artificial neural networks. Int. J. Intell. Syst. Appl. Eng. 3(4), 136–139 (2015)

    Google Scholar 

  20. Yasar, A.; Saritas, I.; Sahman, M.; Cinar, A.: Classification of parkinson disease data with artificial neural networks. In: IOP Conference Series: Materials Science and Engineering, vol 1. IOP Publishing, p. 012031 (2019

  21. Sulistyo, S.B.; Woo, W.L.; Dlay, S.S.: Regularized neural networks fusion and genetic algorithm based on-field nitrogen status estimation of wheat plants. IEEE Trans. Industr. Inf. 13(1), 103–114 (2016)

    Google Scholar 

  22. Sulistyo, S.B.; Woo, W.L.; Dlay, S.S.; Gao, B.: Building a globally optimized computational intelligent image processing algorithm for on-site inference of nitrogen in plants. IEEE Intell. Syst. 33(3), 15–26 (2018)

    Google Scholar 

  23. Gu, K.; Zhou, Y.; Sun, H.; Zhao, L.; Liu, S.: Prediction of air quality in Shenzhen based on neural network algorithm. Neural Comput. Appl. 1–14 (2019)

  24. Koh, B.H.D.; Woo, W.L.: Multi-view temporal ensemble for classification of non-stationary signals. IEEE Access 7, 32482–32491 (2019)

    Google Scholar 

  25. Boashash, B.; Ouelha, S.: Designing high-resolution time–frequency and time–scale distributions for the analysis and classification of non-stationary signals: a tutorial review with a comparison of features performance. Digital Signal Process. 77, 120–152 (2018)

    Google Scholar 

  26. Delsy, T.T.M.; Nandhitha, N.; Rani, B.S.: Feasibility of spectral domain techniques for the classification of non-stationary signals. J. Ambient Intell. Hum. Comput, 1–8 (2020)

  27. Wolpert, D.H.; Macready, W.G.: No free lunch theorems for optimization. IEEE Trans. Evol. Comput. 1(1), 67–82 (1997)

    Google Scholar 

  28. Wienholt, W.: Minimizing the system error in feedforward neural networks with evolution strategy. In: International Conference on Artificial Neural Networks, Springer, Berlin, pp 490–493 (1993)

  29. Seiffert, U.: Multiple layer perceptron training using genetic algorithms. In: ESANN, Citeseer, pp 159–164 (2001)

  30. Mendes, R.; Cortez, P.; Rocha, M.; Neves, J.: Particle swarms for feedforward neural network training. In: Proceedings of the 2002 International Joint Conference on Neural Networks. IJCNN'02 (Cat. No. 02CH37290), IEEE, pp 1895–1899 (2002)

  31. Blum, C.; Socha, K.: Training feed-forward neural networks with ant colony optimization: an application to pattern classification. In: Fifth International Conference on Hybrid Intelligent Systems (HIS'05), IEEE (2005)

  32. Karaboga, D.; Akay, B.; Ozturk, C.: Artificial bee colony (ABC) optimization algorithm for training feed-forward neural networks. In: International conference on modeling decisions for artificial intelligence, Springer, Berlin, pp. 318–329 (2007)

  33. Mirjalili, S.; Hashim, S.Z.M.; Sardroudi, H.M.: Training feedforward neural networks using hybrid particle swarm optimization and gravitational search algorithm. Appl. Math. Comput. 218(22), 11125–11137 (2012)

    MathSciNet  MATH  Google Scholar 

  34. Mirjalili, S.; Mirjalili, S.M.; Lewis, A.: Let a biogeography-based optimizer train your multi-layer perceptron. Inf. Sci. 269, 188–209 (2014)

    MathSciNet  Google Scholar 

  35. Amirsadri, S.; Mousavirad, S.J.; Ebrahimpour-Komleh, H.: A Levy flight-based grey wolf optimizer combined with back-propagation algorithm for neural network training. Neural Comput. Appl. 30(12), 3707–3720 (2018)

    Google Scholar 

  36. Haklı, H.; Uğuz, H.: A novel particle swarm optimization algorithm with Levy flight. Appl. Soft Comput. 23, 333–345 (2014)

    Google Scholar 

  37. Xu, F.; Pun, C.-M.; Li, H.; Zhang, Y.; Song, Y.; Gao, H.: Training feed-forward artificial neural networks with a modified artificial bee colony algorithm. Neurocomputing (2019)

  38. Zhang, X.; Wang, X.; Chen, H.; Wang, D.; Fu, Z.: Improved GWO for large-scale function optimization and MLP optimization in cancer identification. Neural Comput. Appl., 1–21 (2019)

  39. Heidari, A.A.; Faris, H.; Mirjalili, S.; Aljarah, I.; Mafarja, M.: Ant lion optimizer: theory, literature review, and application in multi-layer perceptron neural networks. In: Nature-inspired optimizers. Springer, Berlin, pp 23–46 (2020)

  40. Dalwinder, S.; Birmohan, S.; Manpreet, K.: Simultaneous feature weighting and parameter determination of neural networks using ant lion optimization for the classification of breast cancer. Biocybernet. Biomed. Eng. (2019)

  41. Faris, H.; Aljarah, I.; Mirjalili, S.: Training feedforward neural networks using multi-verse optimizer for binary classification problems. Appl. Intell. 45(2), 322–332 (2016)

    Google Scholar 

  42. Gao, B.; Li, X.; WooyunTian, W.L.G.: Physics-based image segmentation using first order statistical properties and genetic algorithm for inductive thermography imaging. IEEE Trans. Image Process. 27(5), 2160–2175 (2017)

    MathSciNet  MATH  Google Scholar 

  43. Mutluer, M.; Şahman, M.A.; Çunkaş, M.: Heuristic optimization based on penalty approach for surface permanent magnet synchronous machines. Arab. J. Sci. Eng. 1–17 (2020)

  44. Karasekreter, N.; Şahman, M.A.; Başçiftçi, F.; Fidan, U.: PSO based clustering for the optimization of energy consumption in wireless sensor network. Emerg. Mater. Res, 1–7 (2020)

  45. Kiran, M.S.: TSA: tree-seed algorithm for continuous optimization. Expert Syst. Appl. 42(19), 6686–6698 (2015)

    Google Scholar 

  46. Kıran, M.S.: An implementation of tree-seed algorithm (TSA) for constrained optimization. In: Intelligent and evolutionary systems. Springer, Berlin, pp 189–197 (2016)

  47. Babalik, A.; Cinar, A.C.; Kiran, M.S.: A modification of tree-seed algorithm using Deb’s rules for constrained optimization. Appl. Soft Comput. 63, 289–305 (2018)

    Google Scholar 

  48. El-Fergany, A.A.; Hasanien, H.M.: Tree-seed algorithm for solving optimal power flow problem in large-scale power systems incorporating validations and comparisons. Appl. Soft Comput. 64, 307–316 (2018)

    Google Scholar 

  49. Zhou, J.; Zheng, Y.; Xu, Y.; Liu, H.; Chen, D.: A heuristic TS fuzzy model for the pumped-storage generator-motor using variable-length tree-seed algorithm-based competitive agglomeration. Energies 11(4), 944 (2018)

    Google Scholar 

  50. Horng, S.-C.; Lin, S.-S.: Embedding ordinal optimization into tree–seed algorithm for solving the probabilistic constrained simulation optimization problems. Appl. Sci. 8(11), 2153 (2018)

    MathSciNet  Google Scholar 

  51. Zheng, Y.; Zhou, J.; Zhu, W.; Zhang, C.; Li, C.; Fu, W.: Design of a multi-mode intelligent model predictive control strategy for hydroelectric generating unit. Neurocomputing 207, 287–299 (2016)

    Google Scholar 

  52. Chen, W.; Tan, X.; Cai, M.: Parameter identification of equivalent circuit models for Li-ion batteries based on tree seeds algorithm. In: IOP Conference Series: Earth and Environmental Science, vol 1. IOP Publishing, p 012024 (2017)

  53. Chen, W.; Cai, M.; Tan, X.; Wei, B.: Parameter identification and state-of-charge estimation for Li-Ion batteries using an improved tree seed algorithm. IEICE Trans. Inf. Syst. 102(8), 1489–1497 (2019)

    Google Scholar 

  54. Ding, Z.; Zhao, Y.; Lu, Z.: Simultaneous identification of structural stiffness and mass parameters based on Bare-bones Gaussian Tree Seeds Algorithm using time-domain data. Appl. Soft Comput. 83, 105602 (2019)

    Google Scholar 

  55. Zhao, S.; Wang, N.; Liu, X.: Artificial bee colony algorithm with tree-seed searching for modeling multivariable systems using GRNN. In: 2019 Chinese Control And Decision Conference (CCDC), IEEE, pp. 4702–4707 (2019)

  56. Sahman, M.; Cinar, A.; Saritas, I.; Yasar, A.: Tree-seed algorithm in solving real-life optimization problems. In: IOP conference series: materials science and engineering, vol 1. IOP Publishing (2019)

  57. Ding, Z.; Li, J.; Hao, H.; Lu, Z.-R.: Nonlinear hysteretic parameter identification using an improved tree-seed algorithm. Swarm Evolut. Comput. 46, 69–83 (2019)

    Google Scholar 

  58. Ding, Z.; Li, J.; Hao, H.: Structural damage detection with uncertainties using a modified tree seeds algorithm. In: International Conference on Computational & Experimental Engineering and Sciences, Springer, Berlin, pp. 751–760 (2019)

  59. Muneeswaran, V.; Rajasekaran, M.P.: Gallbladder shape estimation using tree-seed optimization tuned radial basis function network for assessment of acute cholecystitis. In: Intelligent engineering informatics. Springer, pp 229–239 (2018)

  60. Cinar, A.; Kiran, M.: A parallel version of tree-seed algorithm (TSA) within CUDA platform. In: Selçuk International Scientific Conference on Applied Sciences (2016)

  61. Cinar, A.C.; Kiran, M.S.: A parallel implementation of tree-seed algorithm on CUDA-supported graphical processing unit. J Fac Eng Archit Gazi Univ 33(4), 1397–1409 (2018)

    Google Scholar 

  62. Muneeswaran, V.; Rajasekaran, M.P. Beltrami-regularized denoising filter based on tree seed optimization algorithm: an ultrasound image application. In: International conference on information and communication technology for intelligent systems, Springer, pp. 449–457 (2017)

  63. Muneeswaran, V.; Rajasekaran, M.P.: Local contrast regularized contrast limited adaptive histogram equalization using tree seed algorithm—an aid for mammogram images enhancement. In: Smart Intelligent Computing and Applications. Springer, Berlin, pp 693–701 (2019)

  64. Ding, Z.; Li, J.; Hao, H.; Lu, Z.-R.: Structural damage identification with uncertain modelling error and measurement noise by clustering based tree seeds algorithm. Eng. Struct. 185, 301–314 (2019)

    Google Scholar 

  65. Oliva, D.; Elaziz, M.A.; Hinojosa, S.: Otsu’s between class variance and the tree seed algorithm. In: Metaheuristic Algorithms for Image Segmentation: Theory and Applications. Springer, pp 71–83 (2019)

  66. Cinar, A.C.; Kiran, M.S.: Similarity and logic gate-based tree-seed algorithms for binary optimization. Comput. Ind. Eng. 115, 631–646 (2018)

    Google Scholar 

  67. Cinar, A.C.; Iscan, H.; Kiran, M.S.: Tree-seed algorithm for large-scale binary optimization. In: KnE Social Sciences, pp. 48–64 (2018)

  68. Sahman, M.A.; Cinar, A.C.: Binary tree-seed algorithms with S-shaped and V-shaped transfer functions. Int. J. Intell. Syst. Appl. Eng. 7(2), 111–117 (2019)

    Google Scholar 

  69. Kiran, M.S.: Withering process for tree-seed algorithm. Proced. Comput. Sci. 111, 46–51 (2017)

    Google Scholar 

  70. Aslan, M.; Beskirli, M.; Kodaz, H.; Kıran, M.S.: An improved tree seed algorithm for optimization problems. Int. J. Mach. Learn. Comput. 8(1), 20–25 (2018)

    Google Scholar 

  71. Çınar, A.C.; Kıran, M.S. Boundary conditions in Tree-Seed Algorithm: analysis of the success of search space limitation techniques in Tree-Seed Algorithm. In: 2017 International Conference on Computer Science and Engineering (UBMK), IEEE, pp. 571–576 (2017)

  72. Beşkirli, A.; Özdemir, D.; Temurtaş, H.: A comparison of modified tree–seed algorithm for high-dimensional numerical functions. Neural Comput. Appl., 1–35 (2019)

  73. Gungor, I.; Emiroglu, B.G.; Cinar, A.C.; Kiran, M.S.: Integration search strategies in tree seed algorithm for high dimensional function optimization. Int. J. Mach. Learn. Cybernet., 1–19 (2019)

  74. Jiang, J.; Jiang, S.; Meng, X.; Qiu, C.: EST-TSA: An effective search tendency based to tree seed algorithm. Physica A 534, 122323 (2019)

    Google Scholar 

  75. Jiang, J.; Xu, M.; Meng, X.; Li, K.: STSA: A sine Tree-Seed Algorithm for complex continuous optimization problems. Physica A 537, 122802 (2020)

    Google Scholar 

  76. Beşkirli, M.; Yüksek, B.: Test Fonksiyonlarında Ağaç Tohum Algoritmasının Performans Analizi. Avrupa Bilim ve Teknoloji Dergisi, pp. 93–101

  77. Chen, F.; Ye, Z.; Wang, C.; Yan, L.; Wang, R.: A feature selection approach for network intrusion detection based on tree-seed algorithm and k-nearest neighbor. In: 2018 IEEE 4th International Symposium on Wireless Systems within the International Conferences on Intelligent Data Acquisition and Advanced Computing Systems (IDAACS-SWS), IEEE, pp 68–72 (2018)

Download references

Funding

The authors wish to thank Scientific Research Projects Coordinatorship at Selcuk University and The Scientific and Technological Research Council of Turkey for their institutional supports.

Author information

Authors and Affiliations

  1. Department of Computer Engineering, Faculty of Technology, Selcuk University, 42075, Konya, Turkey

    Ahmet Cevahir Cinar

Authors
  1. Ahmet Cevahir Cinar
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ahmet Cevahir Cinar.

Ethics declarations

Conflicts of interest

The authors declare that they have no conflict of interest.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Cinar, A.C. Training Feed-Forward Multi-Layer Perceptron Artificial Neural Networks with a Tree-Seed Algorithm. Arab J Sci Eng 45, 10915–10938 (2020). https://doi.org/10.1007/s13369-020-04872-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13369-020-04872-1

Keywords

Search

Navigation