Skip to main content

Advertisement

SpringerLink
Log in

On three intelligent systems: dynamic neural, fuzzy, and wavelet networks for training trajectory

  • Original Article
  • Published:
Neural Computing & Applications Aims and scope Submit manuscript

Abstract

Intelligent systems cover a wide range of technologies related to hard sciences, such as modeling and control theory, and soft sciences, such as the artificial intelligence (AI). Intelligent systems, including neural networks (NNs), fuzzy logic (FL), and wavelet techniques, utilize the concepts of biological systems and human cognitive capabilities. These three systems have been recognized as a robust and attractive alternative to the some of the classical modeling and control methods. The application of classical NNs, FL, and wavelet technology to dynamic system modeling and control has been constrained by the non-dynamic nature of their popular architectures. The major drawbacks of these architectures are the curse of dimensionality, such as the requirement of too many parameters in NNs, the use of large rule bases in FL, the large number of wavelets, and the long training times, etc. These problems can be overcome with dynamic network structures, referred to as dynamic neural networks (DNNs), dynamic fuzzy networks (DFNs), and dynamic wavelet networks (DWNs), which have unconstrained connectivity and dynamic neural, fuzzy, and wavelet processing units, called “neurons”, “feurons”, and “wavelons”, respectively. The structure of dynamic networks are based on Hopfield networks. Here, we present a comparative study of DNNs, DFNs, and DWNs for non-linear dynamical system modeling. All three dynamic networks have a lag dynamic, an activation function, and interconnection weights. The network weights are adjusted using fast training (optimization) algorithms (quasi-Newton methods). Also, it has been shown that all dynamic networks can be effectively used in non-linear system modeling, and that DWNs result in the best capacity. But all networks have non-linearity properties in non-linear systems. In this study, all dynamic networks are considered as a non-linear optimization with dynamic equality constraints for non-linear system modeling. They encapsulate and generalize the target trajectories. The adjoint theory, whose computational complexity is significantly less than the direct method, has been used in the training of the networks. The updating of weights (identification of network parameters) is based on Broyden–Fletcher–Goldfarb–Shanno method. First, phase portrait examples are given. From this, it has been shown that they have oscillatory and chaotic properties. A dynamical system with discrete events is modeled using the above network structure. There is a localization property at discrete event instants for time and frequency in this example.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4a, b
Fig. 5
Fig. 6
Fig. 7a–f
Fig. 8a–c
Fig. 9
Fig. 10
Fig. 11a–f
Fig. 12a–c

Similar content being viewed by others

References

  1. Becerikli Y (1998) Neuro-optimal control. PhD thesis, Sakarya University, Sakarya, Turkey

  2. Becerikli Y, Oysal Y, Konar AF (2002) Modeling of nonlinear systems with dynamic wavelet networks. In: Proceedings of the Turkish symposium on automatic control (TOK 2002), Ankara, Turkey, 9–11 September 2002, pp 71–79

  3. Becerikli Y, Konar AF, Samad T (2003) Intelligent optimal control with dynamic neural networks. Neural Netw 16(2):251–259

    Article  Google Scholar 

  4. Becerikli Y, Oysal Y, Konar AF (2004) Trajectory priming with dynamic fuzzy networks in nonlinear optimal control. IEEE Trans Neural Net 15(2):383–394

    Article  Google Scholar 

  5. Bryson AE, Ho YC (1975) Applied optimal control. Hemisphere, New York

  6. Bullock TE (1966) Computation of optimal controls by a method based on second variations. Department of Aeronautics and Astronautics, Stanford University, SUDAAR 297, December 1966

  7. Cannon M, Slotine J-JE (1995) Space-frequency localized basis function networks for nonlinear system estimation and control. Neurocomputing 9(3):293–342

    Article  MATH  Google Scholar 

  8. Cao SG, Rees NW (1995) Identification of dynamic fuzzy model. Fuzzy Set Syst 74(3):307–320

    Article  MATH  Google Scholar 

  9. Chapra SC, Canale RP (1989) Numerical methods for engineers. McGraw-Hill, New York

  10. Chong EKP Zyak SH (1996) An introduction to optimization. Wiley, New York

  11. Cybenko G (1989) Approximation by superpositions of a sigmoidal function. Math Control Signal 2:303–314

    MathSciNet  MATH  Google Scholar 

  12. Erdol N, Başbuğ F (1996) Wavelet transform based adaptive filters: analysis and new results. IEEE Trans Signal Process 44(9):2163–2169

    Article  Google Scholar 

  13. Geva S, Sitte J (1992) A constructive method for multivariate function approximation by multilayer perceptrons. IEEE Trans Neural Netw 3(4):621–624

    Article  Google Scholar 

  14. Gevarter WB (1983) An overview of artificial intelligence and robotics, vol I. NASA technical memorandum 85836

  15. Gherrity M (1989) A learning algorithm for analog, fully recurrent neural networks. In: Proceedings of the international joint conference on neural networks (IJCNN’89), Washington, DC, June 1989, vol. 1, pp 643–644

  16. Hasdorff L (1976) Gradient optimization and nonlinear control. Wiley, New York

  17. Hopfield JJ (1982) Neural networks and physical systems with emergent collective computational abilities. Proc Nat Acad Sci USA 79:2554–2558

    Google Scholar 

  18. Hopfield JJ (1984) Neurons with graded response have collective computational properties like those of two-state neurons. Proc Nat Acad Sci USA 81:3088–3092

    Google Scholar 

  19. Hornik K, Stinchcombe M, White H (1985) Multilayer feedforward networks are universal approximators. Neural Netw 2:359–366

    Article  Google Scholar 

  20. Hornik K, Stinchcombe M, White H, Auer P (1994) Degree of approximation results for feedforward networks approximating unknown mappings and their derivatives. Neural Comput 6(6):1262–1275

    MATH  Google Scholar 

  21. Hunt KJ, Sbarbaro D, Zbikowski R, Gawthrop PJ (1992) Neural networks for control systems—a survey. Automatica 28(6):1083–1112

    Article  Google Scholar 

  22. Iannou PA, Datta A (1991) Robust adaptive control: a unified approach. Proc IEEE 79(12):1736–1768

    Article  Google Scholar 

  23. Jang JSR, Sun CT, Mizutani E (1997) Neuro-fuzzy and soft computing: a computational approach to learning and machine intelligence. Prentice Hall, Englewood Cliffs, New Jersey

  24. Jordan MI (1986) Attractor dynamics and parallelism in connectionist sequential machine. In: Proceedings of the 8th annual conference of the cognitive science society, Amherst, Massachusetts. Lawrence Erlbaum, Hillsdale, New Jersey

  25. Khalil HK (1996) Nonlinear systems. Prentice-Hall, Englewood Cliffs, New Jersey

  26. Kirk DE (1970) Optimal control theory: an introduction. Prentice-Hall, Englewood Cliffs, New Jersey

    MATH  Google Scholar 

  27. Konar AF (1991) Gradient and curvature in nonlinear identification. In: Proceedings of the Honeywell advanced control workshop, Minneapolis, Minnesota, January 1991

  28. Konar AF (1997) Historical and philosophical perspectives on intelligent and neural control. In: Proceedings of the IEEE international symposium on intelligent control (ISIC’97), Istanbul, Turkey, July 1997

  29. Konar AF, Samad T (1991) Hybrid neural network/algorithmic system identification. Technical report SSDC-91-I 4051–1, Honeywell Technology Center, 3660 Technology Drive, Minneapolis, MN 55418

  30. Konar AF, Samad T (1992) Dynamic neural networks. Technical report SSDC-92-I 4152–2, Honeywell Technology Center, 3660 Technology Drive, Minneapolis, MN 55418

  31. Konar AF, Becerikli Y, Samad T (1997) Trajectory tracking with dynamic neural networks. In: Proceedings of the IEEE international symposium on intelligent control (ISIC’97), Istanbul, Turkey, July 1997

  32. Kosko B (1997) Fuzzy engineering. Prentice-Hall, Upper Saddle River, New Jersey

  33. Kosmatopoulos EB, Polycarpas MM, Christodoulou MA, Iannou PA (1995) High-order neural network structures for identification of dynamical systems. IEEE Trans Neural Netw 6(2):431–442

    Article  Google Scholar 

  34. Lapidus L, Luus R (1996) Optimal control of engineering processes. Blaisdell, Waltham, Massachusetts

  35. Leistritz L, Galicki M, Witte H, Kochs E (2002) Training trajectories by continuous recurrent multilayer networks. IEEE Trans Neural Netw 13(2):283–291

    Article  Google Scholar 

  36. Leonard J, Kramer M (1991) Radial basis function networks for classifying process faults. IEEE Control Syst 11:31–38

    Article  Google Scholar 

  37. Lewis F (1999) Nonlinear network structures for feedback control. Asian J Control 1:205–228

    MATH  Google Scholar 

  38. Lewis FL (1992) Applied optimal control and estimation. Prentice-Hall, Englewood Cliffs, New Jersey

  39. Lippmann RP (1987) An introduction to computing with neural nets, IEEE ASSP Mag 4(2):4–22

  40. Mallat SG (1987) Multifrequency channel decompositions of images and wavelet models. IEEE Trans on ASSP 37(12):2091–2109

    Google Scholar 

  41. Mauer GF (1995) A fuzzy logic controller for an ABS braking system. IEEE Trans on Fuzzy Systems 3(4):381–388

    Article  Google Scholar 

  42. Mitter SK (1996) Successive approximation methods for the solution of optimal control problems. Automatica 3:135–149

    Article  MATH  Google Scholar 

  43. Morris AJ (1996) Artificial neural networks in process engineering. IEE Proc D 138(3):256–266

    Google Scholar 

  44. Nakamura S (1991) Applied numerical methods with software. Prentice-Hall, Englewood Cliffs, New Jersey

  45. Oussar Y, Rivals I, Personnaz L, Dreyfus G (1998) Training wavelet networks for nonlinear dynamic input-output modeling. Neurocomputing 20:173–188

    Article  MATH  Google Scholar 

  46. Oysal Y (2002) Feuro modeling and optimal fuzzy control. PhD thesis, Sakarya University, Sakarya, Turkey

  47. Oysal Y, Becerikli Y, Konar AF (2003) Generalized modeling principles of a nonlinear system with a dynamic fuzzy network. Comput Chem Eng 27(11):1657–1664

    Article  Google Scholar 

  48. Oysal Y, Becerikli Y, Konar AF (2004) Phase portrait modeling of a nonlinear system with a dynamic fuzzy network. Lecture Notes in Computer Science (LNCS) (accepted 2004)

  49. Palm R, Driankow D, Hellendorn H (1997) Model based fuzzy control. Springer, Berlin Heidelberg New York

  50. Parisi R, Diclaudid ED, Orlandi G, Rao BD (1996) A generalized learning paradigm exploiting the structure of feedforward neural networks. IEEE Trans Neural Networks 7(6):1450–1460

    Article  Google Scholar 

  51. Park J, Sandberg I-W (1991) Universal approximation using radial-basis function networks. Neural Comput 3:246–257

    Google Scholar 

  52. Passino KM, Yurkovich S (1998) Fuzzy control. Addison-Wesley, Menlo Park, California

  53. Pearlmutter B (1989) Learning state space trajectories in recurrent neural networks. Neural Comput 1:263–269

    Google Scholar 

  54. Pham DT, Liu X (1990) Dynamic system identification using partially recurrent neural networks. J Syst Eng 2(2):4–27

    MATH  Google Scholar 

  55. Pierre DA (1986) Optimization theory with applications. Dover Publication, New York

  56. Pineda F (1987) Generalization of backpropagation to recurrent and higher order neural networks. In: Proceedings of the IEEE international joint conference on neural networks (IJCNN’87), San Diego, California, June 1987

  57. Ray WH (1981) New approaches to the dynamics of nonlinear systems with implications for process and control system design. In: Seabor E (ed) Chemical process control II, pp 245–267

  58. Rubio FR, Berenguel M, Camacho EF (1995) Fuzzy logic control for a solar power plant. IEEE Trans on Fuzzy Syst 3(4):459–468

    Article  MATH  Google Scholar 

  59. Rumelhart DE, Hinton GE, Williams RJ (1986) Learning internal representations by back-propagating errors. Nature 323:533–536

    Google Scholar 

  60. Sanner R, Slotine J-JE (1992) Gaussian networks for direct adaptive control. IEEE Trans Neural Netw 13(6):837–863

    Article  Google Scholar 

  61. Scales LE (1985) Introduction to non-linear optimization. Springer, Berlin Heidelberg New York

  62. Servan-Schreiber D, Cleeremans A, McClellans JL (1988) Encoding sequential structure in simple recurrent networks. Technical report CMU-CS-88–183, School of Computer Science, Carnegie Mellon University, Pittsburgh, Pennsylvania

  63. Sugeno M, Kang GT (1998) Structure identification of fuzzy model. Fuzzy Sets Syst 28:15–33

    Article  MATH  Google Scholar 

  64. Takagi T, Sugeno M (1985) Fuzzy identification of systems and its application to modeling and control. IEEE Trans on SMC 15(1):116–132

    MATH  Google Scholar 

  65. Tan Y, Dang X, Liang F, Su CY (2000) Dynamic wavelet neural network for nonlinear dynamic system identification. In: Proceedings of the IEEE international conference on control applications (CCA 2001), Anchorage, Alaska, September 2000

  66. Tanaka K, Sugeno M (1992) Stability analysis and design of fuzzy control systems. Fuzzy Sets Syst 45(2):135–156

    Article  MathSciNet  MATH  Google Scholar 

  67. Tanaka K, Sugeno M (1993) Fuzzy stability criterion of a class of nonlinear systems. Inform Sci 71(1–2):3–26

    Google Scholar 

  68. Tsung F-S (1990) Learning in recurrent finite difference networks. In: Touretzky DS, Elman JL, Sejnowski TJ, Hinton GE (eds) Proceedings of the connectionist models summer school. San Diego, California. Morgan Kaufmann, San Mateo, California

  69. Wang LX (1997) A course in fuzzy systems and control. Prentice-Hall, Upper Saddle River, New Jersey

  70. Wang LX, Mendel JM (1992) Fuzzy basis function, universal approximation, and orthogonal least squares learning. IEEE Trans Neural Netw 3:807–814

    Article  MATH  Google Scholar 

  71. Wang D, Romagnoli JA, Safavi AA (2000) Wavelet-based adaptive robust M-estimator for nonlinear system identification. AIChE J 6(8):1607–1615

    Article  Google Scholar 

  72. Zadeh LA, Desoer CA (1963) Linear system theory. McGraw-Hill, New York

  73. Zhang Q (1997) Using wavelet network in nonparametric estimation. IEEE Trans on Neural Netw 8(2):227–236

    Article  MATH  Google Scholar 

  74. Zhang Q, Benveniste A (1992) Wavelet networks. IEEE Trans Neural Netw 3(6):889–898

    Article  Google Scholar 

  75. Zhang J, Walter GG et al (1995) Wavelet neural networks for function learning. IEEE Trans Signal Process 43(6):1485–1497

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Engineering, Kocaeli University, Izmit, Turkey

    Yasar Becerikli

Authors
  1. Yasar Becerikli
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yasar Becerikli.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Becerikli, Y. On three intelligent systems: dynamic neural, fuzzy, and wavelet networks for training trajectory. Neural Comput & Applic 13, 339–351 (2004). https://doi.org/10.1007/s00521-004-0429-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-004-0429-9

Keywords

Search

Navigation