Skip to main content
SpringerLink
Log in

A small spiking neural network with LQR control applied to the acrobot

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

This paper presents the results of a computer simulation which, combined a small network of spiking neurons with linear quadratic regulator (LQR) control to solve the acrobot swing-up and balance task. To our knowledge, this task has not been previously solved with spiking neural networks. Input to the network was drawn from the state of the acrobot, and output was torque, either directly applied to the actuated joint, or via the switching of an LQR controller designed for balance. The neural network’s weights were tuned using a (μ + λ)-evolution strategy without recombination, and neurons’ parameters, were chosen to roughly approximate biological neurons.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Anderson, Moore (1971) Linear optimal control. Prentice Hall, Englewood Cliffs

  2. Beyer H-S (2001) The theory of evolution strategies. Springer, Heidelberg

  3. Boone G (1997) Minimum-time control of the acrobot. In: Proceedings of IEEE international conference on robotics and automation, vol 4, pp 3281–3287

  4. Coulom R (2004) High-accuracy value-function approximation with neural networks applied to the acrobot. European Symposium on Artificial Neural Networks

  5. Federici D (2005) Evolving developing spiking neural networks. In: The IEEE congress on evolutionary computation, vol 1, pp 543–550

  6. Dario Floreano, Yann Epars, Jean-Christophe Zufferey, Claudio Mattiussi (2006) Evolution of spiking neural circuits in autonomous mobile robots: Research articles. Int J Intell Syst 21(9):1005–1024

  7. French RLB, Damper RI (2002) Evolution of a circuit of spiking neurons for phototaxis in a Braitenberg vehicle. In: ICSAB: proceedings of the seventh international conference on simulation of adaptive behavior on From animals to animats. MIT Press, Cambridge, pp 335–344

  8. Gerstner W (2001) Pulsed neural networks, Chap. 1: Spiking neurons, pp 3–53. In: Maass and Bishop [15]

  9. Gerstner W, Werner M.K (2002) Spiking neuron models: single neurons, populations, plasticity. Formal spiking neuron models, Chap. 4. Cambridge University Press, Cambridge

  10. Joshi P, Maass W (2005) Movement generation with circuits of spiking neurons. Neural Comput 17(8):1715–1738

    Article  MATH  Google Scholar 

  11. Kandel ER, Schwartz JH, Jessell TM (2000) Principles of neural science, 4th edn. McGraw-Hill, New York

  12. Kawada K, Fujisawa S, Obika M, Yamamoto T (2005) Creating swing-up patterns of an acrobot using evolutionary computation. Proceedings of IEEE international symposium on computational intelligence in robotics and automation, CIRA 2005, pp 261–266

  13. Lai X, She JH, Ohyama Y, Cai Z (1999) Fuzzy control strategy for acrobots combining model-free andmodel-based control. IEE Proc Control Theory Appl 146(6):505–510

    Article  Google Scholar 

  14. Maass W (1997) Networks of spiking neurons: the third generation of neural network models. Neural Netw 10(9):1659–1671

    Article  Google Scholar 

  15. Maass W, Bishop CM (eds) (2001) Pulsed neural networks. MIT Press, Cambridge

  16. Markram H (2006) The blue brain project. Nat Rev Neurosci 7(2):153–160

    Article  MathSciNet  Google Scholar 

  17. Nam TK, Fukuhara Y, Mita T, Yamakita M (2002) Swing-up control and avoiding singular problem of an acrobot system. In: Proceedings of the 41st SICE annual conference, SICE 2002

  18. Press WH, Teukolsky ST, Vetterling WT, Flannery BP (2002) Numerical recipes in C: the art of scientific computing, Chap. 16.1, 2nd edn. Cambridge University Press, Cambridge, pp 710–714

  19. Spong MW (1995) The swing up control problem for the acrobot. IEEE Control Syst Magaz 15(1):49–55

    Article  Google Scholar 

  20. Xu X, He H (2002) Residual-gradient-based neural reinforcement learning for the optimal control of an acrobot. In: Proceedings of the 2002 IEEE international symposium on intelligent control, pp 758–763

  21. Yoshimoto J, Nishimura M, Tokita Y, Ishii S (2005) Acrobot control by learning the switching of multiple controllers. Artif Life Robot 9(2):67–71

    Article  Google Scholar 

Download references

Acknowledgments

Funding for this research has been supplied in part by the University of Newcastle Research Scholarship (UNRS) and by The ARC Centre for Complex Dynamic Systems and Control (CDSC). We would also like to thank Maria Seron for helpful discussions.

Author information

Authors and Affiliations

  1. School of Electrical Engineering and Computer Science, The University of Newcastle, Callaghan, NSW, 2308, Australia

    Lukasz Wiklendt, Stephan Chalup & Rick Middleton

Authors
  1. Lukasz Wiklendt
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Stephan Chalup
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Rick Middleton
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Lukasz Wiklendt.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Wiklendt, L., Chalup, S. & Middleton, R. A small spiking neural network with LQR control applied to the acrobot. Neural Comput & Applic 18, 369–375 (2009). https://doi.org/10.1007/s00521-008-0187-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-008-0187-1

Keywords

Search

Navigation