Skip to main content
SpringerLink
Log in

Cognitive Computation with Autonomously Active Neural Networks: An Emerging Field

  • Published:
Cognitive Computation Aims and scope Submit manuscript

Abstract

The human brain is autonomously active. To understand the functional role of this self-sustained neural activity, and its interplay with the sensory data input stream, is an important question in cognitive system research and we review here the present state of theoretical modeling. This review will start with a brief overview of the experimental efforts, together with a discussion of transient versus self-sustained neural activity in the framework of reservoir computing. The main emphasis will be then on two paradigmal neural network architectures showing continuously ongoing transient-state dynamics: saddle point networks and networks of attractor relics. Self-active neural networks are confronted with two seemingly contrasting demands: a stable internal dynamical state and sensitivity to incoming stimuli. We show, that this dilemma can be solved by networks of attractor relics based on competitive neural dynamics, where the attractor relics compete on one side with each other for transient dominance, and on the other side with the dynamical influence of the input signals. Unsupervised and local Hebbian-style online learning then allows the system to build up correlations between the internal dynamical transient states and the sensory input stream. An emergent cognitive capability results from this set-up. The system performs online, and on its own, a nonlinear independent component analysis of the sensory data stream, all the time being continuously and autonomously active. This process maps the independent components of the sensory input onto the attractor relics, which acquire in this way a semantic meaning.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. Gros C. Complex and adaptive dynamical systems, a primer. Berlin: Springer; 2008.

    Google Scholar 

  2. Gros C. Emotions, diffusive emotional control and the motivational problem for autonomous cognitive systems. In: Vallverdu J, Casacuberta D (eds) Handbook of research on synthetic emotions and sociable robotics: new applications in affective computing and artificial intelligence. IGI-Global; 2009 (in press).

  3. Fox MD, Raichle ME. Spontaneous fluctuations in brain activity observed with functional magnetic resonance imaging. Nat Rev Neurosci. 2007;9:700–11.

    Article  CAS  Google Scholar 

  4. Arieli A, Sterkin A, Grinvald A, Aertsen A. Dynamics of ongoing activity: explanation of the large variability in evoked cortical responses. Science. 1996;273:1868–71.

    Article  PubMed  CAS  Google Scholar 

  5. Raichle ME, Mintun MA. Brain work and brain imaging. Annl Rev Neurosci. 2006;29:449–76.

    Article  CAS  Google Scholar 

  6. Vogels TP, Rajan K, Abbott LF. Neural network dynamics. Annl Rev Neurosci. 2005;28:357–76.

    Article  CAS  Google Scholar 

  7. Fiser J, Chiu C, Weliky M. Small modulation of ongoing cortical dynamics by sensory input during natural vision. Nature. 2004;431:573–8.

    Article  PubMed  CAS  Google Scholar 

  8. Fox MD, Corbetta M, Snyder AZ, Vincent JL, Raichle ME. Spontaneous neuronal activity distinguishes human dorsal and ventral attention systems. Proc Natl Acad Sci. 2003;103:10046–51.

    Article  CAS  Google Scholar 

  9. Kenet T, Bibitchkov D, Tsodyks M, Grinvald A, Arieli A. Spontaneously emerging cortical representations of visual attributes. Nature. 2003;425:954–6.

    Article  PubMed  CAS  Google Scholar 

  10. Ringach DL. States of mind. Nature. 2003;425:912–3.

    Article  PubMed  CAS  Google Scholar 

  11. MacLean JN, Watson BO, Aaron GB, Yuste R. Internal dynamics determine the cortical response to thalamic stimulation. Neuron. 2005;48:811–23.

    Article  PubMed  CAS  Google Scholar 

  12. Fox MD, Snyder AZ, Vincent JL, Corbetta M, Van Essen DC, Raichle ME. The human brain is intrinsically organized into dynamic, anticorrelated functional networks. Proc Natl Acad Sci. 2005;102:9673–8.

    Article  PubMed  CAS  Google Scholar 

  13. Abeles M, Bergman H, Gat I, Meilijson I, Seidemann E, Tishby N, et al. Cortical activity flips among quasi-stationary states. Proc Natl Acad Sci. 1995;92:8616–20.

    Article  PubMed  CAS  Google Scholar 

  14. Gros C. Self-sustained thought processes in a dense associative network. In: Furbach, U (ed) KI 2005: Advances in artificial intelligence, Springer lecture notes in artificial intelligence 3698; 2005. p. 366–79.

  15. Edelman GM, Tononi GA. A universe of consciousness. New York: Basic Books; 2000.

    Google Scholar 

  16. Edelman GM. Naturalizing consciousness: a theoretical framework. Proc Natl Acad Sci. 2003;100:5520–4.

    Article  PubMed  CAS  Google Scholar 

  17. Baars BJ, Ramsoy TZ, Laureys S. Brain, conscious experience and the observing self. Trend Neurosci. 2003;26:671–5.

    Article  PubMed  CAS  Google Scholar 

  18. Morcom AM, Fletcher PC. Does the brain have a baseline? Why we should be resisting a rest. Neuroimage. 2007;37:1073–82.

    Article  Google Scholar 

  19. Vincent JL, et al. Intrinsic functional architecture in the anaesthetized monkey brain. Nature. 2007;447:83–6.

    Article  PubMed  CAS  Google Scholar 

  20. Greicius MD, et al. Persistent default-mode network connectivity during light sedation. Human Brain Map. 2008;29:839–47.

    Article  Google Scholar 

  21. Pagnoni G, Cekic M, Guo Y. Thinking about not-thinking: neural correlates of conceptual processing during Zen meditation. PLoS. 2008;3:1–10.

    Google Scholar 

  22. Jaeger H. The “echo State”: approach to analysing and training recurrent neural networks. GMD-Forschungszentrum Informationstechnik; 2001.

  23. Jaeger H, Haas H. Harnessing nonlinearity: predicting chaotic systems and saving energy in wireless communication. Science. 2004;304:78–80.

    Article  PubMed  CAS  Google Scholar 

  24. Maass W, Natschlager T, Markram H. Real-time computing without stable states: a new framework for neural computation based on perturbations. Neural Comput. 2002;14:2531–60.

    Article  PubMed  Google Scholar 

  25. Maass W, Markram H. On the computational power of recurrent circuits of spiking neurons. J Comput Syst Sci. 2004;69:593–616.

    Article  Google Scholar 

  26. Maass W, Joshi P, Sontag ED. Computational aspects of feedback in neural circuits. PLoS Comput Biol. 2007;3:e165.

    Article  PubMed  CAS  Google Scholar 

  27. Baars BJ, Franklin S. How conscious experience and working memory interact. Trend Coginit Sci. 2003;7:166–72.

    Article  Google Scholar 

  28. Dehaene S, Naccache L. Towards a cognitive neuroscience of consciousness: basic evidence and a workspace framework. Cognition. 2003;79:1–37.

    Article  Google Scholar 

  29. Shanahan M. A spiking neuron model of cortical broadcast and competition. Conscious Cognit. 2003;17:288–303.

    Article  Google Scholar 

  30. Shadlen MN, Newsome WT. Noise, neural codes and cortical organization. Find Curr Opin Cognit Neurosci. 1998;4:569–79.

    Google Scholar 

  31. Eggermont JJ. Is there a neural code? Neurosci Biobehav Rev. 1998;22:355–70.

    Article  PubMed  CAS  Google Scholar 

  32. Averbeck BB, Lee D. Coding and transmission of information by neural ensembles. Trend Neurosci. 2004;27:225–30.

    Article  PubMed  CAS  Google Scholar 

  33. VanRullen R, Koch C. Is perception discrete or continuous? Trend Cognit Sci. 2003;5:207–13.

    Article  Google Scholar 

  34. Kline K, Holcombe AO, Eagleman DM. Illusory motion reversal is caused by rivalry, not by perceptual snapshots of the visual field. Vision Res. 2004;44:2653–8.

    Article  PubMed  Google Scholar 

  35. VanRullen R. The continuous wagon wheel illusion is object-based. Vision Res. 2006;46:4091–5.

    Article  PubMed  Google Scholar 

  36. Rabinovich M, Huerta R, Varona P, Afraimovich VS. Transient cognitive dynamics, metastability, and decision making. PLoS Comput Biol. 2008;4:e1000072.

    Article  PubMed  CAS  Google Scholar 

  37. Krupa M. Robust heteroclinic cycles. J Nonlinear Sci. 1997;7:129–76.

    Article  Google Scholar 

  38. Rabinovich M, Volkovskii A, Lecanda P, Huerta R, Abarbanel HDI, Laurent G. Dynamical encoding by networks of competing neuron groups: winnerless competition. Phys Rev Lett. 2001;87:068102.

    Article  PubMed  CAS  Google Scholar 

  39. Amit DJ. Modeling brain function: the world of attractor neural networks. New York, NY, USA: Cambridge University Press; 1989.

    Google Scholar 

  40. Hasselmo ME, McClelland JL. Neural models of memory. Curr Opin Neurobiol. 1999;9:184–8.

    Article  PubMed  CAS  Google Scholar 

  41. Tsuda I. Toward an interpretation of dynamic neural activity in terms of chaotic dynamical systems. Behav Brain Sci. 2002;24:793–810.

    Article  Google Scholar 

  42. Horn D, Usher M. Neural networks with dynamical thresholds. Phys Rev A. 1989;40:1036–44.

    Article  PubMed  Google Scholar 

  43. Sompolinsky H, Kanter I. Temporal association in asymmetric neural networks. Phys Rev Lett. 1986;57:2861–4.

    Article  PubMed  Google Scholar 

  44. Gros C. Neural networks with transient state dynamics. New J Phys. 2007;9:109.

    Article  Google Scholar 

  45. Maass W. On the computational power of winner-take-all. Neural Comput. 2000;12:2519–35.

    Article  PubMed  CAS  Google Scholar 

  46. O’Reilly RC. Six principles for biologically based computational models of cortical cognition. Trend Cognit Sci. 1998;2:455–62.

    Article  Google Scholar 

  47. Crick FC, Koch C. A framework for consciousness. Nat Neurosci. 2003;6:119–26.

    Article  PubMed  CAS  Google Scholar 

  48. Koch C. The quest for consciousness—a neurobiological approach. Robert and Company; 2004.

  49. Quiroga RQ, Kreiman G, Koch C, Fried I. Sparse but not grandmother-cell coding in the medial temporal lobe. Trend Cognit Sci. 2008;12:87–91.

    Article  Google Scholar 

  50. Olshausen BA, Field DJ. Sparse coding of sensory inputs. Curr Opin Neurobiol. 2004;14:481–7.

    Article  PubMed  CAS  Google Scholar 

  51. Lin L, Osan R, Shoham S, Jin W, Zuo W, Tsien JZ. Identification of network-level coding units for real-time representation of episodic experiences in the hippocampus. Proc Natl Acad Sci. 2005;102:6125–613.

    Article  PubMed  CAS  Google Scholar 

  52. Lin L, Osan R, Tsien JZ. Organizing principles of real-time memory encoding: neural clique assemblies and universal neural codes. Trend Neurosci. 2006;29:48–57.

    Article  PubMed  CAS  Google Scholar 

  53. Quiroga RQ, Reddy L, Kreiman G, Koch C, Fried I. Invariant visual representation by single neurons in the human brain. Nature. 2005;435:1102–7.

    Article  PubMed  CAS  Google Scholar 

  54. Nelson DL, McEvoy CL, Schreiber TA. The University of South Florida free association, rhyme, and word fragment norms. Behav Res Method Instr Comput. 2004;36:402–7.

    Google Scholar 

  55. Palla G, Derényi I, Farkas I, Vicsek T. Uncovering the overlapping community structure of complex networks in nature and society. Nature. 2005;435:814–8.

    Article  PubMed  CAS  Google Scholar 

  56. Riesenhuber M, Poggio T. Hierarchical models of object recognition in cortex. Nat Neurosci. 1999;2:1019–25.

    Article  PubMed  CAS  Google Scholar 

  57. Kaczor G, Gros C. Evolving complex networks with conserved clique distributions. Phys Rev E. 2008;78:016107.

    Google Scholar 

  58. Gros C, Kaczor G. Semantic learning in autonomously active recurrent neural networks. Preprint.

  59. Arbib MA. The handbook of brain theory and neural networks. Cambridge, MA: MIT Press; 2002.

    Google Scholar 

  60. Von der Malsburg C. The what and why of binding: the modeler’s perspective. Neuron. 1999;24:95–104.

    Article  PubMed  Google Scholar 

  61. Singer W, Gray CM. Visual feature integration and the temporal correlation hypothesis. Annl Rev Neurosci. 1995;18:555–86.

    Article  CAS  Google Scholar 

  62. Berns GS, Cohen JD, Mintun MA. Brain regions responsive to novelty in the absence of awareness. Science. 1997;276:1272–5.

    Article  PubMed  CAS  Google Scholar 

  63. Barceló F, Periáñez JA, Knight RT. Think differently: a brain orienting response to task novelty. NeuroReport. 2002;13:1887–92.

    Article  PubMed  Google Scholar 

  64. Wise RA. Dopamine, learning and motivation. Nat Rev Neurosci. 2004;4:483–94.

    Article  CAS  Google Scholar 

  65. Redgrave P, Gurney K. The short-latency dopamine signal: a role in discovering novel actions? Nat Rev Neurosci. 2006;7:967–75.

    Article  PubMed  CAS  Google Scholar 

  66. Bienenstock EL, Cooper LN, Munro PW. Theory for the development of neuron selectivity: orientation specificity and binocular interaction in visual cortex. J Neurosci. 1982;2:32–48.

    PubMed  CAS  Google Scholar 

  67. Gros C, Kaczor G. Learning in cognitive systems with autonomous dynamics. In: Proceedings of the 2008 international conference on cognitive systems, Karlsruhe; 2008.

  68. Hyvärinen A, Oja E. Independent component analysis: algorithms and applications. Neural Netw. 2000;13:411–30.

    Article  PubMed  Google Scholar 

  69. Choi S, Cichocki A, Park HM, Lee SY. Blind source separation and independent component analysis: a review. Neural Inform Process. 2005;6:1–57.

    Google Scholar 

  70. Földiák P. Forming sparse representations by local anti-Hebbian learning. Biol Cybernet. 1990;64:165–70.

    Article  Google Scholar 

  71. Butko N, Triesch J. Learning sensory representations with intrinsic plasticity. Neurocomputing. 2007;70:1130–8.

    Article  Google Scholar 

  72. Chialvo DR, Bak P. Learning from mistakes. Neuroscience. 1999;90:1137–48.

    Article  PubMed  CAS  Google Scholar 

  73. Haykin S. Neural networks: a comprehensive foundation. Upper Saddle River, NJ: Prentice Hall; 1994.

    Google Scholar 

  74. Dreyfus G. Neural networks: methodology and applications. Berlin: Springer; 2005.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Institute of Theoretical Physics, J.W. Goethe University Frankfurt, 60054, Frankfurt am Main, Germany

    Claudius Gros

Authors
  1. Claudius Gros
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Claudius Gros.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Gros, C. Cognitive Computation with Autonomously Active Neural Networks: An Emerging Field. Cogn Comput 1, 77–90 (2009). https://doi.org/10.1007/s12559-008-9000-9

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12559-008-9000-9

Keywords

Search

Navigation