skip to main content
research-article

Nanoscale electronic synapses using phase change devices

Authors Info & Claims
Published:29 May 2013Publication History
Skip Abstract Section

Abstract

The memory capacity, computational power, communication bandwidth, energy consumption, and physical size of the brain all tend to scale with the number of synapses, which outnumber neurons by a factor of 10,000. Although progress in cortical simulations using modern digital computers has been rapid, the essential disparity between the classical von Neumann computer architecture and the computational fabric of the nervous system makes large-scale simulations expensive, power hungry, and time consuming. Over the last three decades, CMOS-based neuromorphic implementations of “electronic cortex” have emerged as an energy efficient alternative for modeling neuronal behavior. However, the key ingredient for electronic implementation of any self-learning system—programmable, plastic Hebbian synapses scalable to biological densities—has remained elusive. We demonstrate the viability of implementing such electronic synapses using nanoscale phase change devices. We introduce novel programming schemes for modulation of device conductance to closely mimic the phenomenon of Spike Timing Dependent Plasticity (STDP) observed biologically, and verify through simulations that such plastic phase change devices should support simple correlative learning in networks of spiking neurons. Our devices, when arranged in a crossbar array architecture, could enable the development of synaptronic systems that approach the density (∼1011 synapses per sq cm) and energy efficiency (consuming ∼1pJ per synaptic programming event) of the human brain.

References

  1. Abbott, L. F. and Blum, K. I. 1994. Functional significance of long-term potentiation for sequence learning and prediction. Cerebral Cortex 6, 406--416.Google ScholarGoogle ScholarCross RefCross Ref
  2. Abbott, L. F. and Nelson, S. B. 2000. Synaptic plasticity: taming the beast. Nat. Neurosci. 3 Suppl, 1178--1183.Google ScholarGoogle ScholarCross RefCross Ref
  3. Adler, D., Henisch, H. K., and Mott, S. N. 1978. The mechanism of threshold switching in amorphous alloys. Rev. Mod. Phys. 50, 2, 209--220.Google ScholarGoogle ScholarCross RefCross Ref
  4. Allen, C. and Stevens, C. F. 1994. An evaluation of causes for unreliability of synaptic transmission. Proc. Nat. Acad. Sci. 91, 10380--10383.Google ScholarGoogle ScholarCross RefCross Ref
  5. Ananthanarayanan, R., Esser, S. K., Simon, H. D., and Modha, D. S. 2009. The cat is out of the bag cortical simulations with 109 neurons, 1013 synapses. In Proceedings of the Conference on High Performance Computing Networking, Storage and Analysis. ACM, 1--2. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Ananthanarayanan, R. and Modha, D. S. 2010. Anatomy of a cortical simulator. In Proceedings of the ACM/IEEE Conference on High Performance Networking and Computing. 1--2. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. Arthur, J. V. and Boahen, K. 2006. Learning in silicon: Timing is everything. In Advances in Neural Information Processing Systems 17, B. Sholkopf and Y. Weiss, Eds., MIT Press, 281--1185.Google ScholarGoogle Scholar
  8. Bell, C. C., Han, V. Z., Sugawara, Y., and Grant, K. 1997. Synaptic plasticity in a cerebellum-like structure depends on temporal order. Nature 387, 278--281.Google ScholarGoogle ScholarCross RefCross Ref
  9. Bi, G. Q. and Poo, M. M. 1998. Synaptic modifications in cultured hippocampal neurons: dependence on spike timing, synaptic strength, and postsynaptic cell type. J. Neurosci. 18, 10464--10472.Google ScholarGoogle ScholarCross RefCross Ref
  10. Blum, K. I. and Abbott, L. F. 1996. A model of spatial map formation in the hippocampus of the rat. Neural Computation 8, 85--93. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. Bofill, A., Murray, A. F., and Thompson, D. P. 2001. Circuits for VLSI implementation of temporally asymmetric Hebbian learning. In Advances in Neural Information Processing Systems 14. MIT Press, 1091--1098.Google ScholarGoogle Scholar
  12. Boniardi, M., Ielmini, D., Lavizzari, S., Lacaita, A., Redaelli, A., and Pirovano, A. 2009. Statistical and scaling behavior of structural relaxation effects in phase-change memory (pcm) devices. In Proceedings of the IEEE International.Reliability Physics Symposium. 122--127.Google ScholarGoogle Scholar
  13. Brader, J. M., Senn, W., and Fusi, S. 2007. Learning real-world stimuli in a neural network with spike-driven synaptic dynamics. Neural Computation 19, 2881--912. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. Breitwisch, M., Nirschl, T., et al. 2007. Novel lithography-independent pore phase change memory. In Proceedings of the IEEE Symposium on VLSI Technology. 100--101.Google ScholarGoogle ScholarCross RefCross Ref
  15. Burr, G. W., Breitwisch, M. J., et al. 2010. Phase change memory technology. J. Vac. Sci. Technol., B 28.Google ScholarGoogle ScholarCross RefCross Ref
  16. Choi, H., Jung, H., Lee, J., Yoon, J., Park, J., Jun Seong, D., Lee, W., Hasan, M., Jung, G.-Y., and Hwang, H. 2009. An electrically modifiable synapse array of resistive switching memory. Nanotechnology 20, 34, 345201.Google ScholarGoogle ScholarCross RefCross Ref
  17. Chou, S. Y., Krauss, P. R., and Renstrom, P. J. 1996. Imprint lithography with 25-nanometer resolution. Science 272, 5258, 85--87.Google ScholarGoogle Scholar
  18. Dan, Y. and Poo, M. M. 2004. Spike timing-dependent plasticity of neural circuits. Neuron 44, 23--30.Google ScholarGoogle ScholarCross RefCross Ref
  19. Debanne, D., Gahwiler, B. H., and Thompson, S. M. 1998. Long-term synaptic plasticity between pairs of individual CA3 pyramidal cells in rat hippocampal slice cultures. J. Physiol. (Lond.) 507 (Pt 1), 237--247.Google ScholarGoogle ScholarCross RefCross Ref
  20. Djurfeldt, M., Lundqvist, M., Johansson, C., Rehn, M., Ekeberg, O., and Lansner, A. 2008. Brain-scale simulation of the neocortex on the IBM blue gene/l supercomputer. IBM J. Res. Dev. 52, 31--41. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. Egger, V., Feldmeyer, D., and Sakmann, B. 1999. Coincidence detection and efficacy changes in synaptic connections between spiny stellate neurons of the rat barrel cortex. Nat. Neurosci. 2, 10981105.Google ScholarGoogle ScholarCross RefCross Ref
  22. Feldman, D. E. 2000. Timing-based LTP and LTD at vertical inputs to layer II/III pyramidal cells in rat barrel cortex. Neuron 27, 45--56.Google ScholarGoogle ScholarCross RefCross Ref
  23. Friesz, A. K., Parker, A. C., Zhou, C., Ryu, K., and Sanders, J. M. 2007. A biomimetic carbon nanotube synapse circuit. In Proceedings of the Biomedical Engineering Society (BMES) Annual Fall Meeting.Google ScholarGoogle Scholar
  24. Gallagher, W. J. and Parkin, S. S. P. 2006. Development of the magnetic tunnel junction MRAM at IBM: From first junctions to a 16-mb MRAM demonstrator chip. IBM J. Res. Dev. 50, 1, 5--23. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. Gao, C. and Hammerstrom, D. 2007. Cortical models onto cmol and cmos: Architectures and performance/price. IEEE Trans. Circuits Syst. Regul. Pap. 54, 11, 2502--2515.Google ScholarGoogle ScholarCross RefCross Ref
  26. Gerstner, W., Kempter, R., Van Hemmen, J. L., and Wagner, H. 1996. A neuronal learning rule for sub-millisecond temporal coding. Nature 383, 76--78.Google ScholarGoogle ScholarCross RefCross Ref
  27. Hasegawa, T., Ohno, T., Terabe, K., Tsuruoka, T., Nakayama, T., Gimzewski, J. K., and Aono, M. 2010. Learning abilities achieved by a single solid-state atomic switch. Adv. Mater. 22, 16, 1831--1834.Google ScholarGoogle ScholarCross RefCross Ref
  28. Hopfield, J. J. and Brody, C. D. 2004. Learning rules and network repair in spike-timing-based computation networks. Proc. Nat. Acad. Sci. 101, 1, 337--342.Google ScholarGoogle ScholarCross RefCross Ref
  29. Hynna, K. and Boahen, K. 2007. Silicon neurons that burst when primed. In Proceedings of the IEEE International Symposium on Circuits and Systems. 3363--3366.Google ScholarGoogle Scholar
  30. Indiveri, G., Chicca, E., and Douglas, R. 2006. A VLSI array of low-power spiking neurons and bistable synapses with spike-timing dependent plasticity. IEEE Trans. Neural Networks 17, 1, 211--221. Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. Izhikevich, E. M. 2006. Polychronization: computation with spikes. Neural Comput 18, 245--282. Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. Izhikevich, E. M. and Edelman, G. M. 2008. Large-scale model of mammalian thalamocortical systems. Proc. Nat. Acad. Sci. 105, 9, 3593--3598.Google ScholarGoogle ScholarCross RefCross Ref
  33. Jo, S. H., Chang, T., Ebong, I., Bhadviya, B. B., Mazumder, P., and Lu, W. 2010. Nanoscale memristor device as synapse in neuromorphic systems. Nano Lett. 10, 4, 1297--1301.Google ScholarGoogle ScholarCross RefCross Ref
  34. Kandel, E. R. 2000. Nobel Lectures, Physiology or Medicine 1996--2000.Google ScholarGoogle Scholar
  35. Lai, S. and Lowrey, T. 2001. Oum: A 180nm nonvolatile memory cell element technology for stand alone and embedded applications. In Proceedings of the International Electron Devices Meeting, IEDM Technical Digest. 36.5.1--36.5.4.Google ScholarGoogle Scholar
  36. Lazzaro, J. 1992. Low-power silicon spiking neurons and axons. In Proceedings of the IEEE International Symposium on Circuits and Systems (ISCAS'92). Vol. 5. 2220--2223.Google ScholarGoogle ScholarCross RefCross Ref
  37. Likharev, K. and Strukov, D. 2005. Cmol - devices, circuits, and architectures. In Introducing Molecular Electronics, G. Cuniberti, K. Richter, and G. Fages, Eds., Springer, 447--477.Google ScholarGoogle Scholar
  38. Maass, W. and Natschlager, T. 2000. A model for fast analog computation based on unreliable synapses. Neural Comput. 12, 1679--1704. Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. Magee, J. C. and Johnston, D. 1997. A synaptically controlled, associative signal for Hebbian plasticity in hippocampal neurons. Science 275, 5297, 209--213.Google ScholarGoogle Scholar
  40. Markram, H., Lübke, J., Frotscher, M., and Sakmann, B. 1997. Regulation of synaptic efficacy by coincidence of postsynaptic APs and EPSPs. Science 275, 5297, 213--215.Google ScholarGoogle Scholar
  41. Mead, C. 1990. Neuromorphic electronic systems. Proc. IEEE 78, 10, 1629--1636.Google ScholarGoogle ScholarCross RefCross Ref
  42. Mehta, M. R., Quirk, M. C., and Wilson, M. A. 2000. Experience-dependent asymmetric shape of hippocampal receptive fields. Neuron 25, 707--715.Google ScholarGoogle ScholarCross RefCross Ref
  43. Minai, A. A. and Levy, W. B. 1993. Sequence learning in a single trial. In Proceedings of the INNS World Congress on Neural Networks. Erlbaum, 505--508.Google ScholarGoogle Scholar
  44. Nirschl, T., Phipp, J., et al. 2007. Write strategies for 2 and 4-bit multi-level phase-change memory. In Proceedings of the IEEE International Electron Devices Meeting. 461--464.Google ScholarGoogle ScholarCross RefCross Ref
  45. Ovshinsky, S. R. 1968. Reversible electrical switching phenomena in disordered structures. Phys. Rev. Lett. 21, 20, 1450--1453.Google ScholarGoogle ScholarCross RefCross Ref
  46. Ovshinsky, S. R. 2004a. Optical cognitive information processing—A new field. Japanese J. Appl. Phys. 43, 7B, 4695--4699.Google ScholarGoogle ScholarCross RefCross Ref
  47. Ovshinsky, S. R. 2004b. The ovonic cognitive computer: A new paradigm. In EPCOS Library.Google ScholarGoogle Scholar
  48. Pirovano, A., Lacaita, A., Pellizzer, F., Kostylev, S., Benvenuti, A., and Bez, R. 2004. Low-field amorphous state resistance and threshold voltage drift in chalcogenide materials. IEEE Trans. Electron Devices 51, 5, 714--719.Google ScholarGoogle ScholarCross RefCross Ref
  49. Rao, R. P. N. and Sejnowski, T. J. 2001. Spike-timing-dependent hebbian plasticity as temporal difference learning. Neural Comput. 13, 2221--2237. Google ScholarGoogle ScholarDigital LibraryDigital Library
  50. Raoux, S. and Wuttig, M. 2009. Phase Change Materials. Springer.Google ScholarGoogle Scholar
  51. Redaelli, A., Pirovano, A., Benvenuti, A., and Lacaita, A. L. 2008. Threshold switching and phase transition numerical models for phase change memory simulations. J. Appl. Phys. 103, 11, 111101.Google ScholarGoogle ScholarCross RefCross Ref
  52. Roberts, P. D. 1999. Computational consequences of temporally asymmetric learning rules: I. Differential Hebbian learning. J. Comput. Neurosci. 7, 235--246.Google ScholarGoogle ScholarCross RefCross Ref
  53. Rochester, N., Holland, J. H., Haibt, L. H., and Duda, W. L. 1956. Tests on a cell assembly theory of the action of the brain using a large digital computer. IRE Trans. Inf. Theory PGIT-2, 3, 80--93.Google ScholarGoogle ScholarCross RefCross Ref
  54. Scott, J. and Bozano, L. 2007. Nonvolatile memory elements based on organic materials. Adv. Mater. 19, 11, 1452--1463.Google ScholarGoogle ScholarCross RefCross Ref
  55. Silver, R., Boahen, K., Grillner, S., Kopell, N., and Olsen, K. L. 2007. Neurotech for neuroscience: Unifying concepts, organizing principles, and emerging tools. J. Neurosci. 27, 44, 11807--11819.Google ScholarGoogle ScholarCross RefCross Ref
  56. Song, S., Miller, K. D., and Abbott, L. F. 2000. Competitive Hebbian learning through spike-timingdependent synaptic plasticity. Nat. Neurosci. 3, 919--926.Google ScholarGoogle ScholarCross RefCross Ref
  57. Tour, J., Van Zandt, W., Husband, C., Husband, S., Wilson, L., Franzon, P., and Nackashi, D. 2002. Nanocell logic gates for molecular computing. IEEE Trans. Nanotechnol. 1, 2, 100--109. Google ScholarGoogle ScholarDigital LibraryDigital Library
  58. Waser, R., Dittmann, R., Staikov, G., and Szot, K. 2009. Redox-based resistive switching memories nanoionic mechanisms, prospects, and challenges. Adv. Mater. 21, 25, 2632--2663.Google ScholarGoogle ScholarCross RefCross Ref
  59. Wuttig, M. and Yamada, N. 2007. Phase-change materials for rewriteable data storage. Nat. Mater. 6, 5258, 824--832.Google ScholarGoogle Scholar
  60. Zhang, L. I., Tao, H. W., Holt, C. E., Harris, W. A., and Poo, M. 1998. A critical window for cooperation and competition among developing retinotectal synapses. Nature 395, 37--44.Google ScholarGoogle ScholarCross RefCross Ref

Index Terms

  1. Nanoscale electronic synapses using phase change devices

    Recommendations

    Comments

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Sign in

    Full Access

    • Published in

      cover image ACM Journal on Emerging Technologies in Computing Systems
      ACM Journal on Emerging Technologies in Computing Systems  Volume 9, Issue 2
      Special issue on memory technologies
      May 2013
      133 pages
      ISSN:1550-4832
      EISSN:1550-4840
      DOI:10.1145/2463585
      Issue’s Table of Contents

      Copyright © 2013 ACM

      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      • Published: 29 May 2013
      • Accepted: 1 June 2011
      • Revised: 1 May 2011
      • Received: 1 February 2011
      Published in jetc Volume 9, Issue 2

      Permissions

      Request permissions about this article.

      Request Permissions

      Check for updates

      Qualifiers

      • research-article
      • Research
      • Refereed

    PDF Format

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader