Gavin Parpart
ABSTRACT
Neuromorphic processors have garnered considerable interest in recent years for their potential in energy-efficient and high-speed computing. The Locally Competitive Algorithm (LCA) has been utilized for power efficient sparse coding on neuromorphic processors, including the first Loihi processor [4, 5]. With the Loihi 2 processor enabling custom neuron models and graded spike communication, more complex implementations of LCA are possible [9]. We present a new implementation of LCA designed for the Loihi 2 processor and perform an initial set of benchmarks comparing it to LCA on CPU and GPU devices. In these experiments LCA on Loihi 2 is orders of magnitude more efficient and faster for large sparsity penalties, while maintaining similar reconstruction quality. We find this performance improvement increases as the LCA parameters are tuned towards greater representation sparsity. Our study highlights the potential of neuromorphic processors, particularly Loihi 2, in enabling intelligent, autonomous, real-time processing on small robots, satellites where there are strict SWaP (small, lightweight, and low power) requirements. By demonstrating the superior performance of LCA on Loihi 2 compared to conventional computing device, our study suggests that Loihi 2 could be a valuable tool in advancing these types of applications. Overall, our study highlights the potential of neuromorphic processors for efficient and accurate data processing on resource-constrained devices.
- James Bradbury, Roy Frostig, Peter Hawkins, Matthew James Johnson, Chris Leary, Dougal Maclaurin, George Necula, Adam Paszke, Jake VanderPlas, Skye Wanderman-Milne, and Qiao Zhang. 2018. JAX: composable transformations of Python+NumPy programs. Google. http://github.com/google/jaxGoogle Scholar
- Brendt Wohlberg. 2017. SPORCO: A Python package for standard and convolutional sparse representations. In Proceedings of the 16th Python in Science Conference, Katy Huff, David Lippa, Dillon Niederhut, and M Pacer (Eds.). 1 -- 8. Google ScholarCross Ref
- Jacob Carroll, Nils Carlson, and Garrett T. Kenyon. 2017. Phase Transitions in Image Denoising via Sparsely Coding Convolutional Neural Networks. CoRR abs/1710.09875 (2017), 4 pages. arXiv:1710.09875 http://arxiv.org/abs/1710.09875Google Scholar
- Mike Davies, Andreas Wild, Garrick Orchard, Yulia Sandamirskaya, Gabriel A. Fonseca Guerra, Prasad Joshi, Philipp Plank, and Sumedh R. Risbud. 2021. Advancing Neuromorphic Computing With Loihi: A Survey of Results and Outlook. Proc. IEEE 109, 5 (2021), 911--934. Google ScholarCross Ref
- Kyle Henke, Michael Teti, Garrett Kenyon, Ben Migliori, and Gerd Kunde. 2022. Apples-to-Spikes: The First Detailed Comparison of LASSO Solutions Generated by a Spiking Neuromorphic Processor. In Proceedings of the International Conference on Neuromorphic Systems 2022 (Knoxville, TN, USA) (ICONS '22). Association for Computing Machinery, New York, NY, USA, Article 19, 8 pages. Google ScholarDigital Library
- Intel 2018. Intel Neuromorphic Research Community. Intel. https://www.intel.com/content/www/us/en/research/neuromorphic-community.htmlGoogle Scholar
- Yann LeCun, Corinna Cortes, and CJ Burges. 2010. MNIST handwritten digit database. ATT Labs. http://yann.lecun.com/exdb/mnistGoogle Scholar
- Bruno A. Olshausen and David J. Field. 1997. Sparse coding with an overcomplete basis set: A strategy employed by V1? Vision Research 37, 23 (1997), 3311--3325. Google ScholarCross Ref
- Garrick Orchard, Edward Paxon Frady, Daniel Ben Dayan Rubin, Sophia Sanborn, Sumit Bam Shrestha, Friedrich T. Sommer, and Mike Davies. 2021. Efficient Neuromorphic Signal Processing with Loihi 2. CoRR abs/2111.03746 (2021), 6 pages. arXiv:2111.03746 https://arxiv.org/abs/2111.03746Google Scholar
- Gavin Parpart, Carlos Gonzalez Rivera, Terrence Stewart, Edward Kim, Jocelyn Rego, Andrew O'Brien, Steven Nesbit, Garrett Kenyon, and Yijing Watkins. 2022. Dictionary Learning with Accumulator Neurons. In Proceedings of the International Conference on Neuromorphic Systems 2022 (Knoxville, TN, USA) (ICONS '22). Association for Computing Machinery, New York, NY, USA, Article 11, 9 pages. Google ScholarDigital Library
- Christopher J. Rozell, Don H. Johnson, Richard G. Baraniuk, and Bruno A. Olshausen. 2008. Sparse Coding via Thresholding and Local Competition in Neural Circuits. Neural Computation 20, 10 (Oct. 2008), 2526--2563. Google ScholarDigital Library
- Peter F. Schultz, Dylan M. Paiton, Wei Lu, and Garrett T. Kenyon. 2014. Replicating Kernels with a Short Stride Allows Sparse Reconstructions with Fewer Independent Kernels. CoRR abs/1406.4205 (2014), 12 pages. arXiv:1406.4205 [qbio.QM]Google Scholar
- Michael Teti, Garrett Kenyon, Ben Migliori, and Juston Moore. 2022. LCANets: Lateral Competition Improves Robustness Against Corruption and Attack. In Proceedings of the 39th International Conference on Machine Learning (Proceedings of Machine Learning Research, Vol. 162), Kamalika Chaudhuri, Stefanie Jegelka, Le Song, Csaba Szepesvari, Gang Niu, and Sivan Sabato (Eds.). PMLR, Baltimore, MD, USA, 21232--21252. https://proceedings.mlr.press/v162/teti22a.htmlGoogle Scholar
- Aaron R. Voelker, Daniel Rasmussen, and Chris Eliasmith. 2020. A Spike in Performance: Training Hybrid-Spiking Neural Networks with Quantized Activation Functions. CoRR abs/2002.03553 (2020), 18 pages. arXiv:2002.03553 https://arxiv.org/abs/2002.03553Google Scholar
- Yijing Watkins, Oleksandr Iaroshenko, Mohammad Sayeh, and Garrett Kenyon. 2018. Image Compression: Sparse Coding vs. Bottleneck Autoencoders. In 2018 IEEE Southwest Symposium on Image Analysis and Interpretation (SSIAI). IEEE, Las Vegas, NV, USA, 17--20. Google ScholarCross Ref
- Yijing Watkins, Austin Thresher, David Mascarenas, and Garrett T. Kenyon. 2018. Sparse Coding Enables the Reconstruction of High-Fidelity Images and Video from Retinal Spike Trains. In Proceedings of the International Conference on Neuromorphic Systems (Knoxville, TN, USA) (ICONS '18). Association for Computing Machinery, New York, NY, USA, Article 8, 5 pages. Google ScholarDigital Library
- Mengchen Zhu and Christopher J Rozell. 2013. Visual nonclassical receptive field effects emerge from sparse coding in a dynamical system. PLoS computational biology 9, 8 (2013), e1003191.Google Scholar
Index Terms
- Implementing and Benchmarking the Locally Competitive Algorithm on the Loihi 2 Neuromorphic Processor
Recommendations
Introducing Astrocytes on a Neuromorphic Processor: Synchronization, Local Plasticity and Edge of Chaos
NICE '19: Proceedings of the 7th Annual Neuro-inspired Computational Elements WorkshopWhile there is still a lot to learn about astrocytes and their neuromodulatory role in the spatial and temporal integration of neuronal activity, their introduction to neuromorphic hardware is timely, facilitating their computational exploration in ...
Solving constraint satisfaction problems using the loihi spiking neuromorphic processor
DATE '20: Proceedings of the 23rd Conference on Design, Automation and Test in EuropeIn many cases, low power autonomous systems need to make decisions extremely efficiently. However, as a potential solution space becomes more complex, finding a solution quickly becomes nearly impossible using traditional computing methods. Thus, in this ...
Apples-to-spikes: The first detailed comparison of LASSO solutions generated by a spiking neuromorphic processor
ICONS '22: Proceedings of the International Conference on Neuromorphic Systems 2022The Locally Competitive Algorithm (LCA) is a model of simple cells in the primary visual cortex, based on convex sparse coding via recurrent lateral competition between neighboring neurons. Previous work implemented spiking LCA (S-LCA) on the Loihi ...
Comments