Skip to main content
SpringerLink
Log in

A generalized hardware architecture for real-time spiking neural networks

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

This article presents an area- and power-efficient hardware architecture for the brain-implantable spiking neural networks (SNNs). The proposed generalized hardware architecture is parameterizable and reconfigurable such that the maximum supported number of neurons, the interconnection structure among neurons, and the resolution of the time step can be readily adjusted for realizing various SNN topologies. The designed SNN hardware architecture is capable of emulating moderately-sized SNNs with tens of thousands of neurons in real-time with varying degrees of parallelism, while reducing the resource utilization by 34% for similarly sized SNNs implemented on a single field-programmable gate array (FPGA). We evaluate the model using the MNIST digit recognition benchmark and show that the network can accurately classify handwritten digits with 89.8% accuracy. Compared to the other recently implemented SNN emulators based on FPGAs, the designed and implemented single-FPGA system is able to emulate moderately-sized SNNs instead of using a cluster of FPGAs or CPUs. The application-specific integrated circuit (ASIC) implementation of a moderately-sized SNN is estimated to occupy 3.6 mm2 of silicon area. Post-layout synthesis and simulation results show that the ASIC will dissipate 3.6 mW of power from a 1.16 V supply while operating at 34.7 MHz in a standard 32-nm CMOS process.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

Availability of data and material

The MNIST datasets used during this study are available at http://yann.lecun.com/exdb/mnist/.

References

  1. National Spinal Cord Injury Statistical Center, Facts and figures at a glance, f Alabama at Birmingham, (2016)

  2. Yin X-X, Sun L, Fu Y, Lu R, Zhang Y (2022) U-net-based medical image segmentation. J Healthcare Eng 15:2022

  3. Lea C, Vidal R, Reiter A, Hager GD (2016) Temporal convolutional networks: a unified approach to action segmentation, in : European Conference on Computer vision

  4. Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN , Kaiserl, Polosukhin I (2017) Attention is all you need, Adv Neural Inform Process Syst, 30

  5. Raganato A, Tiedemann J (2018) An analysis of encoder representations in transformer-based machine translation, in: EMNLP Workshop BlackboxNLP: analyzing and interpreting neural networks for NLP

  6. Maass W (1997) Networks of spiking neurons: the third generation of neural network models. Neural Netw 10(9):1659–1671

    Article  Google Scholar 

  7. Bonifazi P et al (2013) In vitro large-scale experimental and theoretical studies for the realization of bi-directional brain-prostheses. Front Neural Circuit 7:40

  8. Nawrot A, Pistohl T, Schrader S, Hehl U, Rodriguez V, Aertsen A (2003) Embedding living neurons into simulated neural networks, in: IEEE Conference on Neural Engineering, pp. 229–232

  9. Zbrzeski A et al (2016) Bio-inspired controller on an FPGA applied to closed-loop diaphragmatic stimulation. Front Neurosci 10:275

    Article  Google Scholar 

  10. Mosbacher Y, Khoyratee F, Goldin M, Kanner S, Malakai Y, Silva M, Grassia F, Simon YB, Cortes J, Barzilai A et al (2020) Toward neuroprosthetic real-time communication from in silico to biological neuronal network via patterned optogenetic stimulation. Sci Rep 10(1):1–16

    Article  Google Scholar 

  11. Buccelli S, Bornat Y, Colombi I, Ambroise M, Martines L, Pasquale V, Bisio M, Tessadori J, Nowak P, Grassia F et al (2019) A neuromorphic prosthesis to restore communication in neuronal networks. IScience 19:402–414

    Article  Google Scholar 

  12. Berger T et al (2011) A cortical neural prosthesis for restoring and enhancing memory. J Neural Eng 8(4):046089

    Article  Google Scholar 

  13. Berger T et al (2012) A hippocampal cognitive prosthesis: multi-input, multi-output nonlinear modeling and vlsi implementation. IEEE Transact Neural Syst Rehabilitat Eng 20(2):198–211

    Article  Google Scholar 

  14. Abbott LF (1999) Lapicque’s introduction of the integrate-and-fire model neuron. Brain Res Bullet 50(5):303–304

    Article  Google Scholar 

  15. Hodgkin AL, Huxley AF (1952) A quantitative description of membrane current and its application to conduction and excitation in nerve. J Physiol 117(4):500–544

    Article  Google Scholar 

  16. Izhikevich EM (2003) Simple model of spiking neurons. IEEE Transact Neural Netw 14(6):1569–1572

    Article  MathSciNet  Google Scholar 

  17. Izhikevich EM (2004) Which model to use for cortical spiking neurons? IEEE Transact Neural Netw 15(5):1063–1070

    Article  Google Scholar 

  18. Luk W, Thomas D (2009)FPGA accelerated simulation of biologically plausible spiking neural networks, in: IEEE Symposium on field programmable custom computing machines, pp. 45–52

  19. Ambroise M, Levi T, Bornat Y, Saighi S (2013) Biorealistic spiking neural network on FPGA, in: IEEE information sciences and systems, pp. 1–6

  20. Pani D et al (2017) An FPGA platform for real-time simulation of spiking neuronal networks. Front Neurosci 11(90):1–13

    Google Scholar 

  21. Cheung K, Schultz SR, Luk W (2016) NeuroFlow: a general purpose spiking neural network simulation platform using customizable processors. Front Neurosci 9(516):1–15

    Google Scholar 

  22. Khoyratee F, Grassia F, Saïghi S, Levi T (2019) Optimized real-time biomimetic neural network on FPGA for bio-hybridization. Front Neurosci 13:377

    Article  Google Scholar 

  23. Akbarzadeh-Sherbaf K, Abdoli B, Safari S, Vahabie A-H (2018) A scalable FPGA architecture for randomly connected networks of hodgkin-huxley neurons. Front Neurosci 12:698

    Article  Google Scholar 

  24. Furber SB, Galluppi F, Temple S, Plana LA (2014) The spinnaker project. Proceed IEEE 102(5):652–665

    Article  Google Scholar 

  25. Moradi S, Qiao N, Stefanini F, Indiveri G (2017) A scalable multicore architecture with heterogeneous memory structures for dynamic neuromorphic asynchronous processors (DYNAPs). IEEE Transact Biomed Circuits Syst 12(1):106–122

    Article  Google Scholar 

  26. Qiao N, Mostafa H, Corradi F, Osswald M, Stefanini F, Sumislawska D, Indiveri G (2015) A reconfigurable on-line learning spiking neuromorphic processor comprising 256 neurons and 128k synapses. Front Neurosci 9:141

    Article  Google Scholar 

  27. Seo JS, et al. (2011) A 45 nm CMOS neuromorphic chip with a scalable architecture for learning in networks of spiking neurons, in: IEEE Custom Integrated Circuits Conference

  28. Frenkel C et al (2019) A 0.086-mm\(^2\) 12.7-pJ/SOP 64k-synapse 256-neuron online-learning digital spiking neuromorphic processor in 28-nm cmos. IEEE Transact Biomed Circuits Syst 13(1):145–158

    Google Scholar 

  29. Akopyan F et al (2015) Truenorth: Design and tool flow of a 65 mw 1 million neuron programmable neurosynaptic chip. IEEE Transact Comput-Aided Des Integrat Circuits Syst 34(10):1537–1557

    Article  Google Scholar 

  30. Davies M et al (2018) Loihi: A neuromorphic manycore processor with on-chip learning. IEEE Micro 38(1):82–99

    Article  Google Scholar 

  31. Pei J, Deng L, Song S, Zhao M, Zhang Y, Wu S, Wang G, Zou Z, Wu Z, He W et al (2019) Towards artificial general intelligence with hybrid tianjic chip architecture. Nature 572(7767):106–111

    Article  Google Scholar 

  32. Wells RB (2005) “Cortical neurons and circuits: a tutorial introduction [online]. [cit. 2008-02-05],” URL: http://www.mrc.uidaho.edu/\(^{\sim }\)rwells/techdocs/CorticalNeuronsandCircuits.pdfd

  33. Parhi KK, Wang C-Y, Brown AP (1992) Synthesis of control circuits in folded pipelined dsp architectures. IEEE J Solid-State Circuit 27(1):29–43

    Article  Google Scholar 

  34. Kocaturk M, Gulcur HO, Canbeyli R (2015) Toward building hybrid biological/in silico neural networks for motor neuroprosthetic control. Front Neurorobot 9:8

    Article  Google Scholar 

  35. Tsai D, Sawyer D, Bradd A, Yuste R, Shepard KL (2017) A very large-scale microelectrode array for cellular-resolution electrophysiology. Nat Commun 8(1):1802

    Article  Google Scholar 

  36. LeCun Y, Bottou L, Bengio Y, Haffner P (1998) Gradient-based learning applied to document recognition. Proceed IEEE 86(11):2278–2324

    Article  Google Scholar 

  37. Diehl PU, Cook M (2015) Unsupervised learning of digit recognition using spike-timing-dependent plasticity. Front Comput Neurosci 9:99

    Article  Google Scholar 

  38. Tavanaei A, Maida A (2019) Bp-stdp: Approximating backpropagation using spike timing dependent plasticity. Neurocomputing 330:39–47

    Article  Google Scholar 

  39. Neftci E, Das S, Pedroni B, Kreutz-Delgado K, Cauwenberghs G (2014) Event-driven contrastive divergence for spiking neuromorphic systems. Front Neurosci 7:272

    Article  Google Scholar 

  40. Zhao B, Ding R, Chen S, Linares-Barranco B, Tang H (2014) Feedforward categorization on aer motion events using cortex-like features in a spiking neural network. IEEE Transact Neural Netw Learn Syst 26(9):1963–1978

    Article  MathSciNet  Google Scholar 

  41. Beyeler M, Dutt ND, Krichmar JL (2013) Categorization and decision-making in a neurobiologically plausible spiking network using a stdp-like learning rule. Neural Netw 48:109–124

    Article  Google Scholar 

  42. Brader JM, Senn W, Fusi S (2007) Learning real-world stimuli in a neural network with spike-driven synaptic dynamics. Neural Comput 19(11):2881–2912

    Article  MathSciNet  MATH  Google Scholar 

  43. An S, Lee M, Park S, Yang H, So J (2020) “An ensemble of simple convolutional neural network models for mnist digit recognition,” arXiv preprint arXiv:2008.10400

  44. Davison AP, Brüderle D, Eppler JM, Kremkow J, Muller E, Pecevski D, Perrinet L, Yger P (2009) Pynn: a common interface for neuronal network simulators. Front Neuroinform 2:11

    Google Scholar 

  45. Stewart TC, Tripp B, Eliasmith C (2009) Python scripting in the nengo simulator. Front Neuroinform 3:7

    Article  Google Scholar 

  46. Hofstötter C, et al. (2005) The cerebellum chip: an analog VLSI implementation of a cerebellar model of classical conditioning, in: Advances in neural information processing systems, pp. 577–584

  47. Park J, Ha S, Yu T, Neftci E, Cauwenberghs G (2014) A 65k-neuron 73-mevents/s 22-pj/event asynchronous micro-pipelined integrate-and-fire array transceiver,in: IEEE BioCAS Proceedings, pp. 675–678

  48. Mead C (1989) Analog VLSI and neutral systems, NASA STI/Recon Tech Rep A, 90

  49. Joubert A, Belhadj B, Temam O, Héliot R (2012) Hardware spiking neurons design: Analog or digital? in: IEEE International Joint Conference on Neural Networks, pp. 1–5

  50. Stillmaker A, Xiao Z, Baas B (2011)Toward more accurate scaling estimates of cmos circuits from 180 nm to 22 nm,VLSI Computation Lab, ECE Department, University of California, Davis, Tech. Rep. ECE-VCL-2011-4, 4, p. m8

  51. Brette R, Gerstner W (2005) Adaptive exponential integrate-and-fire model as an effective description of neuronal activity. J Neurophysiol 94(5):3637–3642

    Article  Google Scholar 

  52. Naud R, Marcille N, Clopath C, Gerstner W (2008) Firing patterns in the adaptive exponential integrate-and-fire model. Biol Cybernet 99(4–5):335

    Article  MathSciNet  MATH  Google Scholar 

  53. Pal S, Gupta V, Ki WH, Islam A (2019) Design and development of memristor-based rram. IET Circuits, Dev Syst 13(4):548–557

    Article  Google Scholar 

  54. Chu M, Kim B, Park S, Hwang H, Jeon M, Lee BH, Lee B-G (2014) Neuromorphic hardware system for visual pattern recognition with memristor array and cmos neuron. IEEE Transact Ind Electron 62(4):2410–2419

    Article  Google Scholar 

  55. Shukla A, Ganguly U (2018) An on-chip trainable and the clock-less spiking neural network with 1r memristive synapses. IEEE Transact Biomed Circuits Syst 12(4):884–893

    Article  Google Scholar 

  56. Chen B, Yang H, Zhuge F, Li Y, Chang T-C, He Y-H, Yang W, Xu N, Miao X-S (2019) Optimal tuning of memristor conductance variation in spiking neural networks for online unsupervised learning. IEEE Transact Electron Dev 66(6):2844–2849

    Article  Google Scholar 

  57. Zheng N, Mazumder P (2018) Learning in memristor crossbar-based spiking neural networks through modulation of weight-dependent spike-timing-dependent plasticity. IEEE Transact Nanotechnol 17(3):520–532

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported by the Center for Neurotechnology (CNT), a National Science Foundation (NSF) Engineering Research Center (EEC-1028725) and the NSF Award #2007131.

Author information

Authors and Affiliations

  1. Department of Electrical and Computer Engineering, San Diego State University, San Diego, USA

    Daniel Valencia & Amir Alimohammad

  2. Department of Electrical and Computer Engineering, University of California, La Jolla, USA

    Daniel Valencia

Authors
  1. Daniel Valencia
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Amir Alimohammad
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Daniel Valencia.

Ethics declarations

Conflict of interest

The authors have no competing interests to declare.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Valencia, D., Alimohammad, A. A generalized hardware architecture for real-time spiking neural networks. Neural Comput & Applic 35, 17821–17835 (2023). https://doi.org/10.1007/s00521-023-08650-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-023-08650-6

Keywords

Search

Navigation