Gyuseong Kang
ABSTRACT
As a large number of neurons and synapses are needed in spike neural network (SNN) design, emerging devices have been employed to implement synapses and neurons. In this paper, we present a stochastic multi-bit spin orbit torque (SOT) memory based synapse, where only one SOT device is switched for potentiation and depression using modified Gray code. The modified Gray code based approach needs only N devices to represent 2N levels of synapse weights. Early read termination scheme is also adopted to reduce the power consumption of training process by turning off less associated neurons and its ADCs. For MNIST dataset, with comparable classification accuracy, the proposed SNN architecture using 3-bit synapse achieves 68.7% reduction of ADC overhead compared to the conventional 8-level synapse.
- J. Schmidhuber, "Deep learning in neural networks: An overview," Neural Netw., vol. 61, pp. 85--117, Jan. 2015. Google Scholar
Digital Library
- P. U. Diehl and M. Cook, "Unsupervised learning of digit recognition using spike-timing-dependent plasticity," Front. Comput. Neurosci., vol. 9, 2015.Google Scholar
- Q. Wang, Y. Li, B. Shao, S. Dey, and P. Li, "Energy Efficient Parallel Neuromorphic Architectures with Approximate Arithmetic on FPGA," Neurocomput., vol. 221, no. C, pp. 146--158, Jan. 2017. Google ScholarDigital Library
- J. s Seo et al., "A 45nm CMOS neuromorphic chip with a scalable architecture for learning in networks of spiking neurons," in 2011 IEEE Custom Integrated Circuits Conference (CICC), 2011, pp. 1--4.Google Scholar
- Q. Wang, Y. Kim, and P. Li, "Neuromorphic Processors with Memristive Synapses: Synaptic Interface and Architectural Exploration," J Emerg Technol Comput Syst, vol. 12, no. 4, p. 35:1--35:22, May 2016. Google ScholarDigital Library
- A. F. Vincent et al., "Spin-Transfer Torque Magnetic Memory as a Stochastic Memristive Synapse for Neuromorphic Systems," IEEE Trans. Biomed. Circuits Syst., vol. 9, no. 2, pp. 166--174, Apr. 2015.Google ScholarCross Ref
- G. Srinivasan, A. Sengupta, and K. Roy, "Magnetic Tunnel Junction Based Long-Term Short-Term Stochastic Synapse for a Spiking Neural Network with On-Chip STDP Learning," Sci. Rep., vol. 6, p. srep29545, Jul. 2016.Google ScholarCross Ref
- A. Sengupta, A. Banerjee, and K. Roy, "Hybrid Spintronic-CMOS Spiking Neural Network with On-Chip Learning: Devices, Circuits, and Systems," Phys. Rev. Appl., vol. 6, no. 6, p. 064003, Dec. 2016.Google ScholarCross Ref
- J. Bill and R. Legenstein, "A compound memristive synapse model for statistical learning through STDP in spiking neural networks," Front. Neurosci., vol. 8, 2014.Google Scholar
- D. Zhang, L. Zeng, Y. Zhang, W. Zhao, and J. O. Klein, "Stochastic spintronic device based synapses and spiking neurons for neuromorphic computation," in 2016 IEEE/ACM International Symposium on Nanoscale Architectures (NANOARCH), 2016, pp. 173--178.Google Scholar
- N. Burkitt, "A Review of the Integrate-and-fire Neuron Model: I. Homogeneous Synaptic Input," Biol Cybern, vol. 95, no. 1, pp. 1--19, Jun. 2006. Google ScholarDigital Library
- B. Rajendran and F. Alibart, "Neuromorphic Computing Based on Emerging Memory Technologies," IEEE J. Emerg. Sel. Top. Circuits Syst., vol. 6, no. 2, pp. 198--211, Jun. 2016.Google ScholarCross Ref
- L. Liu, C.-F. Pai, Y. Li, H. W. Tseng, D. C. Ralph, and R. A. Buhrman, "Spin-Torque Switching with the Giant Spin Hall Effect of Tantalum," Science, vol. 336, no. 6081, pp. 555--558, May 2012.Google ScholarCross Ref
- K.-S. Lee, S.-W. Lee, B.-C. Min, and K.-J. Lee, "Thermally activated switching of perpendicular magnet by spin-orbit spin torque," Appl. Phys. Lett., vol. 104, no. 7, p. 072413, Feb. 2014.Google ScholarCross Ref
- T. Branco and K. Staras, "The probability of neurotransmitter release: variability and feedback control at single synapses," Nat. Rev. Neurosci., vol. 10, no. 5, pp. 373--383, 2009.Google ScholarCross Ref
- B. B. Averbeck, P. E. Latham, and A. Pouget, "Neural correlations, population coding and computation," Nat. Rev. Neurosci., vol. 7, no. 5, pp. 358--366, May 2006.Google ScholarCross Ref
- Y. Lecun, L. Bottou, Y. Bengio, and P. Haffner, "Gradient-based learning applied to document recognition," Proc. IEEE, vol. 86, no. 11, pp. 2278--2324, Nov. 1998.Google ScholarCross Ref
- Z. Huang and P. Zhong, "An architectural power estimator for analog-to-digital converters," in IEEE International Conference on Computer Design: VLSI in Computers and Processors, 2004. ICCD 2004. Proceedings., 2004, pp. 397--400. Google ScholarDigital Library
- J. Kim, A. Chen, B. Behin-Aein, S. Kumar, J. P. Wang, and C. H. Kim, "A technology-agnostic MTJ SPICE model with user-defined dimensions for STT-MRAM scalability studies," in 2015 IEEE Custom Integrated Circuits Conference (CICC), 2015, pp. 1--4.Google Scholar
Index Terms
- Spin Orbit Torque Device based Stochastic Multi-bit Synapses for On-chip STDP Learning
Recommendations
GABA inhibition modulates NMDA-R mediated spike timing dependent plasticity (STDP) in a biophysical model
Spike timing dependent plasticity (STDP) has been demonstrated in various neural systems of many animals. It has been shown that STDP depends on the target and the location of the synapse and is dynamically regulated by the activity of adjacent synapses,...
A calcium-influx-dependent plasticity model exhibiting multiple STDP curves
AbstractHebbian plasticity means that if the firing of two neurons is correlated, then their connection is strengthened. Conversely, uncorrelated firing causes a decrease in synaptic strength. Spike-timing-dependent plasticity (STDP) represents one ...
How Bursts Shape the STDP Curve in the Presence/Absence of GABAergic Inhibition
ICANN '09: Proceedings of the 19th International Conference on Artificial Neural Networks: Part IIt has been known for some time that the synapses of the CA1 pyramidal cells are surprisingly unreliable at signalling the arrival of single spikes to the postsynaptic neuron [2]. On the other hand, bursts of spikes are reliably signalled, because ...
Comments