Real-time pattern recognition implementation on FPGA in multi-SNNs

By mimicking or being inspired by the nervous system, Neuromorphic systems are designed to realize robust and power-efficient information processing by highly parallel architecture. Spike timing dependent plasticity (STDP) is a common method for training Spiking Neural Networks (SNNs) for pattern recognition. Here, we present a real-time STDP implementation on FPGA in SNN using digital spiking silicon neuron (DSSN) model. Equipped with Ethernet Interface, FPGA allows online configuration as well as data input and output all in real-time. We show that this STDP implementation can achieve pattern recognition task and the connection between multi-SNNs enlarge the scale of networks and application.


Introduction
Neuromorphic systems are designed by mimicking or being inspired by the nervous system, which aims to realize robust, autonomous, and power-efficient information. There are three common methods to realize the neuromorphic circuits, which are software 1 , analog hardware 2,3 and digital hardware 4,5 . Software-based cannot achieve real-time processing. Compared to analog circuits, digital implementations generally consume higher power but are more scalable because they are less susceptible to noise and fabrication mismatch. Another advantages of FPGA devices are configurability, portability, and low-cost. Silicon neuronal network (SNN) is a neuromorphic circuit that reproduces the electrophysiological activities in the nervous system focusing on the spiking dynamics in the neuronal cells and their transmission via the synapses. Their application includes bio-inspired information processing such as pattern recognition 6,7 and associative memory 8 as well as neuro-prosthetic devices 9 ， 10 .
Digital SNNs are expected to achieve a very large-scale network comparable to the human brain in the future exploiting the scalability of the digital circuits. In these 5 years, several very large-scale SNNs with one million neurons were developed 11 . Spike-timing-dependent plasticity (STDP) is a wellknown rule for updating the synaptic efficacy in SNNs, which uses only local information. Many biological experiments found evidence for STDP process in the synapses 12,13 . In this paper, we report an implementation of digital SNN with online STDP learning on FPGA. The model and implementation of our SNN are explained in the next section and section 3, respectively. Then results and conclusion follows.

DSSN model
Neuronal models need to be chosen taking into account the balance between the reproducibility of neuronal activity and computational efficiency. Integrate-and-fire (I&F)-based models are able to be implemented by compact hardware, but they lack reproducibility of complex neuronal dynamics. Ionic-conductance models have high-ability of reproducing neuronal activities, but cost massive computational resources. The DSSN model is a qualitative neural model 14 , which was designed for efficient implementation in digital circuits. The simplest version of DSSN model supports the Class I and II cells in Hodgkin's Classification 15 .The differential equations of DSSN model are as follows.
Here represents the membrane potential, is a variable that reflects the activities of hyperpolarizing ionic channels. , , and ( = , and = , ) are parameters. The Parameter 0 is a bias constant and represents the input stimulus. The only nonlinearity in the DSSN model is a quadratic function. Thus, solving this model using Euler's method needs one multiplication operation per step if the parameters are carefully selected 8,16 . As multiplication operation requires relatively large resources in digital circuits, the DSSN model is suitable for digital silicon neuronal networks.

Synaptic model
Postsynaptic current (PSC) is a current inserted to the postsynaptic cell that induces temporal change in its membrane potential. The PSCs generated by a pulse stimulus to chemical synapses are able to be modeled by the alpha function with double-exponential generalization 17 . In our network model, the PSP model was simplified as follows. = − .
Here represents PSC generated by a chemical synaptic transmission. It is reset to , the connection strength of the synapse, when the membrane potential of the presynaptic neuron exceeds 0. The initial value of is 6x10 -10 . The PSCs in the synaptic connections to a postsynaptic are summed up as follows. = ∑ .
Here i represents index of presynaptic neurons.

STDP algorithm and architecture
Synaptic plasticity means the magnitude of synaptic efficacy to change in response to the activities of pre and postsynaptic neurons. The STDP is a biological rule that adjusts the strength of synaptic connections ( ) based on the relative timing of the spikes in a pre and postsynaptic neurons. Recently it Fig. 1. Overall Architecture of STDP learning. Fig. 2. Stimulation of a DSSN neuron by a constant input current (0.308).
has been shown how STDP rule play a key role by detecting repeating patterns and generating selective response to them 18 . The STDP rule is a most common form of learning rules used in SNNs. Here is the standard exponential STDP equations.
= + ∆ , Here ∆ is the modification of the synaptic weight.
When a post-synaptic spike arises after a pre-synaptic spike ( ≤ ), the connection is reinforced (long-term potentiation (LTP), ∆ > 0), whereas in the opposite case it is weakened (long-term depression (LTD)).

Implementation
The overall architecture of our SNN is shown in Fig.1.The DSSN, STDP, and PSC blocks were implemented on a FGPA chip. In this chip, single postsynaptic neuron is connected with 500 input afferent neurons. Stimulus spike trains were generated by Brian2 on PC and sent to FPGA via Ethernet connection which are explained in 3.4 and 3.5.

Implementation of DSSN
The DSSN model's differential equations are solved by Euler's method and the value of dt is 0.1s. The solver circuit has time division multiplexing (TDM) and pipelined architecture.

Implementation of PSC
As introduced in 2.2, we use a simplified exponential decay to approximate the PSCs. The connection between a postsynaptic neuron and the 500 pre-synaptic neurons was calculated efficiently by using TDM and two-stage pipeline as in the DSSN solver circuit. Operations for the 500 sets of PSCs were finished in 1000 clock cycles, by using single multiplier and single adder. The time step for the summation of PSCs is within 0.1ms if the system clock is faster than 100MHz.

Implementation of STDP learning
Based on the STDP rule, each presynaptic (postsynaptic) spike, induces an LTD (LTP) where the synaptic weight is updated according to Eq. (7). An example waveform of PSC is shown in Fig.3, where the solid and dotted red curve represent the presynaptic and postsynaptic spikes, as well as the black curve is the strength of synaptic connection. For simulating 500 afferents, 500 clock cycles were consumed. Calculation of exponential function and update of synaptic efficacy are executed by TDM.

Ethernet on FPGA
Ethernet is a computer networking technology commonly used in local area networks (LAN). Devices equipped with Ethernet interface are connectable to LAN as well as the internet by supporting common communication protocol, for instance, TCP/IP and UDP. For neuronal network implementation on FPGAs, transmission of the spikes is always an issue particularly when the number of neurons is large. By implementing Ethernet interface and a full hardware protocol stack including IP, UDP and ARP protocol, our SNN on FPGA is capable of receiving input spike patterns from a PC by Ethernet connection and sending report frame back to the PC for monitoring the SNN working status at any time.
The configuration and parameter setting are also possible by the Ethernet connection, which contribute to take advantage of FPGA in flexibility and portability.

Pattern recognition
Spatiotemporal spike pattern recognition tasks proposed by Masquelier 19 were performed on our network. The stimulus pattern required in them were generated by superimposing a spatiotemporal spike pattern at many time points on a background random spike pattern. The former spike patterns (ones to be detected) were generated by PoissonGroup Function in Brian2 (a Python library) on PCs. The background random spike patterns were generated using the same library. The stimulus patterns were stored in files and sent from the PC to the FPGA via the Ethernet connection. An example of learning process is shown in Fig.6. The postsynaptic Fig.3. Waveform of STDP method implementation.
neuron started to detect the superimposed patterns at 11s.Our circuit performed this task in real-time. FPGA's system clock was 100MHz.

Conclusion
In this paper, a SNN on an FPGA with STDP learning capability was reported. Stimulus spikes were conveyed in real-time to the FPGA chip via Ethernet connection. It was proved that our SNN is capable of spatiotemporal spike pattern recognition in real-time.
In this work, the input patterns were generated on a PC. Perspectives are to generate them in real-time with another SNN. It is also planned to develop an FPGA-FPGA connection bus to expand the scalability of the FPGA-based SNNs, as the scale of the network on an FPGA chip is limited by the amount of on-chip memory used for storing the strength of synaptic connections. An SNN platform with high scalability is advantageous compared to software simulation from the viewpoint of real-time operation and power consumption.