Spike-time dependent plasticity rule in memristor models for circuits design

Spike Time-Dependent Plasticity (STDP) represents an essential learning rule found in biological synapses which is recommended for replication in neuromorphic electronic systems. This rule is defined as a process of updating synaptic weight that depends on the time difference between the pre-and post-synaptic spikes. It is well known that pre-synaptic activity preceding post-synaptic activity may induce long term potentiation (LTP) whereas the reverse case induces long term depression (LTD). Memristors, which are two-terminal memory devices, are excellent candidates to implement such a mechanism due to their distinctive characteristics. In this article, we analyze the fundamental characteristics of three of the most known memristor models


Introduction
Recently, neuromorphic technology has revealed that brainbased computer systems are capable of processing large amounts of data by using low-power operations [1].However, they consist basically of similar structures as the human brain, composed of neurons linked by multiple synapses.These systems able to follow a basic weight update rule across synapses, just like humans remember information.This rule was called spike-timing dependent plasticity (STDP) [2].It demonstrates how the strength of synaptic connections is modulated [2,3].Thus, the synaptic connection is strengthened when the presynaptic neuron fires before the postsynaptic neuron, which is known as long-term potentiation (LTP), and the connection is weakened when the firing order is reversed, which is called long-term depression (LTD).
Since the discovery of STDP rule in biological synapses, researchers have been attracted with the concept of modifying synaptic weight using this rule in bio-inspired electronic synapses. 1Microelectronics and Instrumentation Lab, University of Monastir, Monastir, Tunisia 2 Centre for Research on Microelectronics & Nanotechnology Sousse Technology Park, Sousse, Tunisia However, the circuit-oriented approach is complex in that the variable "synaptic weight" generally must be stored as an electric charge in a capacitor or even numerically in a neuromorphic integrated circuit.This adds complexity to the circuit and increases energy and power consumption.Consequently, non-volatile resistive memories, namely memristors are currently employed in neuromorphic computation systems in order to reproduce the synaptic weight update by suitably changing their internal state [4][5][6][7][8][9].In particular, memristors have been widely studied and developed to mimic synaptic functions such as, LTP, LTD and STDP [4, [10][11][12].Several complex tasks were performed efficiently using the STDP learning rule based on memristors such as unsupervised learning [12], data classification [13] and pattern recognition [14].
Memristor component was first introduced by L. Chua in 1971 as the fourth circuit component, alongside the three fundamental passive components, which are the resistor, the capacitor, and the inductor [15].Memristors are widely used as synaptic devices due to their adjustable resistance [16,17], low-power operation, nano-scale physical structure, and highdensity integration [18,19].
In the literature, several research teams have fabricated memristive devices to emulate synaptic functions using a variety of materials [21][22][23][24].The large variety of the memristor devices structure and materials composition has led to the development of various memristor modeling techniques [18,20,[25][26][27][28][29].This variety of models allows to choose an acceptable compromise between accuracy and simplicity.
Among these models, we will focus on three different models: the linear ion drift model (HP) model [20], the Voltage ThrEshold Adaptive Memristor (VTEAM) model [29], and the Enhanced Generalized Memristor (EGM) model [18].We simulated the I-V curve of these models in order to fit the experimental data of the Ta2O5-RRAM device [23].Next, we implement each model to determine both LTP and LTD functions, and then we validate the STDP learning algorithm.We present a detailed comparison of these models, in order to explore the most promising memristor model to be used as a synapse component.
This article consists of the following sections: Section 2 provides a brief discussion of the memristor switching characteristics.Section 3 introduce the memristor fundamentals including mathematical definition and working principle of HP model, VTEAM model, and the EGM model.Then, we present the I-V curves of these models compared to the experimental data.Section 4 discusses our simulation results obtained from the conductance change in order to validate LTP, LTD and the STDP learning algorithm.Section 5 contain discussion of our results.Finally, section 6 concludes this paper.

An overview of the memristor switching device
A solid-state memistor device is a thin-film nanoscale TiO2 device consisting of a doped region with TiO2-x oxygen gaps and an undoped region of TiO2.These two regions are sandwiched together across two platinum electrodes, as depicted in Fig. 1 (a).The resistance change is induced by migration of the oxygen holes in the TiO2-x region, produced by the application of a voltage across the memristor.Therefore, the oxygen holes migration depends on the sign of the applied voltage.
Table 1 Comparative analysis of the memristor models Applying a positive voltage to the top electrode of the device switches the resistance states from high resistance state (HRS) to low resistance state (LRS), and this switching is referred as SET state.Alternatively, when a negative voltage is applied, it switches from the LRS state to the HRS state, and this is referred as RESET state.Because of the poor oxygen mobility of the vacancies, they may be able to stand for a long time after switching off the power supply to the device.This means that if no power is provided to the device, then the memristors' dynamic resistance will remain constant.This shows that the memristor device has the ability to operate as a non-volatile memory.Moreover, there is a major property that sets memristor devices apart from all other electronic devices, namely the I-V characteristic known as the pinched hysteresis loop, which is illustrated in Fig. 1 (b).This distinctive shape arises from the non-linear relationship between the voltage and the current passing across the memristor device.Besides, both negative and positive voltage pulses can affect the conductance of the device.This is similar to a biological synapse whose synaptic plasticity increases or decreases according to the applied action potential.

Memristor Models
We provide in Table 1 a comparative study of the memristor models, including their mathematical characteristics and their implemented parameters, which we will be implementing and testing in the next section.

No
Yes Yes

Simulation parameters
Roff=140kΩ; Ron=1kΩ; Rinit=36kΩ; D=1nm; uv=1e -14 m 2 s -1 v - We simulate the studied models with the Spectre simulator in Virtuoso® Custom IC Design in order to fit the I-V results of an experimental Ta2O5-RRAM memristor device and then to validate their capabilities to emulate synaptic functions.

• Simulation of the Linear Ion Drift model
The linear ion drift model has been developed based on the first fabricated HP's structure of the memristor device [19].The above memristor behavior is mathematically described using equations in Table 1.The voltage V and current I curves of the HP model are shown in Fig. 2(a) in which we performed a sinusoidal excitation of 2 V magnitude and 1 kHz frequency.Fig. 2 (b) gives a comparative analysis between the experimental data of the Ta2O5-RRAM device [23] and the simulated results of the linear ion drift model.

• Simulation of the Voltage Threshold Adaptive
Memristor model The VTEAM model has been proposed by Kvatinsky et al. [29].It is an extension of the TEAM model which is based on current threshold [28].VTEAM is a voltage threshold-type model developed to fit different experimental results.Fig. 3 (a

• Simulation of the Enhanced Generalized
Memristor model The Enhanced Generalized Memristor model has been proposed in [27] and then explored in [18].A sub-circuit of the memristor model is implemented as a simple netlist, which consists of multiple elements in SPICE [27].Then, The EGM model undergoes some modifications in order to be implemented in Verilog-A [18].The model was correlated to several memristor devices.We can notice that the I-V curves of the HP model and the VTEAM model indicate almost linear characteristics everywhere except the boundary, with a symmetric switching behavior for the HP model and an asymmetric behavior for the VTEAM model.Thus, the HP and VTEAM models cannot correlate well the behavior of the physical device.While the obtained I-V characteristics of the EGM model with a symmetric curve agree well with the experimental data of the Ta2O5-RRAM memristor model [23].

Long term potentiation/ depression
In neural networks, the synaptic connection weight can be modulated through learning protocols, LTP, LTD and STDP.LTP indicates that certain patterns of synaptic activity can cause a lasting increase in synaptic strength, while other patterns of synaptic activity can cause a continuous decrease in synaptic strength, LTD.
To test LTP and LTD mechanisms, we applied a hundred consecutive pulses for different amplitudes values and a fixed pulses duration is the order of 1 µs.Fig. 5 explores the conductance change curves for the three memristor models showing the LTP and LTD cases when a sweeping voltage was applied.
We specified two amplitude ranges for each model, the first one is used to maintain LTP and the second one is used to maintain LTD case.
• The pulse amplitude for the HP model is variated between {-1 V, -4 V} for LTD and {1 V, 4 V} for LTP.Fig. 5 (a) and (b) show a gradual conductance decrease and increase for the HP model according to the input voltage bias.• For the VTEAM model the pulse amplitude is variated between {-1.5 V, -3.5 V} for the LTP case and {1.5 V, 3.5 V} for the LTD case.Fig. 5 (c) and (d) show a gradual conductance decrease when applying positive voltage and no conductance increase for negative input voltage for the VTEAM model.
• For the EGM model, we choose a pulse amplitude variated between {-1.5 V, -2.2 V} for LTD and {1.8 V, 3 V} for LTP.Fig. 5 (e) and (f) show a gradual conductance decrease and increase according to the input voltage bias for the EGM model.These previous results of the I-V characteristic and the conductance gradual change can prove that the EGM is the most effective and suitable model to be used as a synapse component.To further demonstrate this, we will test the capacity of each model to validate the STDP mechanism.

Validation of the spike-time-dependent plasticity
The STDP is a learning mechanism based on mammalian brain synapses [30].Several experimental investigations of this mechanism have been studied in synapses based memristors architectures [10,[21][22][23][24].The STDP modify the synaptic weight depending on the activities of the pre-synaptic and postsynaptic neurons [4].The pre-neuron and post-neuron spikes reach the synapse in opposite directions.In this case, the change in synaptic weight is a function of the relative timing between the two neuron spikes, labeled Δt (Δt = tpost − tpre), where tpre and tpost are the arrival timing of pre-and post-spikes, respectively.
• When Δt > 0, the synapse exhibits long-term potentiation (LTP).• When Δt < 0, the synapse exhibits a long-term depression (LTD).The combination of the synaptic functions, LTP and LTD, subsequently confirm the STDP learning method.
In order to test and validate the STDP function using the memristor models detailed in our work, we employ a spike pair protocol [31,32] in which the lower and upper memristor electrodes are assigned to the presynaptic and postsynaptic inputs respectively, as shown in Fig. 6.We considered the same voltage pulses for the three simulated memristor models.The maximum voltage applied to the memristor could be high enough to change the resistance state of the memristor which is fixed as 2 V.In the STDP demonstration, we consider the device conductance as the synaptic weight.
Then its variability is corresponding to a change in the synaptic weight.The evolution of memristance for each memristor model during the application of positive and negative ∆t was given in Fig. 7.
• For the EGM model, the memristance change was given in Fig. 7(c).If ∆t > 0, the memristance decrease from 36,6 kΩ to 17,3 kΩ, so the conductance presents LTP.Then, if ∆t < 0, the memristance increases from 17,3 kΩ to 36,6 kΩ, so the conductance indicate an LTD.In addition, in order to implement the STDP learning rule, we use the function ∆w (Δt) described by Eq. 1, which is defined as a modification of the synaptic weight for an isolated pair of spikes [33,34].As given in Fig. 7, the amount of potentiation and depression will be determined as a function of the time difference between pre-synaptic and post-synaptic spikes.
Where w -and w + indicate the minimum and maximum modifications of the synaptic weight with respect to the initial state during potentiation and depression functions, respectively.τ -and τ + are the time windows that define the synaptic weight update rate for LTD and LTP cases, respectively.Moreover, we use the previous results illustrated in Fig. 7 to determine the value of the ∆w function for each model.( Using our previous results when testing the ability of the three simulated models to mimic the LTP and LTD, we can conclude that, the conductance programmability of the HP model and the EGM model can demonstrate both LTP and LTD functions, which leads to the validation of the STDP learning rule, as shown in Fig. 8 (a) and (c).Hence, these results are similar to biological synaptic systems [2].While the VTEAM model can only support LTD.Then, its STDP curve can show conformity with biological systems only when ∆ < 0, as shown in Fig. 8 (b).So, we can conclude that we cannot apply the VTEAM model to implement neuromorphic circuit using the STDP learning method.

Discussion
To design neuromorphic circuits based on memristors, the implemented models must be sufficiently effective as compared to the behavior of physical devices, they should support LTP and LTD functions to validate STDP learning mechanism.Also, it is desirable for the memristor model to be simple, flexible and general so it can be adjusted to fit different physical memristive devices.In this work, we have studied the most popular memristor models, HP model [20], VTEAM model [29] and EGM model [18], in order to explore the most appropriate model to operate as a synapse component for neuromorphic systems.Table 1 gives a comparison of these models, we note that the first model [20] has very limited parameters and does not include threshold voltage, it contains only five parameters which make it less flexible to be tuned and to fit multiple memristor device.It's I-V curve show that the HP model does not correctly account for nonlinear behaviour of the experimental memristor device.However, our simulation results prove that this model can support both LTP and LTD functions and we successfully validate the STDP mechanism.The VTEAM model [29] is a voltage-controlled model, it is a general model, which can fit different memristor devices, but the application of a positive voltage allows it to decrease its conductance while the application of a negative voltage allows it to remains unchanged to its initial state.So, this model can only support LTD function.This affect the capability of the VTEAM model to validate the biological STDP rule, which is illustrated in our results in Fig. 8b.The EGM model [18] is flexible, it is a voltage-controlled model, and general, it can fit various memristive devices.The simulated I-V characteristics of this model with a symmetric curve corresponds well with the experimental data of the Ta2O5-RRAM memristor device.Furthermore, the gradual conductance decrease and increase of the EGM model show that this model can exhibit the two synaptic functions LTP and LTD.This prove that we can implement and validate the STDP learning rule.Fig. 9 shows the relative variation curves of the STDP of the three implemented memristor models.We prove from our previous results that the first model can validate the biological STDP mechanism but it has various convergence problems and it is less flexible.The second model is general and flexible but it cannot support LTP function, so in our opinion, the VTEAM model cannot implement the STDP rule in a good way such as biological synapses.The EGM model can show several characteristics which make it a good candidate to act as synapse component.As shown in Fig. 9, we prove that this model can implement the STDP function in a similar way to biological synaptic systems [2].These characteristics make the EGM model the most effective and acceptable model that can be used to validate the synaptic functions and then to successfully perform neuromorphic circuits.

Conclusion
The memristive synapses play a key role in performing neuromorphic circuits, their plasticity is obtained by a variation in memristance values based on the temporal coincidences of the pre-and post-synaptic points.In order to implement such systems, an effective memristor model is needed, in this article, we analyze the fundamental characteristics of three of the most known memristor models, the linear ion drift model (HP), the Voltage ThrEshold Adaptive Memristor (VTEAM) model and the Enhanced Generalized Memristor (EGM) model.We tested their capability to imitate synaptic functions.We compared their I-V characteristics to an experimental result.Then we examined their gradual conductance modulation to prove the LTP and LTD functions which was achieved by modulating the polarity and amplitude of the applied pulses.Based on the conductance modulation of each model, we validate the STDP learning rule.From our simulation results, we note that the obtained I-V characteristics of the EGM model corresponds well with the experimental data of the Ta2O5-RRAM memristor device.It can demonstrate both LTP and LTD functions, which leads to the validation of the STDP learning rule.Hence, the EGM model is the most suitable model to mimic synaptic functions and then to be used in neuromorphic circuits.

Fig. 2
Fig. 2 (a) Current and voltage curves of the simulated HP model versus time, (b) The experimental I-V curve of Ta2O5-RRAM memristor device compared with the simulated results of the HP model.
) gives the current I and voltage V curves of the simulated VTEAM model.Fig.3 (b) shows the I-V characteristics of the VTEAM model compared with the Ta2O5-RRAM memristor device [23].

Fig. 3
Fig. 3 (a) Current and voltage curves of the simulated VTEAM model versus time, (b) The experimental I-V curve of Ta2O5-RRAM memristor device compared with the simulated results of the VTEAM model Fig.4 (a) shows the current I and voltage V curves of the EGM model.Fig.4 (b) shows both the experimental I-V curve of the Ta2O5-RRAM device [23] and the simulated results of the EGM model.

Fig. 4
Fig. 4 (a) Current and voltage curves of the EGM model versus time, (b) The experimental I-V curve of Ta2O5-RRAM memristor device compared with the simulated results of the EGM model.

Fig. 5
Fig. 5 Curves of the conductance change for a voltage sweeping.Potentiations and depressions: (a), (b) for HP model, (c), (d) for the VTEAM model and (e), (f) for the EGM model.

Fig. 8 Fig. 7
Fig. 7 Simulation results of STDP test circuit given pre voltage spikes, post voltage spikes and the memristance evolution: (a) HP model, (b) VTEAM model (c) EGM model.

Fig. 9 .
Fig. 9.The relative STDP curves for HP model, VTEAM model and the EGM model.