An organic synaptic circuit: toward flexible and biocompatible organic neuromorphic processing

In the nervous system synapses play a critical role in computation. In neuromorphic systems, biologically inspired hardware implementations of spiking neural networks, electronic synaptic circuits pass signals between silicon neurons by integrating pre-synaptic voltage pulses and converting them into post-synaptic currents, which are scaled by the synaptic weight parameter. The overwhelming majority of neuromorphic systems are implemented using inorganic, mainly silicon, technology. As such, they are physically rigid, require expensive fabrication equipment and high fabrication temperatures, are limited to small-area fabrication, and are difficult to interface with biological tissue. Organic electronics are based on electronic properties of carbon-based molecules and polymers and offer benefits including physical flexibility, low cost, low temperature, and large-area fabrication, as well as biocompatibility, all unavailable to inorganic electronics. Here, we demonstrate an organic differential-pair integrator synaptic circuit, a biologically realistic synapse model, implemented using physically flexible complementary organic electronics. The synapse is shown to convert input voltage spikes into output current traces with biologically realistic time scales. We characterize circuit’s responses based on various synaptic parameters, including gain and weighting voltages, time-constant, synaptic capacitance, and circuit response due to inputs of different frequencies. Time constants comparable to those of biological synapses and the neurons are critical in processing real-world sensory signals such as speech, or bio-signals measured from the body. For processing even slower signals, e.g., on behavioral time scales, we demonstrate time constants in excess of two seconds, while biologically plausible time constants are achieved by deploying smaller synaptic capacitors. We measure the circuit synaptic response to input voltage spikes and present the circuit response properties using custom-made circuit simulations, which are in good agreement with the measured behavior.


Introduction
The neuromorphic engineering field was pioneered by Carver Mead in the early 90s [1]. Neuromorphic electronic systems aim to apply biologically inspired principles to develop neural computing systems with applications including sensory signal perception and processing, autonomous robotics, or brain-machine interfaces [2,3]. Recently, the term 'neuromorphic' has been used to refer to artificial intelligence sensory processing systems emulated in hardware with full custom spiking neural network chips, rather than being simulated in software using conventional computers and neural network software simulation environments, such as Emergent or Matlab [4,5].
Neuromorphic systems are based on densely interconnected individual units called neurons, consisting of multiple synapses and a single soma [6]. A synapse receives the incoming voltage pulse and weights, or scales, the high operating voltage and current of the circuit that makes any biological interface impossible, the nature, or the shape, of the electrical signals of a biological synapse is compatible with the output of our circuit, and could be interfaced using additional electrical circuits to scale it up or down [51,52]. We demonstrate that the time constant of our organic spiking synapses can reach in excess of two seconds. We also demonstrate synaptic response due to biologically realistic input voltage spikes. Finally, using custom-made circuit simulation based on transconductances of p-and n-type organic transistors, we show that the fabricated synaptic circuit is in good agreement with simulation-predicted behavior.

Materials and equipment
The flexible substrates, 25 μm thick polyimide (PI) films, were purchased from DuPont. The metals-chromium (Cr), gold (Au), and silver (Ag)-were obtained from Kurt J. Lesker Company (KJLC). Parylene diX-SR was given by Specialty Coating Systems (SCS), Inc. and deposited using a SCS LabCoater 3 (PDS 2010) employing the chemical vapor deposition (CVD) process. N,N -bis(n-octyl)-x:y, dicyanoperylene-3, 4:9, 10-bis(dicarboximide)(PDI8-CN2, also referred to as N1200), and Dinaphtho[2,3-b:2 ,3 -f]thieno [3,2-b]thiophene (DNTT) were deployed as the n-and p-type organic semiconductors and obtained from Polyera and Sigma-Aldrich, respectively. The source, drain, gate, and active layers were thermally deposited using a NANO 36 thermal-evaporation thin-film deposition system by KJLC. The I-V characterizations were performed using an HP 4155A semiconductor parameter analyzer. A reactive ion etching (RIE) process was deployed to clean the substrate using a Glow Research RIE system. A Thorlabs AMP100 transimpedance amplifier and a National Instrument USB-6343 data acquisition card collected the high-frequency signals with control via National Instrument LabVIEW.

Fabrication process
The synaptic circuit mainly consists of organic field-effect transistors (OFETs) with top-contact bottom-gate structures. The OFETs are the only elements implemented on the chip to expedite the fabrication process and decrease the procedure's complexity. Figure 1 demonstrates the structure of n-and p-type OFETs (n-and p-OFETs) with photographs of the flexible chip and both types of OFETs. The channel length and width are 100 μm and 1000 μm for both types of devices, respectively. The fabrication starts with cleaning the substrate through 10 min of sonication in isopropanol, followed by the RIE cleaning process for three minutes. A sandwich layer of 3 nm Cr and 30 nm Ag as the gate electrode is thermally deposited at the rate of 0.1 Ås −1 and 1.5 Ås −1 , respectively. Parylene dix-SR is grown using a CVD process as the gate dielectric, resulting in a 200 nm thin film. The active layers, DNTT and N1200, are deposited at the rate of 0.08 Ås −1 and substrate temperatures of 60 • C and 25 • C, respectively. Via holes are produced through mechanical removal of the dielectric film. The source and drain electrodes are obtained by deposition of a 30 nm thick layer of Au at the rate of 1.2 Ås −1 . Finally, the 30 nm Au tracks between OFETs are thermally deposited with the same process as the source and drain electrodes. Shadow masking defines the geometry of electrodes, tracks, and active layers.

Characterization of organic transistors
respectively. The OFF current of n-OFET (4 × 10 −9 A) is larger compared with p-OFET (2 × 10 −10 A), adversely affecting the operational regime of the synaptic circuit, as described later in detail.

DPI synaptic circuit
Synapses are vital for artificial and biological pulse-based neurons to transfer signals and the learning process [53]. Numerous analog synaptic circuits have been proposed to mimic the spatiotemporal behavior of a biological synapse. However, the temporal behavior of biological synapses has often been neglected [35]. Indiveri et al presented a current-mode synaptic circuit, including a DPI that allows the circuit to achieve the functionality of log-domain first-order low-pass filters. Additionally, the circuit has the advantage of tunable gain (V g ) independently from the time constant. The tunable gain acts globally and scales the weights of a cluster of DPI synapses afferent onto one post-synaptic neuron to implement homeostatic plasticity stabilizing mechanisms. The gain voltage forms an ensemble of strategies to control the overall stability of the network [54]. The DPI synapse offers exponential dynamics of pre-and post-synaptic currents of real synapses. More importantly, the circuit can obtain a plausible time constant with a relatively small capacitor [10,34,55]. Figure 4 demonstrates the schematic of the DPI synapse with the analogous biological synapse beside a photograph of the fabricated organic circuit. The DPI synaptic circuit operates in the subthreshold regime and is comprised of four n-OFETs, two p-OFETs, and a single capacitor. Assuming OFETs' subthreshold operation, DPI synaptic circuit behavior can be described as follows: the input voltage signal activates the n-type M pre and allows the synaptic capacitor (C syn ) to discharge, V syn decreases with a rate set by I in − I τ , and I syn increases exponentially. M τ recharges C syn linearly to V Source by deactivating the input voltage signal, and I syn decreases back to the leakage current. M τ and M W are subthreshold current sources in the circuit. The maximum amplitude of I syn is set by three parameters: V W , V g , and V τ . The weighing voltage (V W ) can be adjusted locally to implement learning and plasticity, and the gain voltage (V g ) is an extra degree of freedom to implement global plasticity mechanisms.

Time constant
The temporal dynamics of a pulse-based (spiking) neural network play an important role in decoding spatiotemporal patterns of spikes and learning neural codes [56]. However, modeling the temporal behavior of each synapse in a network of leaky integrate-and-firing (I & F) neurons is a resource-intensive process and needs a large area in very large-scale integration implementation [57]. Synapses in I & F neurons can identify the difference between temporal input spiking patterns only when they have a time constant close to the time constant of the membrane potential of the neurons [58]. A subthreshold DPI synaptic circuit offers a long and biologically plausible time constant (tens of milliseconds) while a small and compact capacitor is deployed. The time constant linearly depends only on synaptic capacitance and I τ , as shown in equation (1) [34], where U T represents the thermal voltage, C syn denotes the synaptic capacitance, I τ is the current from M τ , and κ is the subthreshold slope factor. The synaptic capacitances are 33 nF, 47 nF, 68 nF, with a thermal voltage of 25 mV. I τ is a tunable value controlled through V τ and estimated using figure 2(b). κ is an intrinsic property of field-effect transistors that is determined from the slope of the transconductance curve in the subthreshold regime. The subthreshold slope factor of OFETs is one order of magnitude smaller than MOSFETs due to slower carrier-charge mobilities. The κ value is measured at 0.0379 for p-OFETs. Table 1 summarizes the parameter values in this paper, and the theoretical time constants (τ theo ) are calculated according to equation (1). The time constant can be estimated from experimental measurements and validated the theoretical values. The experimental time constant (τ exp ) is estimated using the measured step response of the DPI synaptic circuit fitting the data with an exponential equation. Equation (2) demonstrates the exponential relationship between the synaptic current and the time constant

DPI synaptic circuit characterization
A series of experiments have been designed and implemented to validate τ theo , theoretical time constant values calculated in section 2.5 and show the DPI circuit's functionalities. Sections 3.3 and 3.2 show the role of weighing (V W ) and gain (V g ) voltages in the DPI circuit, respectively. They either amplify or attenuate the output signal while not affecting the time constant. The effects of synaptic capacitance (C syn ) and the time-constant bias (V τ ) are discussed in sections 3.5 and 3.4 according to equation (1). C syn and I τ are linearly proportional to the time constant that has been estimated in a broader range using an extrapolation. Section 3.6 presents the i-f curves of the DPI synapse. Finally, the circuit's response to a train of simulated tonic spikes has been demonstrated in section 3.7.
To ease the analysis of circuit behavior, input voltage signal is simulated using a square wave altering between ±20 V to turn the circuit ON and OFF. The amplitude of the input voltage signal is determined  Table 2. Experimental parameters regarding figure 5. (V)   47  10  9  7 29, 30, 31 Table 3. Statistical results regarding figure 5(b). by the n-OFET characterization results presented in figure 3(b). The period of the input signal is constant, 16 s, through sections 3.2 to 3.6. The power supply voltage (V Source ) is constant, 10 V, in all the experiments. The experimental time constant is estimated for every step response of the DPI synaptic circuit; therefore, the number of steps (cycles) determines the number of time constants estimated in an experiment. Eighteen cycles have been captured for every experiment, and the results are shown through box plots.

Gain voltage
The gain voltage (V g ) is the DPI's one extra degree of freedom compared to the LDI synaptic circuit [34,59]. V g nonlinearly affects the amount of integrated I in and, consequently, amplifies or attenuates the synaptic current. Figure 5(a) illustrates the step response of the DPI synaptic circuit with three different V g values. The τ exp values are demonstrated in figure 5(b) using box plots. Table 2 shows the experimental parameters, and table 3 summarizes the statistical data regarding figure 5 According to section 2.5, the theoretical time constant is estimated to be 1640 ms for the experimental parameters shown in table 2. Figure 5(b) shows that the time constant is independent of the gain voltage;   )   47  10  9 5.5, 6.5, 7.5 30 Table 5. Statistical results regarding figure 6(b).  [60] that affects the results during data collection. Second, as previously discussed, the estimation method is intrinsically an error-prone process [59].

Weighing voltage
The weighing voltage (V W ) locally modulates the synaptic current amplitude. M W works as a subthreshold current source, and I W needs to be much greater than I τ to allow the DPI circuit to operate as a linear firstorder filter. Figure 6(a) demonstrates the step response of the DPI synaptic circuit to three V W values. The τ exp values are shown in figure 6(b) using box plots. Table 4 shows the experimental parameters, and table 5 summarizes the statistical data regarding figure 6(b).    47 10 8.9, 9, 9.1 7 30 Table 7. Statistical results regarding figure 7(b). (1640 ms). The hysteresis mechanisms and the estimation method's error are two main reasons for the discrepancy.

Time-constant bias
The time-constant bias (V τ ) is a parameter that allows explicit control of the circuit's time response. M τ is a subthreshold current source in the DPI synaptic circuit that charges the synaptic capacitor when M pre is OFF. Increasing V DD − V τ leads to a smaller V GS of p-type M τ and increases I τ , consequently decreasing the time constant. Figure 7(a) presents the step response of the DPI synaptic circuit to three V τ values. Box plots present the estimated time in figure 7(b). Table 6 shows the experimental parameters, and table 7 summarizes the statistical data regarding figure 7(b). Table 7 shows that the τ theo values of 1741 ms, 1640 ms, and 1581 ms are within the 25th to 75th percentiles of the boxplots in figure 7(b). The measurements present a range of time constants resulting from errors in the estimation method and the hysteresis mechanisms.

Synaptic capacitance
The time constant is proportional to the synaptic capacitance (C syn ). One of the advantages of the DPI synaptic circuit over other log-domain integrator synapses is its ability to produce a more extended response using smaller synaptic capacitors. However, in this study, a relatively large capacitor is deployed to mitigate the effect of hysteresis in the circuit and remove the parasitic capacitance caused by the test setup. Figure 8(a) shows the step response of the DPI synaptic circuit to three synaptic capacitance values, namely 33 nF, 47 nF, and 68 nF. The τ exp values are displayed in figure 8 using box plots. Table 8 demonstrates the experimental parameters,  and table 9 summarizes the statistical data regarding figure 8(b).
Statistical results shown in table 8 (of figure 8(b)) show that the theoretical time constants (τ theo ) are within the maximum and minimum τ exp values. The deviation of τ exp values from theory is most likely due to the hysteresis mechanisms that trap more carrier charges due to a higher value capacitance and, consequently, affect the change in the time constant over time.
The inset of figure 8(b) shows the extrapolation of results in a range of 1 nF to 100 nF. The relationship between synaptic capacitance and the median of the τ exp values is described as τ exp = 32.48 × (C syn ) + 87. 48 and indicates values as high as 3400 ms.

Frequency response
To summarize the organic DPI synapse dynamics in a single picture, the mean of the synaptic current in response to spike trains of square pulses of increasing frequency has been measured. Figure 9 demonstrates the average current-frequency (i-f) curve of the circuit in a range of 0.1 to 200 Hz that covers typical biological spiking frequencies. The results show that the mean of the synaptic current is linear and that synaptic input frequencies higher than 10 Hz can be extended beyond the plot. Also, the results show that the mean synaptic current remains almost constant for frequencies lower than 10 Hz but increases for higher frequencies.
Increasing V W − V SS elevates the current's levels but maintains unchanged slope.

The response to simulated biological spikes
Our previous synaptic circuit analysis used square voltage pulses to approximate presynaptic input to simplify the analysis of the synaptic output current. However, in order to elucidate synaptic response to nonideal but more realistic presynaptic input, we used a mathematically derived bio-inspired model to generate the input voltage spikes for the organic DPI synaptic circuit. We built on top of our prior work regarding the hyperbolic model of the neuronal spiking patterns [61] and applied adjustments in real time to suit the organic synapses relevant to this study.
This model is an extension of prior biological models, such as the Izhikevich model [62] and the adaptive exponential I & F model [63], with fewer parameters, less computational cost, and faster upswing, capable of generating different spiking patterns [61]. Hyperbolic functions are known for their extensive applications in solving differential equations. The model is, thus, constructed as a partial differential equation with hyperbolic functions involved, and parametrized to allow for the generation of various spiking patterns: Equation (3) indicates that the membrane potential (V) of a neuron can be modeled as a differential equation. In equations (3) and (4), u represents the membrane-recovery variable. The input current I of this model is considered the output of the synaptic circuit. Biologically, the current represents the ionic movement through cell gates (inward calcium and sodium ionic velocity). If current I is present, the membrane voltage increases and produces spiking patterns, after which it reaches a certain threshold (i.e., +30 mV) and resets to a resting value of V rest . At this point, a delay is added to the membrane-recovery variable u, as denoted in equation (5): if V 30, then:    figure 11. (V)   33  10  9 6.5, 7, 7. 5 30 The insulation of the cell membrane surrounding a neuron is considered a capacitor, which is defined by α in equation (3). Parameter β can determine the sharpness of the spikes, and is an empirically defined parameter related to the general gate voltage of inward calcium and sodium and outward potassium currents. The rate of the spikes, as well as their resting and peak times, can be determined by equation (4).
To observe the response of a single neuronal spiking pattern on the organic DPI synapse, the regular (tonic) spiking pattern is originally generated. For this, the following settings are used: α = 75 000, β = 1000, V rest = −62.5 mV, = 16, a = 0.2, b = 0.02, c = −65, d = 6, and I is a step signal switching from 0 to 14 μA. Initially, V is set to −70 mV. See figure 10.
The regular spiking-pattern neurons are the major type present in the cortex. If a DC-step is provided at the input current I, biologically, the neurons will produce a few (e.g., two) firing spikes with short interspikes before reaching their resting potential, after which the period of the spikes increases [62]. The hyperbolic model, in addition to allowing various spiking patterns to be generated by parameter adjustments, allows for modulating the frequency of the spikes by changing the strength of the DC current I. The frequency of the spikes is proportional to I in the model.
For real-time hardware generation of the spiking patterns, the model was emulated on a LabVIEW controlled DAQ. The dt value of the model denotes the division of time, or time steps, of the calculations (and output voltage signals) being updated. By fully eliminating the need for any vectorization or storage of array elements over time steps, and only updating V and u every dt, the computational complexity is significantlyreduced and, hence, the real-time nature of the model as well. For the purpose of this real-time experiment and the stability of the computations, as well as the need to generate several seconds of data, dt has been set to 0.1 ms.    Voltage scaling is required for the somatic voltages of the neuronal spikes that range between approximately −70 mV and +30 mV (see figure 10) to be compatible with the organic synaptic circuit in the range of ±20 V. The voltage value V generated in each time step of the model is, thus, scaled with a gain of 394.32 and an offset of 8170.347: The real-time implementation of the model allows for parameter adjustment and/or frequency modulation of the neuronal spikes on the fly as the spikes (actual voltages) are being generated and delivered on the DAQ. Figure 11 demonstrates the response of the DPI synaptic circuit to a single spike for three V W − V SS values. As shown in section 3.3, V W modulates the circuit output's amplitude. The experimental parameters are shown in table 10. Increasing the value of V W − V SS leads to elevating the peak of the synaptic current.
The response of the circuit to four sets of tonic spikes with different frequencies is shown in figure 12, with experimental parameters shown in table 10 with a constant value of V W − V SS = 7 V. The high-frequency spikes simulate a tetanized state when a motor neuron maximally stimulates its motor unit for a given period [64]. Starting the spikes, the synaptic current reaches its peak exponentially, and a train of highfrequency spikes increases the maximum amount of current in time. As the input spikes terminate, the current returns to the leakage level in a period equal to the time constant.

Simulation versus emulation
Simulation is often used to predict the behavior of a complicated system and, so far, studies concerned with modeling neuromorphic systems have been mostly restricted to silicon-based neuronal circuits. Therefore, we developed a custom-built circuit simulation that reflects idiosyncrasies of organic transistors to deepen understanding of the organic synaptic circuit and expedite its characterization (e.g., the choice of C syn ) [65]. The response of an organic synaptic circuit is simulated by emulating the function of the circuit's individual organic transistors based on a compact model. From figure 13, it can be found that the simulated transconductance of both p-and n-OFETs matches well with the experimentally characterized transconductance curves, which indicates that the transistor model in our circuit simulator can reflect the real dynamics of the built transistor blocks, not only in the above-threshold regime, but also in the subthreshold regime. Figure 14 demonstrates a comparison between the simulated and experimentally measured step responses of the DPI synapse for three different values of V g . As can be seen in this figure, the simulated synaptic response curves at three different V g values are in good agreement with the experimental measurements. This observation further illustrates the effectiveness of our organic synaptic circuit simulation. In a more realistic trial, as described in section 3.7, a series of simulated biological spike trains were fed simultaneously into our circuit simulator and a real DPI synaptic circuit, and their post-synaptic outputs are presented in figure 15. It can be seen that the simulated post-synaptic current in response to irregular pre-synaptic spike patterns can exhibit similar evolving trends to those shown in the real synaptic circuit. However, the shape of the simulated and experimentally measured post-synaptic currents is not identical to the pre-synaptic spikes. This inconsistency between simulated and actual measured data in terms of post-synaptic current shape can be attributed to the hysteresis observed in fabricated organic transistors. As displayed in figures 2 and 3, there is generally a considerable discrepancy between the transconductance measured by forward and reverse sweeps. The transistor parameters in our simulation are only adjusted according to the reverse sweep, hence it can only reflect the dynamic characteristics of the transistor within certain limits. When the waveform or the input spiking patterns become irregular (e.g., from a pure square wave in figure 14 to a series of biological spike trains in figure 15), the inaccuracy of the model parameters caused by the hysteresis effect of the transistor will be accumulated or even magnified, leading to some inconsistency between simulation results and experimental observations. While circuit simulation can only predict a limited range of synaptic circuit responses with great accuracy, we can still use simulation to analyze the impact of some parameters on the overall performance of a circuit in advance, thus improving the efficiency of electronic-design automation. For instance, the predicted step response of the DPI synapse for three different values of C syn is shown in figure 16. We can see that the post-synaptic current I syn rises more slowly with increasing C syn , implying that a larger charging time constant is rendered for a larger synaptic capacitance selected. Furthermore, we can roughly estimate the theoretical time constants from the simulated step response, in this case τ sim = 260, 1, 650 and 15 600 ms, which indeed satisfies the linear relationship between τ and C syn , obtained by fitting the experimental data.

Conclusion
Neuromorphic systems are biologically inspired hardware implementations of artificial neural networks that require two main functional blocks: artificial somas and artificial synapses. While somas integrate synaptic currents and produce a voltage spikes, the main functions of synapses include interfacing between individual neurons (somas), scaling the input signal, and converting the voltage spike into a synaptic current.
In contrast to two-terminal memristive devices used to approximate synaptic weighting, synaptic circuits offer advantages, including greater synaptic control (via individually tunable parameters, such as V g , V W , V τ , and C syn ) and integration with circuits that implement other synaptic neuromorphic functions (e.g., short-term depression or short-term potentiation, voltage-gated channels, synaptic conductances), as well as fabrication integration with other neuromorphic blocks, such as somas.
Here, we show a type of biologically realistic spiking synaptic circuit, called a DPI synapse, which is implemented using physically flexible and biologically compatible complementary organic electronics. We demonstrate that, indeed, the organic synapse converts somatic voltage spikes into proportional output current.
We characterize the synaptic parameters, including the gain and weighting voltages V g and V W , the timeconstant bias V τ , and synaptic capacitance C syn . We also demonstrate the synaptic circuit response when stimulated with input spikes of different frequencies. Synaptic time constants comparable to the time constants of the neuron's membrane potential are critical in distinguishing different temporal input spiking patterns. We show that the time constant of our organic spiking synapses can reach in excess of two seconds. We demonstrate the synaptic response due to biologically realistic input voltage spikes of various frequencies. Finally, we via custom-build circuit simulation based on transconductances of p-and n-type organic transistors, that the fabricated synaptic circuit is in good agreement with simulation-predicted behavior. This work paves the way for future networks of fully organic neurons and their networks.