Design of a Power Efficient Artificial Neuron Using Superconducting Nanowires

With the rising societal demand for more information-processing capacity with lower power consumption, alternative architectures inspired by the parallelism and robustness of the human brain have recently emerged as possible solutions. In particular, spiking neural networks (SNNs) offer a bio-realistic approach, relying on pulses, analogous to action potentials, as units of information. While software encoded networks provide flexibility and precision, they are often computationally expensive. As a result, hardware SNNs based on the spiking dynamics of a device or circuit represent an increasingly appealing direction. Here, we propose to use superconducting nanowires as a platform for the development of an artificial neuron. Building on an architecture first proposed for Josephson junctions, we rely on the intrinsic non-linearity of two coupled nanowires to generate spiking behavior, and use electrothermal circuit simulations to demonstrate that the nanowire neuron reproduces multiple characteristics of biological neurons. Furthermore, by harnessing the non-linearity of the superconducting nanowire’s inductance, we develop a design for a variable inductive synapse capable of both excitatory and inhibitory control. We demonstrate that this synapse design supports direct fan-out, a feature that has been difficult to achieve in other superconducting architectures, and that the nanowire neuron’s nominal energy performance is competitive with that of current technologies.

Introduction

The human brain has long been a subject of fascination due to the wide variety of complex operations made possible by groups of a single component—the neuron. Now, as computation needs are rapidly approaching the limits of traditional von Neumann architectures, the neuron’s unique features have led it to become a source of inspiration for new directions for the advancement of computing. Unlike conventional computing schemes, the human brain benefits from characteristics such as extensive parallelism and robustness to errors, allowing it to operate efficiently despite slow speeds on the order of a few Hz (Furber and Temple, 2007). These appealing qualities have spurred the development of technologies that use the brain as a platform for information processing, ranging from small scale modeling of single neuron dynamics to large-scale parallel computing.

At the heart of this concept are spiking neural networks (SNNs), which seek to mimic the spiking dynamics of the brain in order to encode information, with the additional benefit of possibly offering new insight into the brain’s functionality. In this scheme, spikes serve as the tokens of information, while neurons act analogously to logic gates, producing a single output in response to a combination of multiple inputs (Furber and Temple, 2007). Synapses, which connect neurons, also play a crucial role in the overall architecture by enabling learning and allowing for adaptable networks. Past approaches to SNNs vary both in degree of bio-realism and in how the spikes are implemented. While software approaches that hard-code spiking dynamics offer flexibility and precision, they are computationally expensive. As a result, other SNNs have used a hardware implementation, relying on devices with intrinsic dynamics that replicate neuron behavior as a means of reducing computation costs. Hardware approaches have been explored in a wide variety of platforms, including CMOS (Sourikopoulos et al., 2017), magnetic materials (Li et al., 2017), memristors (Querlioz et al., 2011; Thomas, 2013), and superconducting Josephson junctions (Crotty et al., 2010; Schneider et al., 2018). CMOS circuits provide large-scale integration, but suffer from high power dissipation and need many components to achieve biological realism. Memristors have characteristics which are similar to biological synapses, but need to be paired (Jeong and Shi, 2018) with spiking neurons made from CMOS circuits for SNNs and thus have similar limitations with their power dissipation. Magnetic materials can produce spiking from the spin-torque effect (Matsumoto et al., 2019), but have yet to be integrated into a full system with synapses and will most likely not be as energy efficient as superconductors. Josephson junctions are a fast and energy-efficient technology, but need more components in large networks due to poor fan-out/fan-in properties (Mukhanov et al., 1978; Likharev and Semenov, 1991; Katam and Pedram, 2018); furthermore, their weak action potentials are unable to be viewed directly, making diagnostics difficult.

These developing technologies highlight several characteristics that are critical for an artificial neuron: (1) inherent device dynamics that are capable of producing spiking behavior; (2) tunable synapses between neurons to allow for the expansion into larger networks with adjustable connectivity; and (3) low power dissipation, both in the dynamic firing state of the neuron and in the static state. As has been emphasized in prior literature (Sourikopoulos et al., 2017), optimizing static power dissipation is particularly vital to the goal of creating an energy efficient network.

Here, we propose an artificial neuron based on superconducting nanowires operating at cryogenic temperatures. Superconductors are prime candidates for generating low-power spiking behavior due to their inherent non-linearity and negligible static power dissipation. Building on an architecture first implemented in Josephson junctions (Crotty et al., 2010), we rely on the coupling between two shunted nanowires to act analogously to a two-channel model for an action potential. We start by describing the non-linear dynamics of superconducting nanowires, and then present the architecture of the nanowire-based neuron. Using electrothermal circuit simulations, we demonstrate that the device is able to replicate several behaviors of a single neuron. Finally, we present a synapse design and discuss the advantages of the nanowire neuron, including fan-out and an energy figure of merit four orders of magnitude better than that of competing technologies, which are critical in moving toward the parallelism of the human brain.

The Nanowire Neuron Model

Superconducting nanowires possess an inherent non-linearity that serves as the building block of our artificial neuron model. We begin by briefly describing this non-linearity in a single nanowire, and then present the nanowire neuron circuit and its basic operation principles.

Relaxation Oscillations

The intrinsic non-linearity of superconducting nanowires makes them ideal candidates for the hardware generation of spiking behavior. When a bias current flowing through a superconducting nanowire exceeds a threshold known as the critical current (I_c), superconductivity breaks down and the nanowire becomes resistive, producing a voltage. The nanowire only switches back to the superconducting state once the bias current is reduced below a level called the retrapping current (I_r), and the resistive portion (the “hotspot”) cools down. When the nanowire is placed in parallel with a shunt resistor, this switching process participates in electrothermal feedback with the shunt, producing relaxation oscillations (Toomey et al., 2018).

Relaxation oscillations can be viewed in the context of a simplified action potential. Like the Na⁺ influx and K⁺ outflux currents of a neuron, the influx and outflux currents from the nanowire to the shunt resistor are governed by different timescales, τ₁ and τ₂. As shown in Figure 1A, the rising edge of the output voltage is defined by τ₁ = L/(R_s + R_hs), where L is the inductance of the nanowire, R_s is the shunt resistance, and R_hs is the resistance of the nanowire hotspot, usually on the order of ∼1–10 kΩ. Conversely, the outflux current occurs when the nanowire is no longer resistive and the bias current is redirected from the shunt; this reduced resistance results in a slower time constant τ₂ = L/R_s which defines the falling edge of the output voltage. For typical nanowires devices, τ₁ ∼ 100 ps and τ₂ ∼ 1 ns. The two currents are “gated,” as shown by the insets in Figure 1A, by the state of the nanowire—when the state is resistive (producing a voltage), the influx current flows into the shunt, and when it is superconducting, the outflux current flows back to the nanowire.

FIGURE 1

Figure 1. Relaxation oscillations in superconducting nanowires, which serve as the foundation of the nanowire neuron’s spiking behavior. (A) Simplified model of a relaxation oscillation. The output voltage is defined by two time constants. The rising edge occurs when the superconducting nanowire has switched into the resistive state (R_hs > 0), and the bias current is redirected to the shunt resistor. The falling edge takes place after the nanowire regains superconductivity (R_hs = 0), and the bias current is redirected from the shunt resistor. (B) Example of a superconducting nanowire in a long, meandered design for obtaining a high kinetic inductance. (C) Experimentally measured relaxation oscillations in a long superconducting nanowire shunted by 50 Ω.

The inductance of superconducting nanowires is dominated by an intrinsic material property known as kinetic inductance (Kerman et al., 2006), and is defined per unit length of the structure. As a result, it is possible to tune these time constants by changing the length of the nanowire, with a longer wire leading to a higher inductance and thus a longer timescale. Figure 1B shows a scanning electron micrograph of a typical superconducting nanowire with a meandering geometry designed for maximizing the total device inductance. An example of experimentally observed relaxation oscillations for such a device is displayed in Figure 1C.

The Neuron Model

Although a shunted nanowire on its own produces oscillations analogous to action potentials, as the bias current increases, the output signal eventually accumulates a voltage offset. This effect deviates from true neuron behavior, as the cell must maintain a constant resting potential (approximately −70 mV). To overcome this difference, we have implemented a neuron architecture based on one that was first proposed for Josephson junctions (Crotty et al., 2010), as shown in Figure 2. The circuit consists of two shunted nanowires—the main oscillator and the control oscillator—linked together in a superconducting loop. A bias current I_bias is applied to both oscillators such that they are each biased right below their critical currents, but in opposite directions. To trigger an action potential, a small input current pulse I_in (Figure 2A) is applied and sums with the bias current to exceed I_c of the main oscillator, causing it to switch (Figure 2D). The control does not fire since the input opposes the direction of its bias.

FIGURE 2

Figure 2. Circuit simulations of the two-nanowire soma, where the two oscillators act analogously to the two ion channels in the simplified neuron model. (A) Input pulse, I_in = 4 μA. (B) Current through the loop inductor. (C) Current through the control nanowire. The control nanowire reduces the amount of counterclockwise current circulating in the loop, allowing the main nanowire to fire again. (D) Current through the main nanowire. (E) Output voltage pulse that is sent to the synapse. For these simulations, the critical current of the control nanowire is I_c,control = 30 μA, the critical current of the main nanowire is I_c,main = 30 μA, and I_bias = 58.6 μA.

Once the main oscillator switches, current is added to the superconducting loop in the counterclockwise direction (Figure 2B), which sums with the bias current to fire the control oscillator (Figure 2C). The control oscillator removes counterclockwise current from the loop, allowing the main oscillator to fire again. Without the presence of the control oscillator, the main oscillator would only be able to fire once, since the counterclockwise current added to the loop would reduce the total current through the nanowire of the main oscillator below its I_c. The voltage from the main oscillator node (Figure 2E) serves as the spiking output that is carried down to the next neuron via the synapse. Unlike the output from the single shunted nanowire, the output of the two-nanowire circuit does not accumulate a bias offset, making it a suitable spiking signal.

In the context of the two-channel neuron model, the main oscillator acts analogously to the Na⁺ influx current by adding flux to the superconducting loop in the form of a circulating current. The control oscillator acts analogously to the K⁺ outflux current by reducing the circulating current, resetting the neuron and allowing the main oscillator to fire again. As described in Toomey et al. (2018), the rate at which each oscillator fires depends on the magnitude of the bias current, paralleling the voltage-dependent rate constants of ion gates in the Hodgin-Huxley model (Ermentrout and Terman, 2010).

Single Neuron Characteristics

Neurons display a wide variety of traits unique to certain populations, allowing them to collectively interact to achieve varied and complex tasks. While no single neuron possesses all possible traits, the basic functionality of an artificial neuron can be evaluated by demonstrating some common bio-realistic characteristics. Here we present multiple neuron behaviors that can be achieved with the nanowire neuron, using electrothermal circuit simulations conducted in LTSpice. The simulations implement material-specific characteristics and nanowire hotspot dynamics as described in previous literature (Kerman et al., 2009; Berggren et al., 2018), and have been shown to reliably reproduce experimental data pertaining to nanowire relaxation oscillations (Toomey et al., 2018).

Threshold Response

A general characteristic of biological neurons is their inability to fire unless the input signal exceeds a certain threshold. Figure 3A shows the threshold voltage response of the nanowire neuron when the bias current is held constant and the input current is varied. As evident in the plot, the neuron does not begin firing until the input current passes a threshold, defined by when the sum of the bias and input current through the main nanowire exceed its I_c. Above the threshold, the peak voltage of the spike output is essentially constant. However, as emphasized by Izhikevich (2004), biological neurons have a threshold that may be varied by previous activity, such as an inhibiting input that reduces it. Figures 3B,C illustrate this process (“threshold variability”) in the nanowire neuron; an initial subthreshold input pulse (Figure 3B) fails to elicit a spike, while a later input pulse of the same magnitude triggers a spike (Figure 3C) after a smaller negative pulse reduces the firing threshold. It should be noted that when the preceding pulse was of the opposite polarity, no spike was triggered. This behavior is consistent with the expectations of Izhikevich (2004).

FIGURE 3

Figure 3. Firing threshold of the two-nanowire neuron. (A) Peak output voltage as a function of input current under a constant bias (I_bias = 58.6 μA). The plot illustrates that the neuron does not spike until the input current exceeds 4 μA, at which point it fires with output voltages of the same amplitude. Inset shows the time domain voltage output of the neuron for different input currents. (B) Input to the neuron, leading to a reduction in firing threshold by a preceding negative pulse. (C) Spiking output of the neuron in response to the inputs of (B), demonstrating that the nanowire neuron’s firing threshold is variable. For this simulation, I_bias = 57.62 μA, the positive inputs I_in = 4.6 μA, and the negative input I_in = –4.3 μA.

Refractory Period

In addition to exhibiting a firing threshold, the nanowire neuron displays a refractory period, which we define to be the minimum time between two input pulses such that both pulses elicit a spike. Figure 4 illustrates this response. When two pulses are separated enough in time so that the main oscillator is biased close to its critical current when the second input pulse arrives, then the second pulse will cause a spike (Figure 4A). However, if the second pulse arrives before the bias current has fully returned to the main oscillator, then the sum of the second input pulse and the bias will not be sufficient to switch the nanowire and trigger the neuron (Figure 4B). As a result, the refractory period is limited by the time it takes to fully bias the main oscillator again, which is a function of the L/R time constants of the circuit described in Section “Relaxation Oscillations,” as well as the series inductance in the loop. The inductance values stem from the kinetic inductance of the superconducting film, and are therefore dependent on the choice of material, film thickness, and nanowire width and length.

FIGURE 4

Figure 4. Refractory period of the two-nanowire neuron. (A) Response when there is sufficient time between two inputs to each elicit a separate spike. Parameters: I_bias = 58 μA, I_in = 6 μA, Δt = 4 ns. The pink dashed lines indicate the beginning of the rising edge of each pulse. (B) Response when there is insufficient time between two input pulses, causing the neuron to fire only once. Parameters are the same as in (A), except Δt = 2 ns. For both cases, panel (i) displays the current through the nanowire of the main oscillator, while panel (ii) displays the output voltage of the neuron.

Class I Behavior

Biological neurons differ in their response to varying signal strengths. Whereas Class I neurons have a spiking frequency that increases with increasing input strength, Class II neurons maintain a constant firing rate (Izhikevich, 2004; Crotty et al., 2010). Figure 5 illustrates the spiking behavior of the nanowire neuron at different levels of bias current. Figure 5A shows the time-domain voltage output of the neuron as the bias current is increased, and suggests an increase in spiking frequency. This response is confirmed by observing the voltage output’s frequency spectrum displayed in Figure 5B, which shows a shift in the spiking frequency to higher levels with increasing bias. Consequently, the nanowire neuron has Class I behavior. The modulation of spiking frequency by bias current demonstrates that the frequency of the nanowire neuron output may be used to glean information about its input conditions.

FIGURE 5

Figure 5. Effect of bias current on spiking frequency. (A) Time domain simulations of the two-nanowire neuron with different bias currents. I_in = 6 μA for all simulations. Traces have been shifted from one another in the y-axis for clarity. (B) Fourier transform of the voltage output for each biasing condition. The shift in peak frequency with bias current indicates that the circuit acts like a Class I neuron.

Axon: Transmission Line Characteristics

After an action potential occurs in a biological neuron, the output signal propagates down the axon as if sent through a delay line (Furber and Temple, 2007). This delay is valuable in that it preserves time domain information, potentially facilitating behaviors that rely on the recognition of specific spatio-temporal patterns (Izhikevich, 2004). Such pattern recognition is often not possible in SNNs with traditional wiring, since signals travel too rapidly for timing information to be maintained (Furber and Temple, 2007). This is not the case for superconducting nanowires that are designed to act as transmission lines. Recent work has shown that the high kinetic inductance of superconducting transmission lines, like those made out of niobium nitride, results in propagation speeds of ∼2% c, where c is the speed of light in vacuum (Zhao et al., 2017b). As illustrated in Figure 6, a simulated nanowire neuron output sent through a superconducting transmission line model (Zhao et al., 2018a) is delayed by ∼100–500 ps, close to the full width of an action potential. In mammalian brains, axonal delays like the cortico-cortical delay (Ferraina et al., 2002) are also on the same timescale as the full width of an action potential [typically a few milliseconds (Bean, 2007)], suggesting that the relative delay in our system with respect to the spike duration is appropriate. If longer delays are needed, the transmission line can simply be made longer.

FIGURE 6

Figure 6. A superconducting transmission line as an axon. Simulations of a superconducting transmission line show that the spikes can be delayed on the order of ∼0.5 ns, depending on the length of the structure. This could enable the storage of timing information in addition to frequency information. Transmission line parameters: nanowire inductance L_n = 0.3 nH/μm, nanowire capacitance C_n = 0.1 fF/μm, propagation speed v = 1.9% c, transmission line length l = 2.5 mm. Shorter transmission lines on the order of 800 um still had delays of ∼140 ps.

The Synapse

The collective dynamics of a neural network depend on the ability of a neuron to influence the behavior of another downstream neuron via a synapse. Here we introduce an inductive synapse that can be integrated with the nanowire neuron to facilitate downstream control. We start by demonstrating excitatory and inhibitory control, and then present a scheme for tuning the synaptic strength.

The Inductive Synapse

Figure 7A illustrates the circuit schematic of an inductive synapse that may be implemented in the nanowire neuron. Similar to the slow release of neurotransmitters in response to an action potential, the inductive synapse relies on the slow charging of a large inductor in response to the nanowire neuron’s more rapid voltage spikes. The energy stored in the large synapse inductor is then discharged as current into the input port of the target neuron, modulating its behavior.

FIGURE 7

Figure 7. Nanowire neuron with an inductive synapse. (A) Circuit schematic. The output voltage of the neuron charges the large synapse inductor L_syn. The inductor discharges current into the input port of the downstream target neuron, modulating its behavior. A series resistor R_series is in place to reduce backaction into the main neuron. (B) Excitatory downstream control. Parameters: I_bias,main = 59 μA, R_series = 14 Ω, R_syn,1 = 40 Ω, L_syn = 0.265 μH, R_syn,2 = 40 Ω, I_bias,target = 57.17 μA, I_in = 4.6 μA. (C) Inhibitory downstream control. Parameters I_bias,main = –58.6 μA, R_series = 24 Ω, R_syn,1 = 45 Ω, L_syn = 0.23 μH, R_syn,2 = 40 Ω, I_bias,target = 57.68 μA, I_in = 4.6 μA. For both cases: Panel (i) displays the output voltage of the main neuron, with the red dashed lines indicating the rising and falling edge of the input signal; Panel (ii) displays the current through the synapse inductor; Panel (iii) displays the output voltage of the downstream target neuron.

The effect of the inductive synapse can be either excitatory or inhibitory based on the sign of the bias current applied to the upstream (main) neuron. These two cases are shown in Figures 7B,C, which displays the voltage outputs of the main and target neurons, as well as the current through the synapse inductor. In the excitatory case (Figure 7B), the upstream neuron is positively biased such that the inductive synapse discharges a positive current, causing an under-biased target neuron to fire. In the inhibitory case (Figure 7C), the upstream neuron is negatively biased, causing a negative current to be sent from the synapse to the target, turning off a target that was biased in the firing state. Consequently, the inductive synapse allows for both types of control between neurons.

Variable Strength Synapse

Although the inductive synapse of Figure 7 can be engineered for both excitatory and inhibitory control of a downstream neuron, a fundamental property of artificial neural networks is the ability to modulate that control by adjusting synaptic strength. One possible scheme for implementing such variability in the inductive synapse is to incorporate superconducting nanowires as a different circuit element: a tunable inductor. A nanowire’s kinetic inductance increases with increasing bias current, reaching an enhancement of 10–20% near I_c (Annunziata et al., 2010). This modulation has been incorporated into the circuit model of the superconducting nanowire used in these simulations (Berggren et al., 2018). By placing a high inductance nanowire with an ideal current source in parallel with the synapse inductor, the overall parallel inductance of the synapse can be modulated, which in turn changes the amount of current sent to the target neuron. Figure 8 shows the simulated results for the case of an inhibitory synapse. When a higher modulating current I_mod is applied to the nanowire inductor, the overall parallel inductance increases, reducing the amount of current sent to the target. It is important to note that the polarity of the modulating current is not important, since the change in kinetic inductance depends only on the magnitude of the modulating current in relation to its I_c. For instance, Figure 8B illustrates that the modulation in synaptic current for I_mod = 5 μA and I_mod = −5 μA is roughly the same. As a result, it is clear that the modulation is due to the change in kinetic inductance, and not simply an injection of current by I_mod in the opposing direction.

FIGURE 8

Figure 8. Modulating the inductive synapse. (A) Circuit schematic of the inductive synapse with a high-inductance nanowire and ideal current source placed in parallel. L₁, L₂ < < L_nanowire, L_syn. R_{series, out} has been added to further prevent backaction from the target neuron. (B) Simulation of the current through R_series,out in an inhibitory synapse as a function of different modulation currents. Spikes represent backaction from the firing of the target neuron. (Inset) Enlarged view of the boxed area. Higher modulation currents increase the overall synapse inductance, reducing the amount of current sent to the target. The ±5 μA modulation currents have nearly the same effect, showing that modulation is not polarity-dependent. Parameters: R_series,in = 25 Ω, R_syn,1 = 39 Ω, R_syn,2 = 40 Ω, R_series,out = 0.1 Ω, L_syn = 0.45 μH, L_nanowire = 0.275 μH, I_{c, nanowire} = 6 μA, L₁ = L₂ = 50 pH, I_bias,target = 57.65 μA, I_bias,main = –59.5 μA.

In conjunction with the plasticity of synapses, the high degree of parallelism in the brain creates a densely connected, adaptable network able to optimize and adjust for different conditions. An example of fan-out in the nanowire neuron is shown in Figure 9. As shown in Figure 9A, a single neuron is connected to four target neurons through four separate tunable synapses. When each of the four modulating currents is set to I_mod = 0, the firing of all four targets is inhibited by the main neuron, as illustrated in Figure 9B. To weaken the connection with one of the targets (target #4), I_mod,4 is set to 8 μA, turning off the inhibiting action on target #4 but allowing it to remain on the three other target neurons, as shown in Figure 9C. Comparison of the synaptic currents for each of the four targets shows that the synaptic current for target #4 is reduced as a result of the modulated inductance. This modulation illustrates that the nanowire neuron is able to be used in a parallel network where the strength of individual synaptic connections can be adapted.

FIGURE 9

Figure 9. Fan-out of nanowire neuron with a tunable inductive synapse. (A) Simplified circuit model of the fan-out circuit. (B) Simulation when I_mod = 0 μA for all four target neurons. (i) Output of the main upstream neuron. (ii) Current through each of the four series resistors R_{series, out} to each of the target neurons. (iii) Output voltage of the four target neurons, shifted on the y-axis for clarity. (C) Simulation when I_mod = 0 μA for targets #1–3 and I_mod = 8 μA for target #4. (i) Output of the main upstream neuron. (ii) Current through each of the four series resistors R_{series, out} to each of the target neurons. The current through R_series,out for target #4 is less than that of the other targets, showing that it has been modulated. (iii) Output voltage of the four target neurons, shifted on the y-axis for clarity. The first three targets have been inhibited, while the inhibitory action on target #4 has been turned off. Parameters: R_series,in = 7 Ω, R_syn,1 = R_syn,2 = 300 Ω, L_syn = 0.4 μH, R_series,out = 1 Ω, L_nanowire = 0.275 μH, I_{c, nanowire} = 6 μA, I_bias,main = –59.5 μA, I_bias,target = 57.65 μA.

Fan-out may be one area where nanowires will be an improvement over Josephson junctions (Crotty et al., 2010), another superconducting technology with promise for artificial neurons. Both nanowires and Josephson junctions have quantized flux outputs (flux here being defined as the time-integral of the voltage). However, Josephson devices have outputs of only a single flux-quantum, while nanowires typically have outputs with many more flux quanta (e.g., 70 flux quanta for the device in Figure 2). For instance, typical niobium nitride nanowires with critical currents in the range of 100 μA and film thicknesses around 10 nm have flux outputs ranging from 50 to 100 flux quanta. In pushing the fan-out and fan-in to larger systems, one expects to be eventually limited by parasitic inductances and thermal noises. In such a case, the more substantial signal of the nanowire neuron would permit a larger fan-out and fan-in, leading to a higher degree of parallelism. In addition to the fan-out, the nanowire voltage signals are long enough and large enough to be digitized directly on an oscilloscope, in contrast to their Josephson junction counterparts, allowing for more direct analysis and readout. Finally, there are a suite of three-terminal, power-control devices made from nanowires (McCaughan and Berggren, 2014), which could be fabricated alongside the neurons and used to boost signals and match impedances.

Benchmarking Synaptic Energy and Speed

The energy dissipation of both the neuron and the synapse can be calculated using LTSpice by taking the time integral of the current-voltage product of each circuit element. Performing this analysis, we find that the nanowire neuron has an energy of about 0.05 fJ for each action potential, while the synapse is about an order of magnitude less at around 0.005 fJ. In large systems, it will be the synapses that will dominate; typically if there are O(N) neurons there will be O(N²) synapses. Hence, even though the neuron dissipates more energy, it will be the synapses that will dominate the power consumption of a large network.

In a SNN, the energy dissipated increases with the speed of the system; spiking twice as often dissipates twice the energy, assuming the energy per spike stays constant. The appropriate figure of merit to compare different technologies is then to take the ratio of speed and power. IBM (Merolla et al., 2014) introduced the figure of merit of synaptic operations per second per watt (SOPS/W). We include a constant factor of about 400 W/W for the nanowire neuron to account for the cryogenic cooling costs. Table 1 compares both the energy per spike and the SOPS/W for the nanowire neuron, the human brain, and two CMOS technologies. We acknowledge that the estimate for the nanowire neuron is a projection from a calculation of a single component, whereas the other entries in the table have actually been measured on large systems. However, given that superconducting platforms have no dissipation in their interconnects in their DC state, we believe that it is reasonable to project in such a way. We have also assumed that the constant applied bias current will not significantly affect the overall power consumption, based on previously proposed superconducting architectures that use a current dividing network made of inductors rather than resistors to eliminate the majority of dissipation from static biasing, as has been suggested for single flux quantum electronics (Mukhanov, 2011). In doing so, it is apparent that the nanowire neuron can be a highly competitive technology from a power and speed perspective, and that operation at cryogenic temperatures does not pose a serious disadvantage to its overall performance.

TABLE 1

Table 1. Energy and figure of merit values for different artificial neural networks in comparison to those of the human brain.

Density is also a relevant parameter for building large-scale neural networks. At present, we offer a conservative estimate that an initial proof-of-concept device would have a cell size of at most 50 μm × 50 μm. We believe this area would likely be made smaller by refining the circuit parameters, the cell layout, and choice of materials. For instance, a superconducting film with a higher kinetic inductance would lead to a smaller device area. However, we imagine this technology initially being used in a centralized cloud-based architecture, where the neuron’s low power consumption relative to that of competing platforms would compensate for the lower device density. This type of tradeoff has been justified in past literature, such as the adiabatic quantum flux parametron which similarly sacrifices device area for low-power logic (Takeuchi et al., 2013). Thus, while the anticipated device density of the nanowire neuron can be improved, its speed and power dissipation offer significant advantages that could compensate for its estimated cell size.

Discussion on Possible Synapse Alternatives

Although the fan-out achieved here is far from the level of parallelism in the human brain (where each neuron connects to 1000s of neighbors), it suggests that nanowires may serve a unique purpose in the development of future superconducting neural networks, which have thus far struggled to support fan-out. Given that nanowires can interface with both CMOS and Josephson junction circuits (Zhao et al., 2017a), it may be possible for nanowire neurons to serve as intermediary devices in a network with both platforms. Indeed, a recently proposed neural network with hybrid technologies employed superconducting nanowires as photon detectors, relying instead on optical signals for facilitating high fan-out (Shainline et al., 2018; Shainline, 2019). Although our work uses nanowires solely as electrical components, they can easily be biased to act as photodetectors (Goltsman et al., 2001; Marsili et al., 2011) as well, illustrating that the two different architectures would be compatible for integration. These two schemes thus illustrate the diverse ways in which superconducting nanowires can be used in neural networks, suggesting that a combination of the two technologies may be possible.

Furthermore, while the inductive synapse proposed in this work relies on an external modulating bias current, it may be possible to use a superconducting memory cell to store the value of this modulating current, or replace it directly in the circuit. Memory cells based on superconducting nanowire loops have been shown to reliably store states in the form of a trapped circulating current (Murphy et al., 2017; McCaughan et al., 2018; Zhao et al., 2018b), with recent work demonstrating that one can program and store different amounts of current in the loop in discrete quantities (Toomey et al., 2019). Incorporating these programmable memory cells into the modulating elements of the inductive synapse could enable storage of a varying synaptic strength in each synapse, allowing for more complex applications. It may also be possible for nanowire neurons to connect through alternative synapse designs, such as one based on inductive coupling. Further analysis is needed to verify the possibility of fan-out in such a scheme.

Conclusion

By taking advantage of intrinsic non-linearities in superconducting nanowires, we have designed a platform for a low-power artificial neuron. In this platform, the coupling of two nanowire-based oscillators acts analogously to the two ion channels in a simplified neuron model, producing an output voltage spike that serves as the information-carrying token in these circuits. Using electrothermal circuit simulations, we have shown that the nanowire neuron is able to reproduce universal biological neuron characteristics, such as a firing threshold, as well as unique characteristics distinct to certain neural classes, such as parabolic bursting (see Supplementary Material). Furthermore, we suggested that a nanowire transmission line with a propagation speed of ∼2% c may be used as an axon delay line, potentially allowing spatio-temporal information to be accessed. These collective behaviors may enable the nanowire neuron to be used in a rich variety of operations.

In addition to harnessing the non-linearity of the nanowire’s switching dynamics, we relied on the non-linearity of the nanowire’s kinetic inductance in order to develop a variable inductive synapse. We demonstrated that the total parallel inductance of the synapse can be tuned with a modulating bias current, thereby changing the strength of the signal sent to a downstream target neuron. This scheme proved to be capable of fan-out, as demonstrated by a single neuron inhibiting the firing of four target neurons. An energy analysis of this circuit in comparison to other spiking networks illustrated that the nanowire neuron has competitive performance in the dynamic firing state and a figure of merit four orders of magnitude better than certain alternative platforms, while the static state benefits from the lack of power dissipation by the superconducting elements. Although experimental realization of these results is a subject of future work, the analysis performed here suggests that the nanowire neuron is a promising candidate for the advancement of low-power artificial neural networks.

Looking forward, networks of superconducting nanowires could be the basis for powerful new computer hardware. Analog blocks of highly connected neurons could be digitally linked to achieve networks with either scale-free or small-worlds connectivity. With superconducting interconnects, entire chips could be wired together with no cost in heat dissipation. The result would be a large-scale neuromorphic processor which could be trained as a SNN to perform tasks like pattern recognition or used to simulate the spiking dynamics of a large, biologically-realistic network. The combination of speed, low power dissipation, and biological realism with only a few components suggests that nanowires could outperform or complement other existing and developing neuromorphic hardware technologies.

Data Availability

The nanowire LTSpice model used for this study can be found on GitHub at https://github.com/karlberggren/snspd-spice. All circuit schematic and data files, including the Matlab code used to generate the figures, are available from the authors upon request.

Author Contributions

ET performed the simulations with input from KS and KB. All authors contributed to the design of the circuits and the writing of the manuscript.

Funding

ET was supported by the National Science Foundation Graduate Research Fellowship Program (NSF GRFP) (Grant No. 1122374). The authors would like to acknowledge the generous support of the Bose Foundation.

Conflict of Interest Statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

The authors thank Murat Onen, Brenden Butters, and Jason Meyers for scientific discussions, Akshay Agarwal and Dr. Donnie Keithley for edits, and the entire Quantum Nanostructures and Nanofabrication group.

Supplementary Material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fnins.2019.00933/full#supplementary-material

References

Annunziata, A. J., Santavicca, D. F., Frunzio, L., Catelani, G., Rooks, M. J., Frydman, A., et al. (2010). Tunable superconducting nanoinductors. Nanotechnology 21:445202. doi: 10.1088/0957-4484/21/44/445202

PubMed Abstract | CrossRef Full Text | Google Scholar

Bean, B. P. (2007). The action potential in mammalian central neurons. Nat. Rev. Neurosci. 8:451. doi: 10.1038/nrn2148

PubMed Abstract | CrossRef Full Text | Google Scholar

Berggren, K. K., Zhao, Q. Y., Abebe, N., Chen, M., Ravindran, P., McCaughan, A., et al. (2018). A superconducting nanowire can be modeled by using SPICE. Supercond. Sci. Technol. 31:055010. doi: 10.1088/1361-6668/aab149

CrossRef Full Text | Google Scholar

Crotty, P., Schult, D., and Segall, K. (2010). Josephson junction simulation of neurons. Phys. Rev. E 82:011914.

PubMed Abstract | Google Scholar

Ermentrout, G. B., and Terman, D. H. (2010). “The hodgkin–huxley equations,” in Mathematical Foundations of Neuroscience, eds S. S. Antman, J. E. Marsden, and S. L. Wiggins (New York, NY: Springer).

Google Scholar

Ferraina, S., Paré, M., and Wurtz, R. H. (2002). Comparison of cortico-cortical and cortico-collicular signals for the generation of saccadic eye movements. J. Neurophysiol. 87, 845–858. doi: 10.1152/jn.00317.2001

PubMed Abstract | CrossRef Full Text | Google Scholar

Furber, S. (2016). Large-scale neuromorphic computing systems. J. Neural Eng. 13:051001. doi: 10.1088/1741-2560/13/5/051001

PubMed Abstract | CrossRef Full Text | Google Scholar

Furber, S., and Temple, S. (2007). Neural systems engineering. J. R. Soc. Interface 4, 193–206.

PubMed Abstract | Google Scholar

Goltsman, G. N., Okunev, O., Chulkova, G., Lipatov, A., Semenov, A., Smirnov, K., et al. (2001). Picosecond superconducting single-photon optical detector. Appl. Phys. Lett. 79, 705–707. doi: 10.1063/1.1388868

CrossRef Full Text | Google Scholar

Izhikevich, E. M. (2004). Which model to use for cortical spiking neurons. IEEE Trans. Neural Netw. 15, 1063–1070. doi: 10.1109/tnn.2004.832719

PubMed Abstract | CrossRef Full Text | Google Scholar

Jeong, H., and Shi, L. (2018). Memristor devices for neural networks. J. Phys. Appl. Phys. 52:023003.

Google Scholar

Katam, N. K., and Pedram, M. (2018). Logic optimization, complex cell design, and retiming of single flux quantum circuits. IEEE Trans. Appl. Supercond. 28, 1–9. doi: 10.1109/tasc.2018.2856833

CrossRef Full Text | Google Scholar

Kerman, A. J., Dauler, E. A., Keicher, W. E., Yang, J. K. W., Berggren, K. K., Gol’tsman, G., et al. (2006). Kinetic-inductance-limited reset time of superconducting nanowire photon counters. Appl. Phys. Lett. 88:111116. doi: 10.1063/1.2183810

CrossRef Full Text | Google Scholar

Kerman, A. J., Yang, J. K. W., Molnar, R. J., Dauler, E. A., and Berggren, K. K. (2009). Electrothermal feedback in superconducting nanowire single-photon detectors. Phys. Rev. B 79:100509.

Google Scholar

Li, S., Kang, W., Huang, Y., Zhang, X., Zhou, Y., and Zhao, W. (2017). Magnetic skyrmion-based artificial neuron device. Nanotechnology 28:31LT01. doi: 10.1088/1361-6528/aa7af5

PubMed Abstract | CrossRef Full Text | Google Scholar

Likharev, K. K., and Semenov, V. K. (1991). RSFQ logic/memory family: a new josephson-junction technology for sub-terehertz-clock-frequency digital systems. IEEE Trans. Appl. Supercond. 50:1.

Google Scholar

Marsili, F., Najaf, F., Dauler, E., Bellei, F., Hu, X., Csete, M., et al. (2011). Single-photon detectors based on ultranarrow superconducting nanowires. Nano Lett. 11, 2048–2053. doi: 10.1021/nl2005143

PubMed Abstract | CrossRef Full Text | Google Scholar

Matsumoto, R., Lequeux, S., Imamura, H., and Grollier, J. (2019). Chaos and relaxation oscillations in spin-torque windmill spiking oscillators. Phys Rev Appl. 11:044093.

Google Scholar

McCaughan, A. N., and Berggren, K. K. (2014). A superconducting-nanowire three-terminal electrothermal device. Nano Lett. 14, 5748–5753. doi: 10.1021/nl502629x

PubMed Abstract | CrossRef Full Text | Google Scholar

McCaughan, A. N., Toomey, E. A., Schneider, M., Berggren, K. K., and Nam, S. W. (2018). A kinetic-inductance-based superconducting memory element with shunting and sub-nanosecond write times. Supercond. Sci. Technol. 32:015005. doi: 10.1088/1361-6668/aae50d

CrossRef Full Text | Google Scholar

Merolla, P. A., Arthur, J. V., Alvarez-Icaza, R., Cassidy, A. S., Sawada, J., Akopyan, J., et al. (2014). A million spiking-neuron integrated circuit with a scalable communication network and interface. Science 345, 668–673. doi: 10.1126/science.1254642

PubMed Abstract | CrossRef Full Text | Google Scholar

Mukhanov, O., Semenov, V., and Likharev, K. (1978). Ultimate performance of the RSFQ logic circuits. IEEE Trans. Magn. 23, 759–762. doi: 10.1109/tmag.1987.1064951

CrossRef Full Text | Google Scholar

Mukhanov, O. A. (2011). Energy-efficient single flux quantum technology. IEEE Trans. Appl. Supercond. 21, 760–769. doi: 10.1038/s41598-019-46595-w

PubMed Abstract | CrossRef Full Text | Google Scholar

Murphy, A., Averin, D. V., and Bezryadin, A. (2017). Nanoscale superconducting memory based on the kinetic inductance of asymmetric nanowire loops. New J. Phys. 19:063015. doi: 10.1088/1367-2630/aa7331

CrossRef Full Text | Google Scholar

Querlioz, D., Bichler, O., and Gamrat, C. (2011). “Simulation of a memristor-based spiking neural network immune to device variations,” in Proceedings of the 2011 International Joint Conference on Neural Networks, San Jose, CA.

Google Scholar

Schneider, M. L., Donnelly, C. A., Russek, S. E., Baek, B., Pufall, M. R., Hopkins, P. F., et al. (2018). Ultralow power artificial synapses using nanotextured magnetic Josephson junctions. Sci. Adv. 4:e1701329. doi: 10.1126/sciadv.1701329

PubMed Abstract | CrossRef Full Text | Google Scholar

Shainline, J. M. (2019). “Fluxonic processing of photonic synapse events,” in Proceedings of the IEEE Journal of Selected Topics in Quantum Electronics (Piscataway, NJ: IEEE).

Google Scholar

Shainline, J. M., Buckley, S. M., McCaughan, A. N., Chiles, J., Mirin, R. P., Nam, S. W., et al. (2018). Circuit designs for superconducting optoelectronic loop neurons. J. Appl. Phys. 124:152130. doi: 10.1063/1.5038031

CrossRef Full Text | Google Scholar

Sourikopoulos, I., Hedayat, S., Loyez, C., Danneville, F., Hoel, V., Mercier, E., et al. (2017). A 4-fJ/Spike artificial neuron in 65 nm CMOS technology. Front. Neurosci. 11:123. doi: 10.3389/fnins.2017.00123

PubMed Abstract | CrossRef Full Text | Google Scholar

Takeuchi, N., Ozawa, D., Yamanashi, Y., and Yoshikawa, N. (2013). An adiabatic quantum flux parametron as an ultra-low-power logic device. Supercond. Sci. Technol 26:035010. doi: 10.1088/0953-2048/26/3/035010

CrossRef Full Text | Google Scholar

Thomas, A. (2013). Memristor-based neural networks. J. Phys. Appl. Phys. 46:093001.

Google Scholar

Toomey, E., Onen, M., Colangelo, M., Butters, B. A., McCaughan, A. N., and Berggren, K. K. (2019). Bridging the gap between nanowires and josephson junctions: a superconducting device based on controlled fluxon transfer. Phys. Rev. Appl. 11:034006.

Google Scholar

Toomey, E., Zhao, Q.-Y., McCaughan, A. N., and Berggren, K. K. (2018). Frequency pulling and mixing of relaxation oscillations in superconducting nanowires. Phys. Rev. Appl. 9:064021.

Google Scholar

Zhao, Q.-Y., McCaughan, A. N., Dane, A. E., Berggren, K. K., and Ortlepp, T. (2017a). A nanocryotron comparator can connect single-flux-quantum circuits to conventional electronics. Supercond. Sci. Technol. 30:044002. doi: 10.1088/1361-6668/aa5f33

CrossRef Full Text | Google Scholar

Zhao, Q.-Y., Santavicca, D. F., Zhu, D., Noble, B., and Berggren, K. K. (2018a). A distributed electrical model for superconducting nanowire single photon detectors. Appl. Phys. Lett. 113:082601. doi: 10.1063/1.5040150

CrossRef Full Text | Google Scholar

Zhao, Q.-Y., Toomey, E. A., Butters, B. A., McCaughan, A. N., Dane, A. E., Nam, S. W., et al. (2018b). A compact superconducting nanowire memory element operated by nanowire cryotrons. Supercond. Sci. Technol. 31:035009. doi: 10.1088/1361-6668/aaa820

CrossRef Full Text | Google Scholar

Zhao, Q.-Y., Zhu, D., Calandri, N., Dane, A. E., McCaughan, A. N., Bellei, F., et al. (2017b). Single-photon imager based on a superconducting nanowire delay line. Nat. Photonics 11, 247–251. doi: 10.1038/nphoton.2017.35

CrossRef Full Text | Google Scholar