Brain‐Inspired Synaptic Resistor Circuits for Self‐Programming Intelligent Systems

Unlike artificial intelligent systems based on computers, which need to be preprogrammed for specific tasks, restricting their functions to their preprogrammed ranges, the human brain does not need to be preprogrammed, and has general intelligence to create new tactics in complex and erratic environments. The basic element in the brain, a synapse, has the function to process and learn from signals in real time by following Hebb's rule, which is a critical function missing from the transistor, the basic device in computers. In this work, a computing circuit based on synaptic resistors (synstors) with signal processing and Hebbian learning functions is modeled and analyzed. A synstor circuit emulates a neurobiological network to concurrently execute signal processing and learning algorithms in parallel mode, does not need to be preprogrammed, and has the capability to optimize and create new algorithms in complex and erratic environments with speed and energy efficiency significantly superior to those of existing computing circuits. The synstor circuit can potentially circumvent the fundamental limitations of existing computing circuits, leading to a new computing platform with real‐time self‐programming functionality and general intelligence in complex and erratic environments.

presynaptic and the nth postsynaptic neuron, and induces a current in the nth postsynaptic neuron, [16] I nm ¼ κÃðw nm x m Þ, where w nm denotes the synaptic weight (conductance), κ denotes a temporal kernel function, and κ Ã w nm x m ð Þ¼ ∫ t 0 κ t À t 0 ð Þw nm t 0 ð Þx m t 0 ð Þdt 0 represents the temporal convolution between κ and w nm x m . For signal processing, a spatiotemporal wave of voltage pulses in M presynaptic neurons, x, induces a collective current via synapses in the N postsynaptic neurons, where w denotes a ðw nm Þ NÂM matrix, x denotes the ðx m Þ MÂ1 vector, and I denotes the ðI n Þ NÂ1 vector. The current IðtÞ induces a train of voltage pulses, yðtÞ, in the N postsynaptic neurons with y as the ðy n Þ NÂ1 vector. When a voltage pulse is fired in the postsynaptic neuron (y n 6 ¼ 0), the postsynaptic current in the neuron, I n ¼ 0. x m and y n voltage pulses have the same amplitudes. The learning rule in neurobiological networks was originally postulated by Hebb, [5] and spike-timing-dependent plasticity (STDP) was discovered experimentally. [6] A synapse is modulated by voltage pulses firing concurrently in pre-and postsynaptic neurons connected by the synapse by following the learning rule, w : nm ¼ αz n x m , where α denotes the conductance modification coefficient and z n is a function of y n . For Hebbian learning, z n ¼ y n , and w : nm ¼ αy n x m ; for anti-Hebbian learning, z n ¼ Ày n , and w : nm ¼ Àαy n x m . For the STDP and anti-STDP learning rules, z n ðtÞ is a function of the timing difference between x m and y n pulses (Equation S1, Supporting Information). In a neural network, the synaptic weight matrix, w, is modified by the spatiotemporal waves of voltage pulses in the presynaptic neurons, x, and postsynaptic neurons, z, for learning. [17] w where z ⊗ x represents the outer product between z and x. Following Equation (2), when z n · x m ¼ 0 (e.g., x m 6 ¼ 0 and z n ¼ 0 for signal processing), w : nm ¼ 0; i.e., w nm remains nonvolatile for memory. By integrating the analog convolutional signal processing, learning, and memory functions in a single synapse, the brain concurrently executes the signal processing (Equation (1)) and learning (Equation (2)) algorithms in a neural network in analog parallel mode with an estimated speed (%10 16 OPS) [15] comparable to the speed (%10 17 floating-point OPS) of the fastest supercomputer, Summit, [9] but consumes much less power (%30 W) than the supercomputer (%10 7 W), and has much smaller volume (%10 À3 m 3 ) than the supercomputer (%10 4 m 3 ). When synapses receive and process signals, they are simultaneously modified in a parallel learning process to dynamically optimize and create new tactics in the human brain. The unique real-time learning function in each synapse facilitates self-programming capability in the human brain and its general intelligence in complex and erratic environments.
Novel neuromorphic devices, such as floating-gate transistors, [18,19] ferroelectric transistors, [20] memristors, [21,22] and phase change memory, [23,24] have been developed to emulate synapses by integrating logic and memory functions in a single device. By circumventing the data transmissions between logic and memory, neuromorphic circuits based on these devices processed signals with energy efficiencies ($10 10 À 10 14 OPS=W) [18][19][20]22,23] significantly higher than those of transistor-based neuromorphic circuits ($10 10 À 10 12 OPS=W). [2,5,[10][11][12][13]25,26] Although learning algorithms such as STDP were executed on individual devices by applying tailored voltage signals, [19,21] learning algorithms needed to be executed in external transistor-based computing circuits to derive desirable device conductances; then the devices were preprogrammed to the conductance values in iterative writing and reading processes with much lower speeds ($10 2 À 10 5 OPS) and energy efficiencies (<10 10 OPS=W) than those for signal processing. [19,21] To avoid the change of conductance after the writing process, the magnitudes of voltage pulses for signal processing were decreased below the magnitudes of voltage pulses for writing. When the signal processing algorithm was executed in the circuits, the writing process was interrupted, and vice versa. [18][19][20][21][22]23] Intrinsically, these neuromorphic devices lack the synaptic function to process and learn from signals with the same magnitude; therefore, the circuits based on these devices cannot compute signal processing and learning algorithms (Equation (1) and (2)) concurrently, and lack the real-time learning and self-programming functionality of the human brain and its general intelligence in complex and erratic environments.
Recently we have developed a synaptic resistor, abbreviated as synstor hereinafter, to emulate a synapse to process and learn from voltage pulses with the same magnitudes. [27] By transmitting spatiotemporal waves of voltage pulses with the same I s m denotes a current flowing into the mth presynaptic (input) neuron, x m denotes input voltage pulses triggered from the mth presynaptic (input) neuron, I n denotes a current flowing through synapses (synstors) into the nth postsynaptic (output) neuron, y n denotes voltage pulses from the nth postsynaptic (output) neuron, and z n denotes feedback voltage pulses from the nth (output) neuron. amplitude in a synstor circuit, the spatiotemporal convolutional signal processing (Equation (1)) and Hebbian learning algorithms (Equation (2)) can be executed concurrently in parallel analog mode. In the synstor circuit, the optimal synstor conductances are not derived by computing learning algorithms based on collected data, and iterative writing and reading processes, but achieved by a self-programming process via real-time learning. A 4 Â 2 crossbar synstor circuit was demonstrated to execute speech signal processing and compute learning algorithms concurrently in analog parallel mode with an energy efficiency of %1.6 Â 10 17 OPS=W, [27] which is higher than the energy efficiencies of the human brain (%10 15 OPS=W), [15] computing circuits based on transistors (%10 10 À 10 12 OPS=W), [2,5,9,[11][12][13]26] and other neuromorphic devices (%10 10 À 10 14 OPS=W). [18][19][20]22,23] In this study, we focus on a theoretical model of synstor circuits that can emulate a neurobiological network to concurrently calculate signal processing (Equation (1)) and learning algorithms (Equation (2)) in parallel analog mode. The computing mechanism, scale, speed, energy efficiency, performance density, and limitations of the synstor circuit will be analyzed in comparison with other computing circuits and neurobiological networks. A mathematical model will be established to describe and analyze the real-time learning process and self-programming function of the synstor circuits to optimize and create new algorithms in complex and erratic environments.

Signal Processing and Learning in Synstors
Previously we have reported an electronic device, the synstor, to emulate a synapse. [27] A synstor is composed of a semiconducting carbon nanotube (CNT) channel which forms Schottky contacts with Al input and output electrodes as a resistor, and a recessed TiO 2 charge storage layer embedded in a HfO 2 dielectric layer sandwiched between an Al reference electrode and the CNT channel as a capacitor ( Figure S1, Supporting Information). The reference electrode is always electrically grounded during the operation. For signal processing (Figure S1a, Supporting Information), a synstor processes a series of voltage pulses, xðtÞ, on its input electrode by charging the capacitor during the pulses, and discharging the capacitor after the pulses, and triggering a current via the resistor, I t ð Þ ¼ κ Ã wx ð Þ, on its grounded output electrode (z ¼ 0) as a convolution of xðtÞ and the product of its conductance, w, and a kernel function κðtÞ. As shown in Figure S1b, Supporting Information, when a series of paired xðtÞ and zðtÞ voltage pulses with the same amplitude (i.e., x ¼ z) are applied on the synstor simultaneously, w is modified by following the Hebb's learning rule, w : ¼ αzx, where α is a nonlinear function of the amplitudes and numbers of x and z pulses. The paired negative (positive) pulses generate a potential difference between the channel and charge storage layer to increase (decrease) the electronic charge stored in the charge storage layer, which in turn attracts (repels) the holes in the semiconducting channel to increase (decrease) its conductance with α > 0 (α < 0). Otherwise, when a synstor experiences x and z pulses under the condition xz ¼ 0, the x or z potential mainly drops beyond the charge storage layer and reference electrode, and the magnitudes of the potential differences between the channel and the recessed charge storage layer are below the threshold values to modify the charge stored in the charge storage layer; thus w : ¼ 0 for nonvolatile memory. The analog convolutional signal processing, Hebbian learning, and nonvolatile memory functions have been integrated in a single synstor to emulate a synapse to process and learn from voltage pulses with the same magnitudes, which facilitates the concurrent signal processing and learning in synstor circuits.

Concurrent Signal Processing and Learning in Synstor Circuits
A circuit composed of M Â N synstors connected with M input and N output neuron circuits is shown in Figure 1. For signal processing, sensory or output signals from the previous layer of circuits, I S , are transmitted to M input neuron circuits, C i , to trigger a wave of voltage pulses, x, input to the circuit. x induces a collective current flowing from synstors into the N output neuron circuits, I ¼ κ Ã ðwxÞ (Equation (1)), where w denotes a ðw nm Þ NÂM matrix of synstor conductances, I denotes the ðI n Þ NÂ1 vector with I n as the current flowing into n th output neuron circuit, and κ denotes a convolutional kernel function of the synstors. The current IðtÞ flows into N output neuron circuits, C o , which generates forward-propagating output voltage pulses, yðtÞ, and back-propagating feedback voltage pulses, zðtÞ, on the N output electrodes connected with the output neuron circuits. When a voltage pulse is fired in the nth output electrode (z n 6 ¼ 0), the nth output electrode is disconnected from the neuron circuit connected with the output electrode, and thus the current flowing into the neuron circuit, I n ¼ 0. For learning, the synstor conductance matrix, w, is modified by the spatiotemporal waves of voltage pulses, x, on the input electrodes and, z, on the output electrodes by following Equation (2), w In the circuit, a synstor connected with the mth input and nth output neuron circuits experiences various combinations of x m and z n voltage pulses: 1) When x m 6 ¼ 0 and z n ¼ 0, a current I mn t ð Þ ¼ κ Ã ðw nm x m Þ 6 ¼ 0 (Equation (1)) is triggered via the synstors connected with the nth output neuron circuit for signal processing, and w : nm ¼ αz n x m ¼ 0 (Equation (2)) for learning. 2) When x m ¼ z n 6 ¼ 0, I mn ¼ 0 for signal processing, and w : nm ¼ αz n x m 6 ¼ 0 for learning. 3) Under all other conditions including x m ¼ z n ¼ 0, and z n 6 ¼ 0 under x m ¼ 0, then I mn ¼ 0 for signal processing, and w : nm ¼ 0 for learning. Therefore, each synstor in the circuit simultaneously processes x and learns from x and z, and the signal processing algorithm IðtÞ ¼ κ Ã ðwxÞ (Equation (1)) and the learning algorithm w (2)) are executed concurrently in the synstor circuit without interrupting each other.

Self-Programming in Synstor Circuits
The learning algorithm, w : ¼ αz ⊗ x (Equation (2)), implemented in a synstor circuit can be viewed as a generic Hebb's learning rule. In principle, all major machine learning algorithms, including unsupervised, supervised, and reinforcement learning, can be implemented in synstor circuits based on w : ¼ αz ⊗ x by setting z ¼ f ðw, x, yÞ. [28] The Hebbian or anti-Hebbian learning www.advancedsciencenews.com www.advintellsyst.com algorithm can be implemented in the synstor circuit by setting z ¼ y. When an output voltage pulse, y n ðtÞ ¼ δðt À t n Þ, is triggered from the nth output neuron circuit at t ¼ t n , a negative (positive) feedback pulse, z n ðtÞ ¼ δðt À t n Þ, is triggered from the nth output neuron circuit simultaneously at t ¼ t n . Substituting z in Equation (2) by y yields w : ¼ α y ⊗ x, with α > 0 (α < 0) for Hebbian (anti-Hebbian) learning. The STDP or anti-STDP learning algorithms can be implemented in the synstor circuit by setting z ¼ y Ãθ (Equation S1, Supporting . When an output voltage pulse, y n ðtÞ ¼ δðt À t n Þ, is triggered from the nth output neuron circuit at t ¼ t n , a train of positive (negative) feedback pulses with a pulse firing rate proportional to e ðtÀt n Þ=τ À =τ À under t < t n and a train of negative (positive) feedback pulses with a pulse firing rate proportional to e ÀðtÀt n Þ=τ þ =τ þ under t > t n are triggered from the nth output neuron circuit on the output electrode to implement STDP (anti-STDP) algorithms. By substituting z ¼ y Ãθ in Equation (2), with α > 0 for STDP learning algorithm, and α < 0 for anti-STDP learning algorithm. Although more advanced machine learning algorithms can be implemented by setting z ¼ f ðw, x, yÞ, practically the computations of complex algorithms involving the iterative computation, memory, writing, and reading processes of the synstor conductance matrix, w, using external transistor-based computing circuits with low speeds and energy efficiencies [18][19][20][21][22]23] should be avoided in the learning process for synstor circuits.
In the synstor circuit,ŵ is not derived by computing learning algorithms offline based on collected data, but spontaneously transformed in a self-programming process via real-time learning. In the learning process, the feedback pulses, z, are set as : the time constants τ þ > 0 and τ À > 0, the function θ þ t ð Þ > 0 and θ À t ð Þ > 0, and μ ¼ 1 or À1. The averageθ over learning period T, hθi ¼ 0, and the average z over learning period T, hzi ¼ 0, and z ¼ z Àz ¼ z.ŷ can be set in different learning algorithms. For example,ŷ ¼ 0 in Hebbian and STDP learning. In supervised learning,ŷ represents the desired value of y; in unsupervised learning,ŷ represents the converged value of y; and in reinforcement learning,ŷ is set as zero. To generate feedback pulses with z n ¼ ðy n Àŷ n Þ Ãθ and μ ¼ 1, when a y n pulse, but noŷ n pulse, is triggered at the moment t ¼ t n , a train of positive feedback pulses with a pulse firing rate proportional to θ À t À t n ð Þwithin the time window t n À τ À < t < t n , and a train of negative feedback pulses with a pulse firing rate proportional to θ þ t À t n ð Þwithin the time window t n < t < t n þ τ þ are triggered from the nth output neuron circuit on the output electrode. When aŷ n pulse, but no y n pulse, is triggered at the moment t ¼ t 0 n , a train of negative feedback pulses with a pulse firing rate proportional to θ À t À t 0 n ð Þ within the time window t 0 n À τ À < t < t 0 n and a train of positive feedback pulses with a pulse firing rate proportional to θ þ t À t 0 n ð Þ within the time window t 0 n < t < t 0 n þ τ þ are triggered from the nth output neuron circuit on the output electrode. The z pulses with the voltage polarities described above with μ ¼ 1 will lead to α < 0 in the learning process; z pulses with their voltage polarities opposite to those described above with μ ¼ À1 will lead to α > 0 in the learning process.

Self-Programming Process and Objective Functions
In an open-loop synstor circuit as shown in Figure 2a, current signals from other circuits or sensors, I s , are processed by input neuron circuits to trigger input voltage pulses, x, which induce currents, I, flowing through the synstor circuit. I flows into output neuron circuits to generate output voltage pulses, y, and feedback voltage pulses z ¼ ðy ÀŷÞ Ãθ (Equation (3)), wherê y represents the desired value of y in supervised learning, and the converged value of y in unsupervised learning. w is modified by x and z by following Equation (2) programming process. The goal of the self-programming process is to minimize an objective function (Equation S6, Supporting Information) where hw : i denotes average w : , hwi denotes average w, and hŵi denotes averageŵ over learning period T. β ∘ hwi À hŵi ð Þdenotes the Hadamard product between β and hwi À hŵi, and β denotes a ðβ nm Þ NÂM matrix with β nm ¼ Àαg δw ¼ O½ðw ÀŵÞ 2 represents the higher order terms, ðw ÀŵÞ k , with k ≥ 2. When w approacheŝ w, O½ðw ÀŵÞ 2 can be omitted. In the self-programming process, the change of w leads to the change of output signals y and objective function F defined in Equation (4). Based on Equation (5), the average change rate of F can be expressed as (Equation S7, Supporting Information) , and O½ðw ÀŵÞ 3 represents the higher order terms, ðw ÀŵÞ k , with k ≥ 3.
In a closed-loop synstor circuit as shown in Figure 2b, the synstor circuit is connected with an external system or another circuit via sensors and actuators. Output signals y from the synstor www.advancedsciencenews.com www.advintellsyst.com circuit are transmitted to actuators to trigger actuation signals a, which modify the states of the system s. s is detected by sensors to generate currents I s , which are transmitted to the input neuron circuits to trigger input signals x. x induces currents I flowing through the synstor circuit and collected by output neuron circuits to generate output voltage pulses y, and feedback voltage pulses z ¼ y Ãθ (Equation (3) withŷ ¼ 0). Concurrently w is modified by x and z by following Equation (2), The goal of the self-programming process is to modify s toward the desired state of the system,ŝ, and an objective function can be defined as When s ¼ŝ, w reaches an equilibrium valueŵ. By substituting z in Equation (2) In the selfprogramming processes, the change of w leads to the change of output signals y, which modifies s and the objective function F defined in Equation (7). The dynamic change of F in the selfprogramming process can also be described by Equation (6), hF In a self-programming process of a closed-loop or open-loop synstor circuit, when δF satisfies δF < βhFi, hF : i ¼ ÀβhFi þ δF < 0, hFðwÞi represents a Lyapunov function, and is asymptotically decreased toward its dynamic equilibrium value F e , leading hwi to be modified toward hŵi in the selfprogramming process; when δF ¼ βhFi, hF : i ¼ 0, hFi reaches its dynamic equilibrium value F e ¼ δF=β under hwi ¼ hŵi ¼ argmin hŵi hFi (Theorem 1 and 2, Supporting Information).

Simulation of Self-Programming Dynamic Process
A closed-loop synstor circuit connected with an external system is simulated by a MATLAB software and is shown in Figure 3a as an example of the self-programming process. The goal of the self-programming process is to modify state s with an arbitrary unit in a system (Figure 3b) toward the desired state,ŝ ¼ 0. s is detected by sensors to generate a sensory current I s ¼ s 0 when s ≥ 0, and I s ¼ 0 jsj when s < 0. I s triggers input pulses x with amplitudes of AE1V and a duration of 2.5 μs (Figure 3c) from input integrate-and-fire neuron circuits (Section S4 and Figure S2, Supporting Information) with a capacitance C IF ¼ 50 pF, a leakage current I L ¼ 1 nA, and a threshold voltage V IF th ¼ 0.3V. x induces currents, I (Figure 1), by following Equation (1), I ¼ wx, which flows into output integrate-and-fire neuron circuits to generate feedback pulses, z (Figure 3d), with amplitudes of AE1 V and a duration of 2.5 μs, and output pulses, y (Figure 3e), with an amplitude of 1 V and a duration of 2.5 μs. The output integrate-and-fire neuron circuits (Section S4 and Figure 2. a) For signal processing in a synstor circuit, currents from previous layer of circuits or sensors, I s , are transmitted to input neuron circuits to trigger input voltage pulses, x. x induces currents, I, flowing from synstors into the output neuron circuits by following Equation (1), I ¼ κ Ã ðwxÞ, where w denotes a matrix of synstor conductances. The currents I flow into output neuron circuits, which generate forward-propagating output voltage pulses, y, and back-propagating feedback voltage pulses, z, by following Equation (3), z ¼ ðy ÀŷÞ Ãθ. For learning, w is modified by x and z by following A synstor circuit is integrated with a system or another circuit. Output signals, y, are transmitted to actuators to trigger actuation signals, a, which modify the states of the system, s. s is detected by sensors to generate currents, I s , flowing into the input neuron circuits. The synstor circuit is operated in the same way as described before.
www.advancedsciencenews.com www.advintellsyst.com Figure S2, Supporting Information) have a capacitance C IF ¼ 4 nF, a leakage current I L ¼ 0, and a threshold voltage When a y pulse is triggered, a 1V z pulse is triggered simultaneously, and a À1 V z pulse is triggered at 2.5 μs after the y pulse is triggered. y pulses trigger actuation pulses a (Figure 3f ) with an amplitude of 1 V and a duration of 2.5 μs by following a ¼ y 1 À y 2 , and the system state s is modified by a with s : ¼ g sa a þ n s , where g sa represents a modification coefficient, and n s represents the random perturbation from environment. When jsj ≥ 12, it is assumed that the system reaches its boundary, is out of control, and the self-programming process fails. When the system is modified, the objective function of the system, F ¼ 1 2 s 2 (Figure 3g), is also modified accordingly. The synstor conductances are set to random values with 0 ≤ w nm ≤ 20 nS before the self-programming process, and the synstor conductances w nm (Figure 3h) are modified by x and z following Equation (2), w : ¼ α z ⊗ x within the range of 0 ≤ w nm ≤ 20 nS in the self-programming process. In the simulation shown in Figure 3, α ¼ 3 nSV À2 s À1 , g sa ¼ 1au, and À0.25 ≤ n s ≤ 0.25 au: The dynamic self-programming process shown in Figure 3 is analyzed, and the change of the average objective function hFi is shown versus time t and the average synstor conductances hw nm i over a moving time window T in Figure 3i. The change of w nm Figure 3. a) A simulated closed-loop system including a synstor circuit and an external system. In the simulated synstor circuit and system, b) s, the state of the system (arbitrary unit), c) x 1 and x 2 , the input pulses on the input electrodes 1 and 2 (unit: V), d) z 1 and z 2 , the feedback pulses on the output electrodes 1 and 2 (unit: V), e) y 1 and y 2 , the output pulses from the output neuron circuits 1 and 2 (unit: V), f ) a, the actuation pulses to the system (unit: V), g) F, the objective function of the system (arbitrary unit), h) w 11 , w 12 , w 21 , and w 22 , the conductances of the synstors in the circuit (unit: nS), are shown versus time t. i) The average objective function, hFi (arbitrary unit), and the difference between the average synstor conductances and their optimal conductances, hw 11 i À hŵ 11 i (red solid line), hw 12 i À hŵ 12 i (magenta solid line), hw 21  www.advancedsciencenews.com www.advintellsyst.com leads to the change of F, which is also influenced by the random environment noise n s . When the change of w nm leads to the change of F in a learning period, the covariance between F and w nm hF,w nm i 6 ¼ 0; when w nm does not change beyond the learning period, hF,w nm i ¼ 0. Learning periods within which hF,w nm i 6 ¼ 0 and the minimal length of moving time window T that satisfies hF : i ≤ 0 or hF : i ≥ 0 within each learning period are identified in the analysis. Within a learning period, if hF : i < 0, hŵ nm i is identified as hw nm i at the end of a learning period when hF,w nm i ¼ 0, hF : i ¼ 0, and hFi approaches its minimal equilibrium value F e (F e may not be equal to zero due to the environment noise or other synaptic conductances w n 0 m 0 ). After hŵ nm i is identified, hw nm i À hŵ nm i and hFi within the learning period are shown versus time t in Figure 3i. Within each learning period, hFi decreases asymptotically versus time t with hF : i < 0, and hw nm i is gradually modified toward hŵ nm i; at the end of the learning period, hw nm i ¼ hŵ nm i, hF : i ¼ 0, and hFi reaches an equilibrium value F e . hFi represents a Lyapunov function within each learning period. The system changes dynamically by the random environment noise. When the system is perturbed, and F is increased from F e above a threshold value,ŵ is also shifted accordingly, and a learning period will spontaneously be triggered in the self-programming process to modify w toward w and F toward F e in the dynamically changing environment.
The self-programming processes in the synstor circuit are simulated and analyzed statistically by setting the initial synstor conductance matrix w to random values within the range of 0 ≤ w nm ≤ 20 nS, and the system state s within the range of 4 ≤ jsj ≤ 8 au in multiple simulations. s is also influenced by random environment noise, with 0 ≤ jn s j ≤ 0.25 au: w is modified by following the learning rule, and hŵi is extrapolated in selfprogramming processes. The objective function F is modified as the function of w and the random environment noise, and hFi is shown versus hwi À hŵi in multiple self-programming processes in Figure 4a. After w is set to its initial value wð0Þ, w is modified towardŵ to reduce hFi asymptotically, and hFi reaches its equilibrium value F e when hwi ¼ hŵi. When w nm is set to an initial value that triggers actuations to decrease F toward its equilibrium value F e , the initial w nm is close tô w nm , w nm ð0Þ Àŵ nm % 0, and hFi is shown versus w nm ðtÞ Àŵ nm as lines with large slopes in Figure 4a. When w nm is set to an initial value that is far away fromŵ nm , jw nm ð0Þ Àŵ nm j has a large value, and hFi is shown versus w nm ðtÞ Àŵ nm as lines with small slopes in Figure 4a. hFi is best fitted as a parabolic function of jhwi À hŵij, and is shown versus jhwi À hŵij in Figure 4b. As shown in Figure 4b, the synstor conductance matrix w does not need to be preprogrammed, and can be spontaneously modified towardŵ, decreasing hFi toward F e in all of the 100 simulated self-programming processes.

The Stability and Equilibrium States of Self-Programming Processes
The self-programming processes in the synstor circuit are simulated and analyzed statistically by setting the initial synstor conductance matrix w and the system state s to random values, as described earlier, the conductance modification coefficient α ¼ 0.2, 1, and 10 nSV À2 s À1 , respectively, and the leakage current I L ¼ 0.1nA in the output neuron circuits. The average objective hFi is shown versus hwi À hŵi in multiple self-programming processes in Figure 5a under α ¼ 0.2, Figure 5b under α ¼ 1, and Figure 5c under α ¼ 10. hFi is best fitted as a parabolic function of jhwi À hŵij, and is also shown versus jhwi À hŵij in Figure 5a-c. When α ¼ 0.2 nSV À2 s À1 (Figure 5a), due to the random perturbation from the environment, hF : i > 0 in some of the self-programming processes. Under these conditions, F is not stable, and gradually increases to its upper limit value ($72). When α ¼ 1 nSV À2 s À1 (Figure 5b) and α ¼ 10 nSV À2 s À1 (Figure 5c), hF : i < 0 in all the self-programming processes. Under these conditions, F is stable, and w is gradually modified towardŵ to decrease hFi to its equilibrium values F e .
In a self-programming process, the synstor conductance matrix w is modified by following Equation (5), hw : i ¼ Àβ ∘ hwi À hŵi ð Þþδw, and its solution gives hwi ¼ hŵiÀ ½hwð0Þi À hŵie À∫ t 0 β τ ð Þdτ þ ðδw À hŵi : Þ Ã e À∫ t 0 βðτÞdτ , where hβi represents the speed to modify hwi toward hŵi in a self-programming process. β nm ¼ Àαg y=I n hθ Ã κihx m , x m i increases linearly with respect to the conductance modification coefficient α defined in Equation (2), and α increases exponentially with the magnitudes of x and z pulses. [27] In the simulation of selfprogramming processes, β nm is extrapolated from w : nm and w nm Àŵ nm , β nm ¼ hw : nm , w nm Àŵ nm i=hw nm Àŵ nm , w nm Àŵ nm i. Figure 4. a) The average objective function, hFi (arbitrary unit), in multiple self-programming processes, is plotted versus the differences between the average synstor conductances and their optimal conductances, hwi À hŵi, with the unit of nS and its display azimuth proportional to the simulation number. hw 11 i À hŵ 11 i, hw 12 i À hŵ 12 i, hw 21 i À hŵ 21 i, and hw 22 i À hŵ 22 i are shown by red, magenta, green, and blue lines, respectively, and the initial differences are marked by circles. b) hFi shown in (a) is best fitted and plotted versus hwi À hŵi.
www.advancedsciencenews.com www.advintellsyst.com As shown in Figure 5d, β nm increases linearly with respect to α, and the average hβi can be best fitted by hβi ¼ g βα α þ β 0 with g βα % 0.41 nS À1 V 2 and β 0 % 11 s À1 for 1 nSV À2 s À1 ≤ α ≤ 100 nSV À2 s À1 under the stable conditions. The selfprogramming speed can be increased by increasing α, but it is limited by the dynamic response speed of the simulated system (%10 2 s À1 ). In a synstor circuit and system, hF : i ¼ ÀβhFi þ δF < 0 when βhFi > δF, which is the condition for the stability of the circuit and system. δF is mainly induced by the random perturbation from the environment, and β increases versus α; therefore, βhFi is increased versus α, and satisfies the stability condition βhFi > δF when α is increased above a critical value (α ≳ 1 nSV À2 s À1 as shown in Figure 5b-d). When α is decreased below the critical value (α ≲ 1 nS V À2 s À1 as shown in Figure 5a,d), βhFi < δF, and hF : i > 0, F is unstable. When βhFi > δF and hF : i < 0, hFi is asymptotically decreased to an equilibrium value F e in a self-programming process.
To understand the influence of the conductance modification coefficient α on the equilibrium state and stability, the selfprogramming processes in a synstor circuit are simulated when α changes between 0.1 and 100 nSV À2 s À1 and the rest of the conditions remain the same as described previously. F e is plotted versus α in Figure 5e. When α decreases from 1 nSV À2 s À1 to 0.1 V À2 s À1 , β decreases with decreasing α, leading to hF : i > 0 and unstable F. When α increases from 1 nSV À2 s À1 to 10 2 V À2 s À1 , β increases with increasing α, and hF : i ¼ ÀβF e þ δF also decreases, leading to hF : i < 0 and equilibrium F e . As shown in Figure 5e, the average F e can be best fitted by hF e i ¼ g Fα α þ F 0 e with g Fα % 10 À3 ðauÞ and F 0 e % 0.15ðauÞ for 1 nSV À2 s À1 < α < 10 2 V À2 s À1 . F e increases with increasing α under the stable condition hF : i ≤ 0. β increases linearly with increasing α (Figure 5d), δF ¼ Àg F=w ∘ hðw ÀŵÞ ∘ŵ : iþ O½ w Àŵ ð Þ 3 is mainly induced by the environmental perturbation, and also includes the terms related to w Àŵ and α k with k ≥ 1; therefore, F e ¼ δF=β % F 0 e þ g Fα α þ Oðα 2 Þ near the equilibrium condition, as shown by the fitting line in Figure 5e.
To understand the influence of the leakage currents, I L , in the output neurons on F e , the self-programming processes in a synstor circuit are simulated when I L changes between 1 pA and 1 nA and the rest of the conditions remain the same as described previously. F e is plotted versus I L in Figure 5f. When I L increases from 0.1 to 1 nA, F e increases with increasing I L . As shown in Figure 5f, the average F e can be best fitted by hF e i ¼ g 1 FI ðI L À I 1 L Þ with g 1 FI % 1.5ðauÞ and I 1 L % 0.01 nA for 0.1 nA ≤ I L ≤ 1 nA. When I L decreases from 0.1 nA to 1 pA, F e increases with decreasing I L . As shown in Figure 5f, the Figure 5. The average objective function, hFi, in arbitrary unit (lines), and the best fit to hFi (3D surfaces) in multiple self-programming processes, is plotted versus the differences between the average synstor conductances and their optimal conductances, hwi À hŵi, with the unit of nS and its display azimuth proportional to the simulation number under a) α ¼ 0.2 nSV À2 s À1 , I L ¼ 0.1 nA; b) α ¼ 1 nSV À2 s À1 , I L ¼ 0.1nA; and c) α ¼ 10 nSV À2 s À1 , I L ¼ 0.1nA. hw 11 i À hŵ 11 i, hw 12 i À hŵ 12 i, hw 21 i À hŵ 21 i, and hw 22 i À hŵ 22 i are shown by red, magenta, green, and blue lines, respectively, and the initial differences are marked by circles. d) β representing the speed of self-programming processes is shown by green open cycles in the unit of s À1 versus the conductance modification coefficient α in the unit of nSV À2 s À1 in multiple self-programming processes. The average β is shown by filled blue squares, and best fitted by hβi ¼ g βα α þ β 0 with g βα % 0.41 nS À1 V 2 and β 0 % 11 s À1 . The equilibrium objective function, F e (arbitrary unit), is plotted versus e) the conductance modification coefficient α in the unit of nSV À2 s À1 under I L ¼ 0.1 nA, and f ) the leakage current in the integrate-and-fire neuron circuit I L in the unit of nA under α ¼ 1 nSV À2 s À1 . The green open circles mark F e , the blue filled squares mark average F e , and the red crosses mark the final F under unstable condition. The average F e is fitted by hF e i ¼ g Fα α þ F 0 e with g Fα % 10 À3 ðauÞ and F 0 e % 0.15ðauÞfor 1 nSV À2 s À1 < α < 10 2 V À2 s À1 in (e), and fitted by hF e i ¼ g FI ðI L À I 0 L Þ with g FI % À0.89ðauÞ and I 0 L % 0.67 nA for 1pA ≤ I L ≤ 0.1 nA and g FI % 1.5ðauÞ and I 0 L % 0.01 nA for 0.1 nA ≤ I L ≤ 1nA in (f ). average F e can be best fitted by hF e i ¼ Àg 2 FI ðI L À I 2 L Þ with g 2 FI % 0.89ðauÞ and I 2 L % 0.67 nA for 1 pA ≤ I L ≤ 0.1 nA. When I L increases from 0.1 to 1nA, the firing rate of output pulses y from the output neuron circuits decreases with increasing I L , leading to the decrease of the firing rate of the actuation pulses to control the system state and the increase of F e . When I L decreases from 0.1 nA to 1 pA, the firing rate of output pulses y from the output neuron circuits increases with decreasing I L , leading to the increase of the firing rate of the actuation pulses, the fluctuation of the system state, and the increase of F e . F e is a nonlinear function of I L with an optimal value to minimize F e .
F e is also influenced by the critical parameters in selfprogramming processes, such as the conductance modification coefficient, α, the firing rates and amplitudes of the feedback pulses, z, the upper limit of synstor conductances, transfer functions between the system and synstor circuit, g sa , the gain between the firing rates of output pulses versus input currents, g y=I n , the capacitance of the capacitors, C IF , the leakage currents, I L , the threshold voltages, V IF th , and the saturated firing rates of output pulses in neuron circuits. The critical parameters that influence the equilibrium state and stability condition can be modified and optimized via learning in self-programming processes. In a simulated self-programming process, α and I L do not have fixed values as described previously, but are modified by following the correlative learning rules similar to Equation (2), Δα ¼ k α hα, hFii and ΔI L ¼ k I hI L , hFii, where k α ¼ 1 (au) and k I ¼ 10 (au) denotes the modification coefficients for α and I L respectively, hFi denotes the average objective function during a learning period T ¼ 1 s. hα, hFii denotes the covariance between α and hFi, and hI L , hFii denotes the covariance between I L and hFi. During the self-programming process, when w is continuously modified by following Equation (2), α and I L remain at constant values within each learning period T, and are discretely modified by following the learning rules given before after the end of each learning period. In the self-programming process, the objective function, F, is shown versus time t in Figure 6a. As shown in Figure 6b, α is set to an initial value α 0 ¼ 0.15 nSV À2 s À1 at the beginning of the self-programming process, and is gradually modified to its equilibrium value α e % 0.98 nSV À2 s À1 , which is approximately equal to the optimal value of α % 1 nSV À2 s À1 as shown in Figure 5e. As shown in Figure 6c, I L is set to an initial value I 0 L ¼ 0.9 nA, and is gradually modified to its equilibrium value I e L % 0.14 nA, which is equal to the optimal value of I L % 0.1 nA as shown in Figure 5f, and F is reduced from its initial value F 0 ð Þ ¼ 68 au to its equilibrium value F e % 0.4 au at end of the self-programming process. The simulation indicates that critical parameters that influence the objective function F can also be dynamically modified to their optimal equilibrium values in a self-programming process to minimize F.

Comparisons between Self-Programming Synstor Circuits and Programmable Computing Circuits
In comparison with computers, the equivalent computing operations in an M Â N synstor circuit are approximately equal to 3MN to implement the signal processing algorithm (I ¼ κ Ã wx ð Þ, Equation (1), 2MN for multiplications between w, κ, and x; MN for accumulations), and 3MN to implement the learning algorithm (w : ¼ α z ⊗ x), Equation (2), 2MN outer products between α, x, and z; MN for w modifications). The speed for the synstor circuit to implement 6MN equivalent computing operations for parallel signal processing and learning V c ¼ 6MNf c (8) where f c denotes the operation frequency of the circuit. When voltage pulses are applied on its input or output electrode of an M Â N synstor circuit connected with M input and N output integrate-and-fire neuron circuits, the average power consumption in an M Â N synstor circuit, Figure 6. a) The average objective function, hFi, in arbitrary unit (black line), b) the conductance modification coefficient α in the unit of nSV À2 s À1 (blue solid line), and c) the leakage current in the integrate-and-fire neuron circuit I L in the unit of nA (blue solid line) are plotted versus time t in a selfprogramming process. The green dashed lines in (b) show the optimal value of α ¼ 1 nSV À2 s À1 and the optimal value of I L ¼ 0.1nA. www.advancedsciencenews.com www.advintellsyst.com P c % MNhwiV 2 a D p þ ME p r in þ NE p r out (9) where hwi denotes the average conductance of the synstors, V a denotes the magnitude of pulses, D p denotes the average dutycycle of the pulses, E p denotes the average energy consumption to trigger a pulse from integrate-and-fire neuron circuits, and r in and r out denote the average firing rates of pulses from input and output neuron circuits, respectively. The computing energy efficiency of a synstor circuit is equal to its computing speed (V c ) divided by its power consumption (P c ) The computing energy efficiencies of 10 3 Â 10 3 and 10 4 Â 10 4 synstor circuits operated with operation frequencies f c ranging from 100 Hz to 1 GHz, average synstor conductances hwi ranging from 1 pS to 100 nS, V a ¼ 1 V, D p ¼ 0.01, E p ¼ 1 pJ, r in ¼ r out ¼ 0.01 f c , and energy efficiencies E c ≳ 5 Â 10 16 OPS W À1 are shown versus their computing speeds and power consumptions in Figure 7a. The computing energy efficiencies, speeds, and power consumptions of a 4 Â 2 synstor circuit, [27] the human brain, [15] the Summit supercomputer, [9] Volta V100 GPUs from Nvidia, [10] TPUs from Google, [5] a Stratrix 10 field-programmable gate array (FPGA) from Intel, [11] a TrueNorth neuromorphic circuit from IBM, [12] memristor circuits from UMass/HP, [21] Tianjic neuromorphic Figure 7. a) The computing energy efficiencies of 10 3 Â 10 3 (green) and 10 4 Â 10 4 (blue) synstor circuits operated with operation frequencies ranging from 100 Hz to 1 GHz, average synstor conductances ranging from 1pS to 100 nS, and energy efficiencies E c ≳ 5 Â 10 16 OPSW À1 are shown versus circuit computing speeds and power consumptions. b) The computing energy efficiencies are shown versus circuit operation frequency and average synstor conductance. c) The computing speeds of synstor circuits are shown versus the number of synstors and power consumption in integrated circuits with the number of synstors ranging from 10 2 to 10 9 . The energy efficiencies, computing speeds, power consumption, operation frequencies, average device conductances, and number of devices in a 2 Â 4 synstor circuit (green), the human brain (blue), the Summit supercomputer, Volta V100 GPUs from Nvidia, TPUs from Google, a Stratrix 10 FPGA from Intel, a TrueNorth neuromorphic circuit from IBM, memristor circuits from UMass/HP (signal processing only, learning excluded), Tianjic neuromorphic circuits from Tsinghua University, and a phase change memory circuit from IBM (signal processing only, learning excluded) are also shown in (a), (b), and (c). The energy efficiency of the brain is benchmarked by a blue line at 5 Â 10 14 OPS W À1 in the speed-power plane in (a). The circuits with energy efficiencies less than the energy efficiency of the brain (5 Â 10 14 OPS W À1 ) are marked in red. The ranges of the speeds and power consumption of synstor circuits that operate with energy efficiencies higher than the energy efficiency of the brain are marked by the green areas in the speed-power plane in (a) and the frequency-conductance plane in (b). The ranges of the frequency and the average device conductance of synstor circuits that operate with energy efficiencies higher than the energy efficiency of the brain are marked by the green area in the frequency-conductance plane in (b).
www.advancedsciencenews.com www.advintellsyst.com circuits from Tsinghua University, [13] and a phase change memory circuit from IBM [24] are also shown in Figure 7a for comparison. The speed of the Summit supercomputer exceeds the speed of the human brain, but consumes much more power (%10 7 W) than the human brain (%30 W), and has a much larger volume (%10 4 m 3 ) than the human brain (%10 À3 m 3 ). It is not practical to send a large amount of data to the supercomputer to calculate a learning algorithm remotely in real time; thus, the supercomputer needs to collect "big data" and then calculate the learning algorithm off-site with significant delay with respect to the dynamic changes of the systems and environments. To calculate learning algorithms on-site in real time, a computing circuit needs to be embedded in intelligent systems, consume less power than the human brain (≲30 W), have a computing speed exceeding the speed of the human brain (≳10 16 OPS), and have a computing energy efficiency exceeding the efficiency of the human brain (≳5 Â 10 14 OPS W À1 ). However, the energy efficiencies of the existing electronic computing circuits have not exceeded the efficiency of the human brain ( Figure 7a). A synstor circuit can execute spatiotemporal signal processing (Equation (1)) and correlative learning (Equation (2)) algorithms concurrently with high energy efficiency by circumventing the fundamental computing limitations in existing electronic circuits such as physically separated logic and memory units, data transmission between memory and logic, the execution of the inference and learning algorithms in serial mode in different circuits, and the signal transmissions between the circuits. The equivalent computing energy efficiency of the 4 Â 2 synstor circuit is 1.6 Â 10 17 OPS W À1 , [27] which exceeds the efficiency of the human brain (%5 Â 10 14 OPS W À1 ). As shown in Figure 7a, when synstor circuits are scaled up to 10 8 synstors, the computing energy efficiencies of the synstor circuits are increased up to 5 Â 10 18 OPS W À1 due to the decrease of average power consumptions to trigger pulses from neuron circuits (E p r in =N and E p r out =M in Equation (10)). Based on Equation (10), the computing energy efficiency of synstor circuits also increases with increasing circuit frequency f c and decreasing average device conductance hwi. As shown in Figure 7b, the average conductances of synstors (10 À3 À 10 3 nS) and synapses (10 À3 À 10 À1 nS) are much lower than the average conductances of transistors and memristors (10 4 À 10 6 nS) in existing computing circuits, resulting in the computing energy efficiencies of synstor circuits and the human brain surpassing the energy efficiencies of the existing computing circuits. Computers are operated in serial mode, and the total latency of the serial computations is equal to the sum of the latency of individual computations. The latency is proportional to the individual transistor resistance and needs to be reduced to extremely small values by decreasing the transistor resistance. Synstor circuits and human brains are operated in parallel mode, and the latency of the parallel analog computation is proportional to the total resistance of synstors and synapses connected in parallel with each neuron. The latency can be reduced by increasing the numbers of synstors and synapses connected in parallel (M in a synstor circuit), while keeping the high resistance of synstors and synapses to reduce the power consumption. Based on Equation (10), the computing energy efficiency of synstor circuits can also be increased by increasing the circuit frequency f c from 100 Hz to 1 GHz (Figure 7b). Due to the limit of ion transportation speed in the human brain, it operates at the frequency of %1 kHz. The operation of synstor circuits is based on electron transportation; therefore, the operation frequency and energy efficiency of synstor circuits can be significantly higher than those of the human brain. However, the average power consumption to trigger pulses from neuron circuits (E p r in =N and E p r out =M in Equation (10)) also increases with increasing operation frequency, leading to the saturation of the energy efficiency of synstor circuits at high operation frequency ( Figure 7b). As shown in Figure 7c, the computing speed and power consumption of an M Â N synstor (synapse) circuit increases with increasing scale of the circuit (i.e., MN, the numbers of parallel input/output electrodes; Equation (8) and (9)). With the energy efficiency of the 4 Â 2 circuit demonstrated experimentally (1.6 Â 10 17 OPS W À1 ) [27] and an operation frequency f c ¼ 1 MHz, a 10 3 Â 10 3 synstor circuit has a speed of 6Â10 12 OPS (Figure 7c), which is comparable with the speeds of TPU, GPU, and FPGA transistor-based circuits (%10 12 À 10 14 OPS); a power consumption of %40 μW, much less than the power consumptions of the transistor-based circuits (%40 W); and the number of synstors %10 6 , much less than the number of transistors in the transistorbased circuits (%10 9 À 10 11 ). A 10 5 Â 10 4 synstor circuit has a speed of 6Â10 15 OPS, comparable with the speeds of the human brain (%10 16 OPS) and the Summit supercomputer (% 10 17 OPS); a power consumption of $40 mW, much less than the power consumptions of the human brain (%30 W) and Summit supercomputer (%10 7 W); and the number of synstors %10 9 , much less than the number of devices in the human brain (%10 14 ) and Summit supercomputer (%10 14 ). Synstor circuits with high speed, low power consumption, high energy efficiency, and small circuit scale/volume could be embedded in intelligent systems and powered by batteries to compute signal processing and learning algorithms in the self-programming process.

Conclusion
We have modeled and analyzed a synstor circuit that emulates a neurobiological network to execute signal processing and learning algorithms concurrently in parallel mode. The synstor conductance matrix w in the circuit does not need to be preprogrammed, and can be spontaneously modified toward the optimal matrixŵ, minimizing the average objective function hFi in a selfprogramming process to optimize and create new algorithms in complex and erratic environments. The stability, learning speed, and equilibrium state of the self-programming process are influenced by critical parameters such as the conductance modification coefficient, α, and neuron circuit leakage currents, I L , which can also be optimized in the self-programming process. The computing speed and power consumption of the synstor circuit can be improved by scaling up the circuit, increasing its operational frequency, and reducing synstor conductances. A circuit of 10 6 synstors will have a speed (6Â10 12 OPS) comparable with the speeds (%10 12 À 10 14 OPS) of TPU, GPU, and FPGA circuits with %10 9 À 10 11 transistors, and consume much less power (%40 μW) than the transistor-based circuits (%40 W). A circuit of 10 9 synstors will have a speed (6Â10 15 OPS) comparable with the speeds of the human brain (%10 16 OPS) and the Summit supercomputer (%10 17 OPS), and consumes much less power www.advancedsciencenews.com www.advintellsyst.com ($40 mW) than the human brain (%30 W with %10 14 synapses) and the Summit supercomputer (%10 7 W with %10 14 transistors). There is "plenty of room at the bottom" to scale up synstor circuits with high speed, low power consumption, high energy efficiency, and small circuit scale/volume for a new computing platform that can self-program in real time in arbitrary and erratic environments, embedded in intelligent systems, and powered by batteries.

Supporting Information
Supporting Information is available from the Wiley Online Library or from the author.