A building block for hardware belief networks

Belief networks represent a powerful approach to problems involving probabilistic inference, but much of the work in this area is software based utilizing standard deterministic hardware based on the transistor which provides the gain and directionality needed to interconnect billions of them into useful networks. This paper proposes a transistor like device that could provide an analogous building block for probabilistic networks. We present two proof-of-concept examples of belief networks, one reciprocal and one non-reciprocal, implemented using the proposed device which is simulated using experimentally benchmarked models.

Probabilistic computing is a thriving field of computer science and mathematics and is widely viewed as a powerful approach for tackling the daunting problems of searching, detection and inference posed by the ever increasing amount of "big data" [1][2][3][4][5][6][7][8][9][10][11] . Much of this work, however, is software-based, utilizing standard general purpose hardware that is based on high precision deterministic logic 12 . The building block for this standard hardware is the ubiquitous transistor which has the key properties of gain and directionality that allow billions of them to be interconnected to perform complex tasks. This paper proposes a transistor-like device that could provide an analogous building block for probabilistic logic.
A number of authors [13][14][15] have recognized that the physics of nanomagnets can be exploited for stochastic logic and natural random number generators to replace the complex circuitry that is normally used. However, these are individual stochastic circuit elements within the standard framework of complementary metal oxide semiconductor (CMOS) transistors, which provide the necessary gain and directionality. By contrast, what we are proposing in this paper are networks constructed out of magnet-based stochastic devices that have been individually engineered to provide transistor-like gain and directionality so that they can be used to construct large scale circuits without external transistors (Fig. 1a).
Feynman (1982) alluded to a probabilistic computer based on probabilistic hardware that could efficiently solve problems involving classical probability, contrasting it with a quantum computer based on quantum hardware that could efficiently solve quantum problems. This paper inspired much work on quantum computing, but we would like to draw attention to his description of a probabilistic computer: "… the other way to simulate a probabilistic nature, which I'll call N .. is by a computer C which itself is probabilistic, .. in which the output is not a unique function of the input. … it simulates nature in this sense: that C goes from some .. initial state .. to some final state with the same probability that N goes from the corresponding initial state to the corresponding final state. … If you repeat the same experiment in the computer a large number of times … it will give the frequency of a given final state proportional to the number of times, with approximately the same rate … as it happens in nature. " The possibility of probabilistic computing machines has also been addressed by more recent authors [16][17][18][19][20][21][22] . The primary purpose of this paper is to introduce the concept of a 'transynapse' , a device that can be interconnected in large numbers to build probabilistic computers (Fig. 1).
The transynapse combines a synapse-like function with a transistor-like gain and directionality and in the next section we describe a device that uses the established physics of nanomagnets to implement it. We present a specific design for the transynapse which is simulated using experimentally benchmarked models (Supplementary section 1) for established phenomena to demonstrate the stochastic sigmoid transfer function. Reciprocal and non-reciprocal networks are discussed next where the same models are used to show how transynapses can be used to build either of two fundamentally different class of networks, an Ising like network with symmetric interactions and a non-Ising network with directed interactions. We then present two proof-of-concept examples of belief networks, one reciprocal for Boltzmann machines and the other non-reciprocal for Bayesian networks, implemented using transynapses.

Transynapse: The Building Block
The transynapse consists of a WRITE unit and a READ unit (Figs 1 and 2), electrically isolated from each other. The WRITE unit of transynapse T i sums a set of input signals I IN,j (t), integrates them over a characteristic time scale τ r to obtain a quantity Figure 1. This paper defines a transynapse that can be interconnected to build probabilistic networks as shown schematically. Next section describes a specific transynapse design based on experimentally benchmarked models which are then used to illustrate the use of transynapse networks to solve problems involving Boltzmann machines and Bayesian networks.  23 which provides the required gain, fan-in and fan-out, making use of the established physics of the spin Hall effect (SHE) for the input and the magnetic tunnel junction (MTJ) for the output. (see also refs 32,33) However, instead of the deterministic mode described earlier, we operate it in a probabilistic mode as described next. (b) The WRITE and READ magnets are both initialized along their hard axis and allowed to relax in the presence of an exponentially decaying current (see inset) I IN (t) and decay time parameter τ dec . The outputs obtained from a statistical average of 100 Monte Carlo runs for different input currents all fall on a single universal curve when plotted against Q, the time integrated current weighted by the factor τ − e t/ r . The dashed curves 1 and 2 are obtained for nanomagnets with energy barriers 48 k B T and 12 k B T respectively and are described well by Eq. 4.
Scientific RepoRts | 6:29893 | DOI: 10.1038/srep29893 which determines the mean value of state S of the device through a sigmoidal function of the form where I 0 is a characteristic current. The READ unit produces multiple weighted output currents proportional to the average state S: OUT j ji i , 0 In this paper we use a specific design for this device following that described in Datta et al. 23 which uses an input WRITE unit magnetically coupled to, but electrically isolated from an output READ unit. It provides the required gain, fan-in and fan-out, making use of experimentally benchmarked models (Supplementary section 1) for the established physics of the spin Hall effect (SHE) for the input and the magnetic tunnel junction (MTJ) for the output. However, for transynapse operation, we need to operate it in a probabilistic mode not considered before, where the input and output are not deterministic variables but stochastic ones. This could be done by using nanomagnets with low energy barriers (E b < 5 k B T) that are in the super paramagnetic 24 regime considering long enough programming times. The output (y-axis in Fig. 2b) could then be interpreted as the time averaged magnetization of a single magnet along its easy axis. However, in this paper we use a different approach as explained below.
For the simulations presented in this paper, nano-magnets are initialized along their hard axis at t = 0 and then allowed to relax. The output is obtained from a statistical average of the magnetization M z along the easy axis obtained from 100 Monte Carlo runs based on the stochastic Landau-Lifshitz-Gilbert (LLG) equation, one for each magnet (WRITE and READ) coupled through a magnetic interaction as in ref. 23. The details are described in the methods section. Figure 2b shows that the numerically obtained average M z is described well by the relation , H K being the anisotropy field, γ, the gyromagnetic ratio and α is the damping parameter. Also It is important to note the key attributes of the device that are needed to enable the construction of belief networks by interconnecting hundreds of devices. Firstly, it is important to ensure input-output isolation, which is achieved by having magnetically coupled WRITE and READ magnets separated by an insulator as shown. This separation would not be needed if the magnet itself were insulating (like YIG, Yttrium iron garnet). The second important attribute is its gain defined as the maximum output charge current relative to the minimum charge current needed to swing the probability from 0.5 (fully stochastic) to 1 (fully deterministic). This is the quantity that determines the maximum fan-out that is possible which is particularly important if we want a high degree of inter connectivity. The physics of SHE 28-31 helps provide gain since for each device, it can be designed 23,32,33 to provide more spin current to the WRITE magnet than the charge current provided by the READ unit of the preceding stage.
The third attribute of the proposed device is its ability to sum multiple inputs and this can be done conveniently since it is current-driven. A WRITE circuit consisting of a SHE metal like Tantalum provides a current-driven low impedance input, different from the voltage-driven high input impedance field-effect transistors (FET's). The low input impedance ensures that the total current into the WRITE unit is determined by the output impedance of the READ units of preceding stages 23,32,33 . This impedance is set by the intrinsic resistance of the READ units which could be on the order of a kΩ if using magnetic tunnel junctions (MTJ's) or could be much lower if using the inverse spin Hall effect (ISHE) 34 . In either case an external series resistor R could be used to raise the output impedance as shown in Fig. 2a.
OUT j DD AP P AP P A P P j I N z , V DD being the external voltage, R P and R AP , the parallel and anti-parallel resistance of the MTJ, R IN , the input resistance of the next device and R j is the external series resistance which can be used to weight the outputs appropriately. The weighting of the output can also be accomplished by tuning V DD where multiple bipolar output weights sharing the same input can be implemented via a common WRITE unit with multiple READ units (An example of this is shown in the Bayesian network section) 35 .
We envision that the detailed physics used to implement the transynapse will evolve, especially the physics used for the WRITE, the READ and/or the weighting, since this field is in a stage of rapid development with new discoveries being reported on a regular basis. The input (or WRITE) circuit could utilize phenomena other than the SHE used here, just as the output (or READ) circuit could use mechanisms other than MTJ's. Similarly, the nanomagnet can be initialized in a neutral state with modern voltage driven mechanisms 36,37 like voltage controlled anisotropy, or with established methods like an external magnetic field 38,39 or spin torque 30,40,41 , or thermal assistance 42 . Alternatively, as mentioned earlier nanomagnets in the super paramagnetic 24 regime could be used with the mean state S defined by a time average instead of an ensemble average. The purpose of this manuscript is simply to establish the general concept of a transynapse that integrates a synapse-like behavior with a transistor-like gain and isolation, thus permitting the construction of compact large scale belief networks.
Note also that our transynapses are assumed to communicate via charge current since that is a well-established robust form of communication. However, communication could be influenced through spin channels (as in all-spin logic 40,43 ) or through spin waves requiring very different WRITE and READ units.

Reciprocal and Non-Reciprocal Networks
A key feature of transynapse is the flexibility it affords in adjusting the weight w ji that determines the influence of one transynapse (T i ) on another (T j ), by adjusting the parameters of the READ unit of T i . The weight w ij on the other hand is controlled independently through the READ unit of T j . If we choose w ij = w ji , we have a bidirectional or reciprocal network similar to the type described by an Ising model described by a Hamiltonian H. In such networks the probability P n of a specific configuration, = = n s { 1, 1} i is known to be given by the principles of equilibrium statistical mechanics.
where the energy E n of configuration n is given by Ising models are closely related to Boltzmann machines 1,2,5,6,10,11 whose probabilities described by Eq. 6 seek configuration with low E n . For example, with three transynapses connected through w ij = w ji > 0, E n is minimized for configurations (111) and (111) with equal s i . This is the ferromagnetic (FM) Ising model. But if w ij = w ji < 0, E n would be a minimum if all s i had opposite signs. Since this is impossible with three transynapses, the energy is lowest for all six configurations that have one 'frustrated' pair 44,45 : The numerical simulation of the 3-transynapse network shows (Fig. 3a) this expected behavior with equal probabilities for configurations A, B, C, and reduced probabilities for the two remaining configurations (111) and (111) for which all three pairs are frustrated. Situation is different when one of the bonds is directed as in Fig. 3b. Not surprisingly, the probability is highest for the configuration having T 2 and T 3 as the frustrated pair (configuration A in Eq. 8). Less obviously, configuration B with T 1 , T 3 as the frustrated pair has a higher probability than configuration C with T 1 , T 2 as the frustrated pair. This is because T 2 only has one bond (from T 1 ) dictating its state (no conflict) but T 3 has two bonds (from T 1 and T 2 ) dictating its state which can be at odds with each other. Such configuration of bonds and the resulting configuration space probabilities as shown in Fig. 3b have no Ising analog.
Note that our numerical results are all obtained directly by simulating a set of coupled LLG equations, one for each of the six magnets, two per transynapse. The time evolution of each magnet in each device is a function of its instantaneous state  → M t ( ) i , internal, external and thermally fluctuating fields (determined by temperature T), plus the spin torque τ → ij it receives from other devices:

Implementing Belief Networks
Boltzmann machine. The connection between Ising model of statistical mechanics 46 and hard combinatorial optimization problems of mathematics has been known for decades 47 . Boltzmann machines 10,11 and subsequently their restricted version for deep belief networks are Ising models in which the weights of interactions are learned and adjusted with breakthrough algorithms 1,2,5 . There is also widespread activity and innovation on the connection of inference, commonly used in belief networks, and phase transitions in statistical physics (see e.g. ref. 48 for a thorough review). Figure 4 shows how networks described in this paper (Fig. 1) can mimic magnetic phase transition which is also a well known result of the Ising model. The caption provides more detail for the particular procedure used for obtaining this. Phase transition is evident as the rate of change of magnetization with respect to temperature exhibits a maximum followed by a decrease. This transition is not sharp because of the small lattice sizes used here (see ref. 49 for a more in-depth discussion). Solid line shows the analogous Scientific RepoRts | 6:29893 | DOI: 10.1038/srep29893 Ising model result with the same lattice size (4 by 4 array) using equilibrium laws of statistical mechanics. The peak exhibited is reminiscent of the Curie temperature of magnetic phase transition (Supplementary section 3).  From this data, differential of magnetization with respect to temperature can be obtained as shown. This is reminiscent of magnetic phase transition 49 exhibiting a Curie temperature in Ising model depicted by the solid line. In deep belief networks 1,2,8,9 , the closely related restricted Boltzmann machines 10,11 trained by breakthrough algorithms that determine the interactions are used to solve search, detection and inference problems 5,7,8 .
Indeed, the effective Curie temperature observed in these networks depends linearly on the strength of device to device communication set by V DD (Fig. 2). (Supplementary section 4 provides spontaneous magnetization curves with interactions of various strength). This is in agreement with Onsager's 50 results for a two dimensional array of ferromagnetic atoms for which T C is proportional to J-coupling strength. We take these as an indication that stochastic networks of transynapses could be used to construct (restricted) Boltzmann machines for deep belief networks where weights can be adjusted by the bipolar voltages applied to transypases or by load resistances at the output of transynapses (Fig. 2).

Bayesian network. Symmetric interactions are inherent to Hamiltonian based systems as in Ising model
and Boltzmann machines. On the other hand, directed interactions have their own prominence in Bayesian networks 3,4 . Figure 5a shows a 3-transynapse network, with each transynapse representing one of three variables which we could call carrot, stick and performance as shown in Table 1. These variables can be in one of two possible states with distinct probabilities. The transynapse network is interconnected to reflect the causal interconnections among the three variables. The carrot affects both the state of stick and the state of performance through the voltages V SC and V PC which determine the weights w SC and w PC . The only other causal effect is that of the stick on the performance which is reflected in the voltage V PS and the resulting weight w PS .
A direct simulation of this 3-transynapse network using coupled LLG equations (Eq. 10) yields the plot shown in Fig. 5b. With V SC = − 1, we get the diagonal lines reflecting perfect correlation of the stick with the carrot, while with V SC = + 1, we get the other diagonal line reflecting perfect anti-correlation. (Voltages and currents  are normalized by the magnitude required for deterministic switching). We could view these respectively as a COPY gate and a NOT gate with probabilistic inputs and outputs. The other curves shown in Fig. 5b correspond to I S ≠ 0 reflecting a situation where the stick state is not entirely controlled by the carrot, but has a probability of no punishment irrespective of the carrot. The network can naturally generate probabilities of various variables. Consider e.g. the triangle (scenario A) in Fig. 5b where carrot has 0.6 probability of reward (scenario B is the square). Instead of performing the necessary algebra of to obtain the probability of stick being in the punishment mode, the transynapse network takes in I C = − 0.02, V SC = 1, I S = 0.9 and produces the directly observable probability of stick being in punishment mode. (Voltages and currents are normalized by the magnitude required for deterministic switching). This generalizes to more variables and an example for three is discussed next. Figure 6a,b shows how the network in Fig. 5a can be used in predictive mode based on known casual connections 3,4 among different variables which determine the electrical signals V ij and I i (explicitly provided in the caption). These in turn can provide the values in the (conditional) probability tables of Fig. 6a. For example, the element indexed by (1,1) in the p(S|C) table is the mean value of the state of stick in the 1 mode when C = 1. (This can also be obtained independently by dictating the carrot is in the reward (C = 1) state e.g. by providing a strong bias (I C ) and finding the mean value for the state of stick due to V SC ) While the likelihood of better performance can be found from tables of Fig. 6a by calculating p(P = 1|, S, C), this is directly observable from the mean value of the state of 'performance' which is naturally generated by the network as provided in Fig. 6b. Alternatively, the network can address inference problems. Suppose performance is better, is it due to carrot or stick or both? For instance, the likelihood that performance is better because of reward is essentially p(C = 1|P = 1). This can be obtained by the algebra, ∑ = = ∑ = p P S C p P S C ( 1, , 1)/ ( 1, , ) S C S , , or directly observed by taking the mean value of the state of carrot in the reward mode when performance is better. The resulting values are provided in Fig. 6c.

Concluding Remarks
Probabilistic computing is a thriving field of computer science and mathematics that deals with extracting knowledge from available data to guide decisive action. The work in this area is largely based on deterministic hardware and major advances can be expected if one could build probabilistic hardware to simulate probabilistic logic. In this paper we define a building block for such stochastic networks, which we call a transynapse combining (c) Inference can be addressed by such networks. For example, the likelihood that reward has caused better performance is the mean value of carrot in the reward state for cases that have better performance.
Scientific RepoRts | 6:29893 | DOI: 10.1038/srep29893 the transistor-like properties of gain and isolation with synaptic properties. In principle, many implementations are possible including those that make at least some use of standard CMOS circuitry. We present a possible implementation based on the established physics of nano magnets for the transynapse and use experimentally benchmarked models to illustrate the implementation of both Boltzmann machines and Bayesian networks. More realistic examples will be addressed in future publications 51 .

Methods and Verification
This section outlines the methodology and its implementation underlying the simulations that have been carried out. It includes the steps taken to verify the model against well known principles or experimental data.
The where q is the charge of electron, γ is the gyromagnetic ratio, α is the Gilbert damping coefficient and N s ≡ M s Ω/μ B (M s : saturation magnetization, Ω: volume) is the net number of Bohr magnetons comprising the nanomagnet and → I s is the spin current entering the magnet. This equation is transformed to its standard mathematical form (see e.g. ref. 40) and is solved numerically using a second order Runge-Kutta method (a.k.a Heun's method) in MATLAB. This methodology essentially applies the Stratonovich stochastic calculus to the stochastic integration during time dependent simulations involving thermal fluctuations. The inclusion of thermal fluctuations in LLG and its implementation has been verified against equilibrium laws of statistical mechanics in ref. 40.
→ I s , the spin current entering the magnet, can have components both due to the Slonczweski torque as well as the field-like torque. The inclusion of spin transfer torque in LLG (last term) and its implementation has been verified against experimental data in ref. 26. In this manuscript, β → =Î Iz s is generated by the spin Hall effect as outlined in ref. 23 where I is the charge current entering the W unit generated by the READ stage of the previous device (see   23 . This is essentially how Transynapses communicated with each other whereby one drives the next; the dynamics of each one being governed by the coupled LLG equations that describe the dynamics of READ and WRITE magnets.
The magnetic field, → H , represents both internal and external fields: To validate the approach and its implementation, we made use of the available experimental data for the coupling fields that have been measured in magnetic tunnel junctions. First figure of the Supplementary Information shows the comparisons between the calculated values from the model and the data from various experiments. These comparisons show that the model is generally in good agreement with experimental demonstrations.