Feasibility of Multiplex Communication in a 2 D Mesh Asynchronous Neural Network with Fluctuations

It remains a mystery how neural networks composed of neurons with fluctuating characteristics can reliably transmit information. In this study, we simulated a 9 × 9 2D mesh neural network consisting of an integrate-and-fire model without leak, and connection weights that were randomly generated. The characteristics of the refractory period and output delay of the neurons were fluctuated time to time. Spikes from transmitting neuron groups spread (propagated as spike waves) to receiving neurons. For 9 to 1 multiplex communication with a back propagation neural network (BPN), the receiving neurons successfully classified which neuron group transmitted the spike at a rate of 99%. In other words, the activity of the neuron group is propagated in the ne ural network as spike waves in a broadcasting manner and the wave fragment is received by receiving neurons. Next, point-to-point signal transmission in the neural network is carried out by multi-path, multiplex communication, and diversity reception. Each neuron can function in 3 ways of transmit, relay (transfer), and receive; however, most neurons act as a local relaying media. This type of mechanism is similar to sound propagation through air. Our research group studies the functions of neural networks by combining experiments with cultured neuronal networks with artificial neural network simulations. This current study corresponding to our previous work on the ability of remote receiving neurons to identify two transmitting neuron groups stimulated in a cultured neuronal network, i.e., 2 to 1 communication. These mechanisms may be the basis of higher cortical functions.

We have also visualized information flow/communication in the brain [16,17].Furthermore, we have shown that spike waves, which spread and propagate from stimulated ne urons, are received by afferent neurons as random-like sequences in natural asynchronous neuronal networks [18].These networks can be well-simulated by 2D mesh asynchronous neural networks composed of an integrate-and-fire model without leakage [19,20].The theme of our research combines wet-lab experiments of cultured neuronal networks with computer simulations.Importantly, communication within the neural networks of the brain may be the basis for higher cognitive functions.
In the current study, we show using a 9 × 9 2D mesh neural network simulation that 9 to 1 multiplex communication is possible at a success rate of 99%.This study corresponds with our previous publication [21] on the ability of remote receiving neurons to identify two transmitting neuron groups stimulated in a cultured neuronal network, i.e., 2 to 1 communication.Section 2 explains the integrate-and-fire model without leakage.Furthermore, we show that spikes spread from transmitting neuron groups, propagate as spike waves, and are received by receiving neurons.In section 3, we show using the back propagation neural network (BPN) method that receiving neurons can classify which neuron group transmitted the spike waves.The simulation results and discussion are provided in sections 4 and 5, respectively.

Simulation
We performed a computer simulation to observe spike propagation.We designed a 2D network with a 9 × 9 mesh of neurons with connections that were randomly generated and uniformly distributed between +1 and -1/3 in each experiment; thus, the number of positive weights was three times the number of negative weights in the mesh.Each neuron has connections to and from eight neighboring neurons, except for peripheral ones (Figure 1).The neuron model used was previously reported [20] (Figure 2), where thresholding effect to small variation of action potentials is modeled by fluctuations of neuron characteristics of accepting period and output delay.Neuron n accumulates weighted inputs during the accepting period, A nk .If the weighted input sum is positive and the neuron is not in its refractory period, the neuron produces
a spike after the output delay time, D nk, in the k-th firing.Some of the parameters used in the simulation are provided below.R A (a n ) = {a n − 1, a n , a n + 1}.
A nk = a n − 1, a n , or a n +1 with probability p a , 1-2 × p a , and p a , respectively, and p a = 0.2.
We define the refractory period of a neuron as an accepting period plus the output delay.The output delay is comparably smaller than the accepting period; thus, we can regard the accepting period roughly equals the refractory period when the output delay is negligible.In the present simulation, the intrinsic accepting period, a n , was set around 2.0 ms to raise the simulation speed; however, this value may be several times smaller than that of a typical true accepting (refractory) period.
Next, we defined the instantaneous variance of A nk by Similarly, the output delay is set with the following equation: d n is an intrinsic output delay time of neuron n, and randomly selected from {2, 3, … 8}.Though our network model is a regular 2D mesh type, the real distances between neurons will be various, and this d n allows for this condition.
, and p d = 0.2, which is the same probability as p a .
The stimulation of class c is represented by the spatiotemporal pattern, S c, on 3 neurons in a transmitting neuron group, c: where N c1 N c2 ,N c3 are stimulated neuron number, and t c1 , t c2 , t c3 are stimulated time.
One of S c 's is selected in a trial and all neurons in S c are stimulated at the same time t Though the number of neurons Q in the transmitting neuron group is not always 3, Q = 3 is used for simplicity in the following examples.The same holds true for the number of receiving neuron groups (i.e., M = 3) in the following examples.
An example of the spike waves in a simulation of a 9 × 9 neural network was previously shown [20]; thus, the extended result for a 25 × 25 neural network is shown in Figure 4, which includes "spike waves" that propagate from stimulated transmitting neurons.This figure illustrates the aim of our full-scale research.

Classification by BPN
A trial denotes the process from when the neurons are stimulated by 1 of 9 spatiotemporal pattern classes to the final determination of which pattern class was used for the stimulation.It is not possible to identify the kind of information that is transmitted within the waves by only observing wave propagation.The BPN method has moderately strong pattern recognition ability [22]; thus, the BPN method can be applied to show the feasibility of multiplex communication in neural networks.
We used this method as the receiving/recognizing scheme for communication in the neural network.
The logical composition of the simulation is shown in The encoded value for the interval  k between the k-th and (k + 1)-th spikes is defined by (5): where  k = t k+1 − t k and t k = time of the k-th spike.Now, we have already known the four wave arrival times to the reference neuron, and we can utilize them to encode the spikes at another receiving neuron n.A spike near the k-th spike at the t k of the reference neuron is encoded by the following equations (Figure 8): If a j-th spike is received by neuron n at time t n j  [t k − Tr/2, t k + Tr/2], ∃j = 1, 2, … k = 1, 2, 3, 4, then Otherwise, Practically, receiving neurons are spatially close; thus, their firings are similar and do not often fit (8), i.e., j = k in most cases.
The refractory period (roughly equal to the accepting period) fluctuates time to time in our model; thus, Tr is defined as the minimum of these value.In the current simulation, the minimum instantaneous accepting period was 18 − 1 (= 17), the output delay was 2 − 1 (= 1), and the total Tr was 17 + 1 = 18 (1.8ms).The difference, f n k , of the arrival time at another receiving neuron n from the reference receiving neuron as well as its sign, g n k , (+1, −1; precede or delay) were obtained.The 3 wave arrival interval lengths, f ( k ), at the reference neuron were also sent to the BP N; thus, the total number of input neurons to the BPN is 8(4M − 1) + 3. The number of middle layer neurons was 45, and the learning rate of all computational units was 0.2.The reference receiving neuron and other receiving neurons can receive spikes at any time, which is defined as asynchronous reception.

Classification Results
The correct rate of classification by the BPN depends on how much the neuron characteristics fluctuate, how many kinds of spatiotemporal patterns are given in a trial, and how many receiving neurons are employed.After many trials, the correct rates of pattern classification by the BPN after learning are shown in Figure 9.Each data point was obtained from more than 50 randomly generated networks and more than 250 learning cycles within each network after the learning converged.The fluctuation sizes are F OD = F AC = 0.4.Parameters Q (number of neurons in a transmitting neuron group) = 1 (9), 3 (27), and M (number of receiving neuron groups) = 1 (4), 2 (8), 3 (12).The numbers in parentheses are the total number of neurons transmitting or receiving.The maximum correct rate of 0.987 was obtained for Q = 3 and M = 3.In this case, the average number of learning cycles needed to reach convergence was 72.5.
It appears that a larger number of neurons involved in communication, e.g., 3 neurons in a transmitting neuron group and 3 receiving groups, results in smooth and stable transmission, whereas fewer neurons make communication more difficult.Increasing M is more effective than increasing Q.
In addition, larger fluctuations in neuron characteristics, a larger number of kinds of patterns, and a wider range of connections (e.g., 24 neighbors) decrease the correct classification rate of the BPN.
Results of extensive simulations are in process and will be reported in subsequent papers.In these simulations, i5 PC's are employed with Basic program.Computation time varies according to various conditions such as M, Q, F AC , F OD , mesh size, and also need many runs to obtain statistical averages.It ranges from several minutes to a few months as well as parallel usages of several PC's.

Discussion and Conclusion
The coding and communication mechanisms of neural networks are yet to be determined.In a simulation of a 2D mesh neural network, we showed that 9 to 1 communication is possible with classification by the BPN, irrespective of fluctuations in neuron characteristics.Although the specific classification algorithms are different, this result corresponds to our wet-lab experiment on 2 to 1 communication in cultured neuronal networks, which showed discrimination with the dynamic time warping (DTW) significance test [21].In this previous study, some (receiving for test) neuron groups were able to discriminate which of the two (transmitting) neuron groups was stimulated, while others failed to make the discrimination.In the current simulation, neural networks could identify even 9 multiplexed signal sources despite various randomly generated weights.This ability was likely due to the strong learning and discrimination abilities of the BPN and combination of neuron groups.
These experiments support the hypothesis that the spatiotemporal firing pattern is transmitted through the neural network as spike waves in a broadcast manner.At the receiving side, neurons may decode the spatiotemporal pattern of the wave and respond according to the transmitted pattern class.
In other words, each neuron acts as a local minor relaying media and is relatively insensitive to the communication function of the whole network.For example, some spike losses will not cause a loss in communication.In this process, a refractory period helps to regulate and stabilize the spike waves.
The wave proceeds to a new area that has not fired recently, and as a result, the wave front formed often appears as a synfire chain or synchronous and coherent firing.In other words, spatially close neurons often fire similarly and together.Synchronistic group activity, rather than single neuron activity, is effective for stable communication, and is often observed in neuronal networks [23][24][25].This communication process is similar to sound wave propagation in the air (Figure 10).With this type of communication, we can identify the source and type of sounds.The communication process used in neural networks is also partially similar to diversity communicat ion, in which plural antennae are spatially separated at base stations to stabilize mobile communication [26].We confirmed that each neuron can perform 3 types of tasks, including signal emission, relay, and reception.However, within our model neurons mainly function as a relaying media for multiplex communication.In this study, we showed that 9 to 1 transmission is possible in a 9 × 9 2D neural network.Currently, we are assessing wider networks with more natural recognition filters, i.e., Laplacian Gaussian functions, instead of the BPN method which is inconsistent with natural functioning.We believe these communication functions may explain the physiological basis of higher order cognitive functions.
Rather than high precision processors, many low precision processers are used in the AlphaGo of Google AI, which showed strong power for AI [27].In our case, although each neuron has a low precision processing function, e.g., fluctuating characteristics, the overall neural network communicates well.
The features of this paper are summarized as follows: (1) To our knowledge this paper is the first attempt to simulate multiplex communication in a neural network.Our work shows a signal transmission principle in neural networks which provides a possible solution to the mystery of the manner of reliable neural communication.
(2) The simulation corresponds to our wet lab experiment [21] of two to one communication in cultured neuronal network.
(3) The simulation showed quantitatively that grouping firing of neurons is effective for stable information transmission, which is often observed in the naturally occuring neuronal networks.

Figure 1 .
Figure 1.Weighted connections from 8 neighboring neurons.Weights are randomly generated so that the rates of positive and negative weights are 3/4 and 1/4, respectively.

Figure 2 .
Figure 2. Integrate-and-fire model without leakage but with fluctuation in the parameters of neuron n.Each neuron has an inherent accepting period, a n , and output delay time, d n , which vary with time within certain ranges defined as R A (a n ) and R D (d n ), respectively.Neuron n integrates weighted input spikes during the accepting period, A nk , for the k-th firing.After the accepting period ends, the neuron determines if the integrated

Figure 4 .
Figure 4. Spike waves between 0.1-5.0ms in a 25 × 25 network after the 3 top center neurons are stimulated.Yellows represents when F OD = F AC = 0.167, red represents when F OD = F AC = 2.0, and light blue represents both of these conditions overlapped.

Figure 5 . 3 Figure 5 .Figure 6 .
Figure 5. Logical information flow in the simulation.One of the transmitting neuron groups is stimulated in a trial, and BPN estimates/classifies which neuron group is stimulated based on spatiotemporal firing pattern of receiving neuron groups.

Figure 7 .
Figure 7. Encoding of spike intervals at the reference receiving neuron.

Figure 8 .
Figure 8. Spikes at receiving neurons.Spikes to receiving neurons except for the reference neuron are encoded with time difference from the reference one and sign of delayed (+1) or preceding (−1), separately.

Figure 9 .
Figure 9. Correct classification rate by the BPN for 9-channel communication with plural neurons co-working in the transmitting and receiving groups.M is the number of receiving neuron groups.Q is the number of neurons in a transmitting neuron group.In case of M = 1 and Q = 1, BPN learning did not converge well.

Figure 10 .
Figure 10.Sound transmission has characteristics of multiplex communication utilizing common space, multi-paths, and diversity antenna, that enables the identification of the source and type of sounds.The neural network is similar to sound transmission.Namely, the receiving neuron group can identify what type of activity occurred at a remote position.
The true instantaneous accepting period, A nk, of neuron n for the k-th firing is randomly given within the fluctuation range R A (a n ) around a n , i.e., A nk R A (a n ), where