It may be time to perfect the neuron of artificial neural network

In recent years, artificial neural networks (ANNs) have won numerous contests in pattern recognition, machine learning, and artificial intelligence. The basic unit of an ANN is to mimic neurons in the brain. Neuron in ANNs is expressed as   f wx b  or   f wx . This structure does not consider the information processing capabilities of dendrites. However, recently, studies shown that dendrites participate in pre-calculation in the brain. Concretely, biological dendrites play a role in the pre-processing to the interaction information of input data. Therefore, it is time to perfect the neuron of the neural network. This paper, add dendrite processing section, presents a novel artificial neuron, according to our previous studies (CR-PNN or Gang transform). The dendrite processing section can be expressed as WA X . Because I perfected the basic unit of ANNs— neuron, there are so many networks to try. This article gives the basic architecture


Introduction
An artificial neural network (ANN) is an algorithmic architecture that imitates the biological brain [1,2].Today, neural networks are gradually changing our lives and making the world better [3].First of all, I want to thank those who contributed to the present state of the art, e.g., Geoffrey Hinton, YannLeCun, Yoshua Bengio, and so on.
Recently, artificial neural networks (ANNs), particularly "deep learning" [4], have made some impressive recent advances, such as machine vision, speech recognition, autonomous vehicles, and machine translation.Thus, in the tech world today, optimism is high.Some people think ANNs are approaching human intelligence gradually, even it is still far away [1].
Neurons are the fundamental units in the biological brain.Accordingly, the fundamental units of ANNs are artificial neurons.With the development of technology, people have a deeper understanding of the biological brain and neurons.However, the artificial neuron has maintained its original structure.It may be time to perfect the neuron of the artificial neural network.
The following presents a very brief introduction to the basic knowledge and latest developments in artificial neural networks and biological Neural Network.

Artificial neural networks
An ANN comprises many interconnected functional units or neurons and can learn the relationship of input space and output space.Thus, they usually are used to solve classification or regression problems [5].Over the years, based on the application and the characteristics of the data involved, different architectures of ANNs have been developed.For example, Convolutional Neural Networks (CNN) [6,7] in computer vision, Recurrent Neural Networks (RNN) [8] or Long Short Term Memory Network (LSTM) [9] in sequence and time series modeling, and Generative Adversarial Network (GANs) [10].
In the last decade, some newer architectures with newer learning algorithms are developed to endure the need to develop human-like efficient machines in different application areas.From the perspective of information dimension, it can be divided into a network focusing on spatial information [CNN, the representative field is computer vision (CV).] and a network focusing on temporal information [RNN, the representative field is natural language processing (NLP).].There are several newer CNN architectures and efficient mechanisms: Alexnet [7], VGG [11], Googlenet [12], Inception-V3 [13], ResNet [14], ResNeXt [15], Convolutional Block Attention Module (CBAM) introduced by Woo et al. [16], and competitive squeeze-excitation (SE) mechanism introduced by Hu et al. [17].There are several newer RNN architectures and efficient mechanisms: Deep Recurrent Attentive Writer (DRAW) [18], Grid Long Short-Term Memory [19], gating mechanism introduced by Jing et al. [20], and factorized recurrent network architecture introduced by Belletti et al. [21].
Besides, the learning of ANNsby changing the distribution of weights to approximate the relationship between input and output space-has been studied.Lately, success was achieved in many techniques, such as L1 and L2 regularization [22], batch normalization [23], a good collection of weight initialization techniques [24,25], and a good set of activation functions [5].

Biological Neural Network
The human brain has approximately 100 billion biological neurons, and neurons are connected via specialized structures known as synapses to sense stimulations and to pass signals to other neurons [26].A neuron is the fundamental structural and functional unit in the neural information network and is composed of a cell body (soma), dendritic trees, and an axon [27,28].The most extended parts of many neurons are dendrites [29], and the active electrical properties of dendrites shape neuronal input.
In the field of biology, researchers have been studying the mechanism of the element of neurons over the years [28,[30][31][32].Recently, studies discovered a class of calcium-mediated dendritic action potentials (dCaAPs) [32,33].Here, we quote the original words in the literature: "These dCaAPs enabled the dendrites of individual human neocortical pyramidal neurons to classify linearly nonseparable inputs-a computation conventionally thought to require multilayered networks."[32] The intersection of dendrites may exist computing.This means dendrites participate in precalculation in a biological neuron or biological neural network.
Revisiting the previous ANNs, we found few studies highlight perfecting the neuron of ANNs.Fortunately, it happens that Gang et al. [34] proposed a module of fast calculation.The module can process linearly nonseparable inputs.Thus, it can simulate the function of dendrites and perfecting the neuron.

Traditional artificial neuron
In 1943, McCulloch and Pitts proposed ANN models with adjustable weights [35].More than ten years later, Rosenblatt put forward the Perceptron Learning Algorithm [36].In 1986, Rumelhart et al. proposed learning representations by backpropagating errors [37].By then, a typical artificial neuron was established.
Figure 1 showed the traditional artificial neuron.This architecture of artificial neuron ignores the pre-calculation of Dendrites.If X is the input and f is the nonlinear activation function, the output y can be represented by, Where new W are the updated weight matrix for W , and E is the cost function and  is the learning rate.

Novel Gang artificial neuron
For the biological neurons, the XOR operation is performed in the dendrites with dCaAPs [32,33], and AND/OR operations are performed at the soma and at tuft and basal dendrites with sodium and NMDA spikes [38][39][40].The XOR operation means or contains the brain pre-processing to the interaction information of input data.For understanding a picture task, It refers to the relationship across parts in an input-picture.For understanding an article or a speech task, It refers to the relationship across parts in an input-word.However, traditional artificial neurons only imitate the soma.

Dendrite processing section
In previous studies, inspired by Taylor series and Jiushao Qin or Horner Algorithm, we presented a module that upgrades the input order and introduces the interaction of input [34,41].The module is represented as follows.
, 1 Where and i A are the input and output of the previous layer, respectively, X is the original input.Coincidentally, It can simulate the dendrite processing section.
In figure 2a, we use one module to simulate the dendrite.The "dendrite" contains the interaction item across both input variables and can be represented as follows. 10 Where A is the output of dendrite or the input of the cell body.
Additionally, we also can use more modules to simulate the dendrite (see Fig. 3).The "dendrite" contains the interaction item across multiple input variables (the number of modules plus 1).For example, if we use two modules to simulate the dendrite.The "dendrite" contains the interaction item across three input variables and can be represented as follows.A is the output of dendrite or the input of the cell body.
The operation of the "dendrite" only uses matrix multiplication and Hadamard product, and due to Jiushao Qin or Horner Algorithm, and upgrading interaction item only need to add one module; thus, the computational complexity is lower [34,41].

Gang neuron of unicellular organism
In nature, there are unicellular organisms, such as bacteria and fungus.The first organisms to appear on Earth were presumably single-celled [42].Thus, figure 3 shown the Gang artificial neuron with multiple interactions to imitate unicellular organisms.The architecture has been shown to be able to realize strong nonlinear expression [34,41] and can be represented as follows.
  Where l represents the number of modules to simulate the dendrite.

simplified Gang neuron
The weight matrix of the cell body 21  W in Fig. 2 or 1, ll W  is defined as a constant matrix ones W where all its elements are 1.[Considering batch training, we described it as a matrix rather than a vector.]Thus, the architecture in figure 2 can be represented as And the architecture in figure 3 can be represented as

Single-layer network
Figure 4 shows the single-layer network.This architecture imitates the intricate dendrite of multiple neurons, and one information can be obtained by multiple neurons [32,33,42].

BPNN using Gang artificial neuron
The backpropagation neural network (BPNN) is the most typical neural network.Here, we show a BPNN using Gang artificial neuron with one module (see Fig. 5).The forward propagation can be represented as follows.

Information fusion network
Figure 6 shows the information fusion from different neurons.This kind of information interaction is common in the biological brain [38][39][40].This architecture in Fig. 6 can be expressed as follows.
Where  

Convolution layer using Gang neuron
Figure 7 shows the comparison of using Gang neuron or traditional neuron in the convolution layer (an example).The Gang neuron extracts information about the interactions between inputs of space.

Discussion and conclusion
In this paper, we compared the development of artificial neural networks with biological neural networks.As expected, development directions are different.Researchers in ANNs paid more attention to the architectures aiming to different application areas, while researchers in biology focused on exploring the neural mechanisms.Both directions achieved good success.It may be time to learn from each other.
As we know, artificial neural networks are inspired by biological neural networks.Seventy years ago, people designed artificial neurons by imitating the knowledge about biological neurons at that time.Today, due to the development of biology, we have a relatively good understanding of how the work of neurons, especially dendrites.
We found, at the time of design, the traditional artificial neurons ignored a fact that dendrites participate in precalculation in a biological neuron or biological neural network.More specifically, biological dendrites play a role in the brain pre-processing to the interaction information of input data.This can be illustrated briefly by two tasks in life.For understanding a picture task, biological dendrites play a role in extracting the relationship across parts of an input-picture.For understanding an article or a speech task, biological dendrites play a role in extracting the relationship across parts in an input-word.This paper perfected the neuron of ANNs by introducing the dendritic module and presented a novel neuron, named Gang neuron.

Outlook
In this paper, we have shown some basis architecture using Gang neuron.These architectures can be implemented easily.However, because I perfected the basic unit of ANNsneuron, there are a lot of networks to try.I show my guess: traditional artificial neurons are insufficient in extracting the interaction information of input data.Thus we have designed a lot of convolutional layers.Gang neurons maybe reduce the number of layers in an existing network for the same task.

Figure 2 are the updated weight matrix for 21 W
Figure 2 shown the Gang artificial neuron with one module.The architecture of Gang artificial neuron can be represented as
is represented according to formula 7 and 8.
e is the number of neurons.The network can learn by backpropagation is represented according to formula 7 and 8.

Figure 8
Figure8shows the comparison of using Gang neuron or traditional neuron in schematic diagram of RNN.The Gang neuron extracts information about the interactions between inputs of time.Where x represents the current input and h represents the output of the previous time.