Optimizing the Weights and Thresholds in Dendritic Neuron Model Using the Whale Optimization Algorithm

In recent years, with the great success of dendritic neuron model (DNM) in various fields, the application of intelligent optimization algorithms in dendritic neuron model has attracted increasing attention of researchers. The training process of neural network is regarded as one of the great challenges of machine learning because of its non-linear nature and unknown optimal parameters. The traditional training algorithm of DNM is prone to fall into local optimum and speed of convergence slowly and so on, resulting in the problem of accuracy and low efficiency. In this paper, for solving the classification problem of dendritic neural model, an innovative intelligent optimization algorithm which named whale optimization algorithm (WOA), is applied to the training of DNM for the first time. Compared with six traditional and classic intelligent optimization algorithms in four classic datasets, the results indicate that WOA-DNM has good performance in various aspects, and its advantage is remarkable.


Introduction
Artificial neural network consists of a lot of widely interconnected processing units to simulate the human brain of the nervous system. There are billion neurons in human brain, a single neuronal cell is made of cell body, axon, cell membrane and dendrites. Dendritic cells account for more than 90% of neural cells, which plays a pivotal role in human learning. In 1943, McCulloch and Pitts [1] came up with the abstract neuron model firstly with reference to the biological neuronal structure, which has been widely applied in various fields as the basic unit in the research of modern neural network. However, it ignores the dendritic structure of real biological neurons, the network model was approved too simple.
As the research further develops, the application of dendritic structures in neural computing has attracted extensive attention. In 1997, Koch and Segev [2,3] presented that the mutual effect between neuronal synapses and branching turning points can be approximately understood as logical operations. In recent years, the dendritic computing model based on dendritic structure has been proposed [4,5]. A single dendritic neuron model (DNM) stands out due to the non-linear nature of synapse, which has been successfully applied in the classification of breast cancer [6], stock market futures prediction [7], photovoltaic power generation prediction [8] and so on. Although it is a breakthrough success for the application of DNM in various aspects [9][10][11][12], traditional DNM uses Back-Propagation Learning Algorithm (BP) [13], a gradient descent optimization method in essence, for training, which is cushy to trap local optimal in the training process.
In order to improve accuracy and solve existing problems, this paper adopts innovative intelligent optimization algorithm, Whale optimization algorithm [14], to train DNM. Traditional classical intelligent optimization algorithm such as Particle Swarm Optimization Algorithm (PSO) [15], Genetic Algorithm (GA) [16], Artificial Bee Colony Algorithm (ABC) [17], Differential Evolution Algorithm (DE) [18], Population-based Incremental Learning Algorithm (PBIL) [19], and BP algorithm were compared and analyzed. The aim is to determine the effective learning algorithm to train the DNM and further explore the application of DNM in the classification problem.

Dendritic Neural Model
There are four layers in DNM: synaptic layer, dendritic layer, membrane layer and soma layer. In this section, we will introduce the structure and function of these layers in detail.

Synaptic Layer
Synapse is a special structure of a neuronal impulse connection to another neuron or inter-cell contact structure. The signal is conducted from the pre-synaptic neuron to the synaptic neuron, the excitability or inhibitory property of synapses depends on the change in the membrane potential of the induction of the imitation effect. For the input, the synaptic layer is activated by the Sigmoid function. The description of ith (i = 1, 2, ..., N) synapse input to jth (j = 1, 2, ..., M) synapse layer is shown in (1).
where Yij is the output the jth synaptic layer, xi is input of ith synapse and xi∈[0,1], k is a positive constant. Parameters wij and θij trained by the algorithm. According to the value of wij and θij, there exists four types of connection instances: (a) excitatory state: when 0 < θij < wij, the output of synaptic layer is proportional to the input of synaptic layer.
(b) inhibitory state: when wij < θij < 0, the output of synaptic layer is inversely proportional to the input of synaptic layer.
(c) constant-0 state: when wij < 0 < θij or 0 < wij < θij, regardless of the input of synaptic layer from 0 to 1 to transform, the output of synaptic layer is approximately 0 (d) constant-1 state: when θij < wij < 0 or θij < 0 < wij, regardless of the input of synaptic layer from 0 to 1 to transform, the output of synaptic layer is approximately 1.

Dendritic Layer
For each branch, dendritic layer multiplies the outputs of synaptic layer, the output formula of jth dendritic layer is shown in (2).

Membrane Layer
The membrane layer is connected to each dendritic layer. It sums the results of all branches. The output formula of membrane layer is shown in (3).

Soma Layer
As the last layer in DNM, soma layer is nonlinearly calculated on the results it received. If the input exceeds a preset threshold, neuronal cells will be triggered. The output formula of soma layer is shown in (4).
where the parameter ks is a positive constant, the value of θs between 0 and 1.

Whale Optimization Algorithm
Whales as the largest animals on the planet, what's interesting about humpbacks whale is their unique hunting method-the bubble-net feeding method. When humpback whales find prey, they dive 12 meters deep firstly, then they create bubble nets to catch their prey. Inspired by this, WOA models the predation mechanism of humpback whales to achieve the purpose of optimization. WOA initiate population by generating randomly, its advantage is that each candidate solution is optimized and improved at each step of the optimization process to achieve greater accuracy. Also it has a global search capability and few parameters to be adjusted. The main formula proposed by WOA is as follows (5). MaxT , R is a random vector and R∈[0,1]. Similar to other optimization methods based on swarm intelligence, Its search procedure is divided into two phases: exploration and development. In process of iterative optimization, searching individual according to the random probability select the individual or obtain the best solution to update its position. The pseudocode of WOA is shown in Algorithm 1.
The optimal parameters of weights wij and thresholds θij in the synaptic layer of DNM are obtained by iterative training of WOA algorithm. The process of optimizing DNM by WOA can be expressed as equation (6).
where Xi (i = 1, 2, ..., m) is the ith individual of population, m is size of population. Through the process described in algorithm 1, WOA use MSE (the mean-squared errors) to calculate the output error of DNM, which is shown in (7): where Q is the total amount of training samples, Tq is the target vector for the qth sample, and Oq is the real output vector. Update the position of agent by D ⃗⃗⃗ = |C ⃗⃗ . X * ⃗⃗⃗⃗⃗ (t) − X ⃗ ⃗⃗ (t) else if2(|A|>1) Select a random agent and update position by X ⃗ ⃗⃗ (t + 1) = X rand ⃗⃗⃗⃗⃗⃗⃗⃗⃗⃗⃗ − A ⃗ ⃗⃗ . D ⃗⃗⃗ end if2 else if1(p>0.

5)
Update position of the agent by X ⃗ ⃗⃗ (t + 1) = D ′ ⃗⃗⃗⃗⃗ • e bl • cos(2πl) + X * ⃗⃗⃗⃗⃗ (t) end if1 end for Calculate the fitness value of agents Update optimal solution X * Increment Counter t=t+1 end while return optimal solution X *

Experiment and Analysis
To confirm the validity of whale optimization algorithm on dendritic neural model, the experiment compares the test results of six classical algorithms (PSO, GA, ABC, DE, PBIL, BP) on four classical classification data sets (Banknote Authentication, Car Evaluation, Breast Cancer, Glass Identification). The four classification datasets are derived from the machine learning knowledge base in the University of California [20], which are widely used as experimental test set in the field of artificial intelligence. Table 1 summarizes the number of attributes, sample quantities and categories of datasets. All data samples are divided into two groups in a random classification, 70% as training samples, 30% as test samples. In order to exclude the incidental chance of the experiment, all of the experimental results are the average results of 30 experiments. The software environment of experiment is MA TLAB2018b, hardware environment is Intel(R) Core (TM) i5-9500 CPU @ 3.00GHz,8GB.  There are 4 user-defined parameters in the DNM, the number of dendrites (M), parameter (k) of synaptic layer and the parameters (ks and θs) of the Sigmoid function in soma layer. Table 3 shows the DNM parameters of different datasets.   Table 4 is the accuracy of each algorithm in each dataset. As shown in the table, no matter what data set, WOA's accuracy is the highest and significant, proves that WOA is trustworthy. In order to facilitate intuitive comparison of the algorithm's optimization ability, experiment usually uses the convergence graph to visualize the process. The more drastic the decline of the convergence curve is, the more excellent the global optimization ability of algorithm is. Figure 1  displays the convergence results of each algorithm in four datasets. The abscissa is the count of algorithm iterations, the ordinate is the minimum mean square error of the model in the training process. The downward trend of the four images shows that the convergence of WOA algorithm is obviously better than other algorithms, the performance of PSO and DE algorithm is slightly worse, the performance of GA and PBIL algorithm is general, and BP and ABC algorithm is the worst.
The high classification accuracy and low mean-squared errors of WOA indicate that this method can reliably prevent precocity from converging to local optimum, effectively find the best parameters of weights and thresholds in DNM. Boxplot is a method of describing data with its minimum, first quartile, median, third quartile, and maximum. The size of the box is connected with the range of quadrants, the horizontal line in the box represents the median and the point away from the box is an abnormal value. Figure 2 is a boxplot of four data sets. The results show that WOA has a good effect on DNM training.
In summary, the performance of WOA-DNM on data set is superior to existing algorithms. Due to the algorithm mechanism of WOA, On the one hand, the random choice of prey avoids a lot of local solutions in DNM training; On the other hand, its hunting mechanism needs to search for the surrounding space around the agent. Therefore, WOA not only has high precision, but also the advantages of good convergence.

Conclusion
In this paper, WOA is applied to the training the best weight and threshold of DNM for the first time. The training problem of DNM is defined as a minimization problem, the target is to minimize the mean square error and the parameter is the connection weight and threshold of DNM. By comparing with traditional classical algorithms PSO, GA, ABC, DE, PBIL and BP, the performance of WOA algorithm in the classification of data sets is proved to be significantly better than other existing algorithms, which indicates that WOA not only has high accuracy, but also has good convergence. It provides an effective analysis method to solve the application of DNM in engineering classification issues.