On global asymptotic stability of neural networks with discrete and distributed delays

We study the dynamical behavior of a class of neural network models with time-varying delays. By constructing suitable Lyapunov functionals, we obtain sufficient delay-dependent criteria to
ensure local and global asymptotic stability of the equilibrium of the neural network. Our results are applied to a two-neuron system with delayed connections between neurons, and some novel asymptotic stability criteria are also derived. The obtained conditions are shown to be less conservative and restrictive than those reported in the known literature. Some numerical examples are included to demonstrate our results.


Introduction
An artificial neural network model is usually described by a system of ordinary differential equations.But experimental studies have demonstrated that, in general, time delays exist in electronic circuits because of the finite switching speed of amplifiers and the model should be described by the system of delay differential equations.Therefore, theoretical and computational studies of the dynamical system for the Hopfield-type analog neural network with time delays have advanced greatly in the recent years (see, e.g., [5, 7-9, 11-13, 16, 17, 19-22] and the references therein).It is well known that the effects of time delays on dynamical properties of neural networks are very complicated.If the connection matrix is symmetric or antisymmetric, Hopfield-type neural network without time delays is always either a convergent gradient networks or a stable network, respectively.However, if the delays are present, their convergences and stability properties may be lost even for very small delays, and oscillations or chaos may occur (see, e.g., [1,14,15,18,21]).
On the other hand, it is also known that delays may lead to increased stability for some dynamical systems [2,4,22].The existence of equilibria of neural networks is studied in 2 Delay-dependent asymptotic stability [3, 7-9, 11-13, 16, 17, 19, 20, 22].Detailed analysis of local stability (or, the so-called linear stability) of equilibria, oscillation, and existence of periodic solutions for some models simpler than neural network model (e.g., ring neural network models) is presented in [1,14,15,18,21], based on the classical methods of characteristic equations and Hopf bifurcation analysis (see [10,14,15,18]) and numerical computations.
Global asymptotic stability of the equilibrium of neural network with time delays, which is more important and usually more difficult to analyze than local stability, is investigated in [7-13, 16, 17, 19, 20, 22] by Lyapunov functions.The results in [3, 6-9, 11-20, 22] essentially say that if the gains of the activation functions or the synaptic connection strength are small enough, the network is globally asymptotically stable independent on the delay.Global asymptotic stability of the equilibrium of a class of delayed Hopfield-type ring neural network model which satisfies positive feedback condition is also considered in [21], and a very general sufficient criterion for global asymptotic stability of equilibrium is given based on the properties of monotone semidynamical system [4,22].However, in many practical time-delay neural networks, the time delays appearing in the systems are time-varying or are only known to be bounded in a certain range.Typical time-delay neural networks with multiple time-varying delays include the Hopfield neural network model [7-9, 12, 16, 19, 22], cellular neural network model [12,17,20], and bi-directional associative memory [3,13].Consequently, the stability analysis of timedelay systems has been a main concern of researchers.The stability criteria for timedelay systems can be classified into two categories, namely, delay-independent criteria and delay-dependent criteria, depending on whether they contain the delay argument as a parameter.There have been a number of significant developments in searching the stability criteria for systems with constant delays.However, the criteria are mostly delayindependent ones for time-delay systems with constant delays [3, 6-9, 11-13, 16, 17, 19, 20, 22] and only a few of them are for neural networks with time-varying delays, see, for example, [7,8,11,12].
In this paper, we will give some new criteria for local and global asymptotic stability of the equilibrium of neural networks with time-varying delays.Our results essentially show that if the equilbirum of the network remains globally asymptotically stable when time delays are small enough, suitable Lyapunov functionals are constructed to prove our results.
The organization of this paper is as follows.In the following section, we will give the network system to be considered and some needed preliminaries.The proofs of the main results will be given in Section 3. In Section 4, we apply our results to two-neuron system with time delays and numerical simulations to illustrate the applications of our results.Some conclusions are also given in Section 5. Finally, we will give the detailed constructive procedure of the Lyapunov functional V used in Appendix A.

Statement of networks and preliminaries
In this paper, we will consider the following neural networks with time-varying delays: for t ≥ 0, where w i j , w τ i j , and I i are real constants.The delays τ i j (t) are more than zeros.The functions g j are continuously differentiable on R = (−∞,+∞) and such that g j (0) = 0, j = 1,2, ...,n.
Lemma 2.3.Let f be a nonnegative function defined on R + such that f is integrable and uniformly continuous on R + .Then lim t→+∞ f (t) = 0.

Stability analysis
In this section, we will consider the stability of the equilibrium (u * 1 ,u * 2 ,...,u * n ) of system (2.1).
Let us first consider the cases w τ i j = 0, for some i, j = 1,2,...,n.We further assume the following hypothesis (H 5 ).
8 Delay-dependent asymptotic stability (H 5 ) There exist positive constants λ i , i = 1,2,...,n, such that By the process of the proof of Theorem 3.1 (see Appendix A), we can easily obtain.
Similar to the process of proof for Theorem 3.1, the following corollary is immediate.
1), then system (2.1) is reduced to the following Hopfield-type neural networks with constant delays: ui (t) = −d i u i (t) + n j=1 w i j g j u j (t) + n j=1 w τ i j g j u j t − τ i j + I i , i = 1,2,...,n, (3.22)where u i (t) corresponds to the membrane potential of the units i at time t; g j (•) denotes a measure of response of activation to its incoming potentials; w i j and w τ i j denote the synaptic connection weights of unit j to unit i; τ i j corresponds the transmission delay along the axon of unit j to unit i; the constant I i corresponds to the external bias of input from outside to unit i; and the coefficient d i is the rate with which unit i self-regulates or resets its potential when isolated form other units and inputs.

Application to two-neuron system with delays
In this section, the following two-neuron system with different time delays which is capable of firing or responding continuously with time is considered.Particularly, the firing is modulated by the difference between its current status and a weighted average of the firing history [6]: where "•" denotes the derivative, with t, a 1 , a 2 , b 1 , and b 2 are arbitrary real numbers.In (4.1), x 1 and x 2 denote the mean soma potential of the neuron, while a 1 and a 2 correspond to the range of the continuous variables x 1 and x 2 , respectively.b 1 and b 2 denote the measure of the inhibitory influence of the past history.The terms x 1 and x 2 in the argument of the f and g function denote local feedbacks.In biological literature, such feedback is known as reverberation, while in the literature on artificial neural networks, it is known as an excitation from other neurons (see Figure 4.1).
14 Delay-dependent asymptotic stability The initial condition for (2.1) is given as follows: where Δ ≡ max{τ,σ}, Φ(s) and Ψ(s) are continuous on [−Δ,0].By using the following transformation: we can transform system (4.1) to for t ≥ 0, where a 1 , a 2 , b 1 , and b 2 are real constants.The delays τ and σ are nonnegative constants.The function f and g are continuously differentiable on R = (−∞,+∞) and such that f (0) = 0, g(0) = 0.For (4.1) or (4.4), we assume that the following conditions are satisfied.(A 1 ) f and g are bounded on R, that is, there exist positive constants Q and L such that, for any w ∈ R, However, here we only require that f and g satisfy the conditions (A 1 ) and (A 2 ).Moreover, our model (4.1) and the obtained results are more general than those of [6].
It is also easy to show that (4.4) has always an equilibrium (y * 1 , y * 2 ), that is, there exist y * 1 and y * 2 such that In fact, we consider the map P = (P 1 ,P 2 ) on the compact convex set Ω, where Xiaofeng Liao et al. 15 It follows from (A 1 ) that P is a continuous map which maps Ω into itself.Thus, it follows from Brouwer's fixed point theorem that P has at least one fixed point (y * 1 , y * 2 ) in Ω, that is, This shows that (y * 1 , y * 2 ) satisfies (4.6).In this section, we will consider the stability of the equilibrium (y * 1 , y * 2 ) of (4.4).Let us first consider the case a 1 a 2 (1 − b 1 )(1 − b 2 ) = 1.We further make the following assumption.
(A 3 ) There exist positive constants d 1 and d 2 such that (4.9) Hence, the equilibrium (y * 1 , y * 2 ) of (4.4) is also unique.Similar to the above approach (see Appendix B), we can easily obtain the following.
) and the following assumption (A 4 ) are satisfied: (A 4 ) there exist positive constants d 1 and d 2 such that then, the equilibrium (y * 1 , y * 2 ) of (4.4) is locally asymptotically stable.In general, the delay-independent criteria are particularly restrictive for system parameters.Thus, it is reasonable to apply these criteria first.If they are found inappropriate, the delay-dependent criteria will then be applied.To illustrate the results presented in Theorems 4.2 and 4.3, some simple examples are given and a comparison of the results is given in Table 4.1.
From Table 4.1, we can easily find that the delay-independent conditions given in [6] are not applied and satisfied.This illustrates that the delay-independent criteria are more conservative and restrictive than the delay-dependent criteria.Numerical simulation results are shown in Figures 4.

System Parameters
Results given by [6] O u r r e s u l t s    4.3).This shows that our results are actually rather restrictive and there is much room for Xiaofeng Liao et al. 17 improvement.Numerical simulations also show that for τ = 18, almost all solutions of (2.1) are oscillatory and ultimately tend to some periodic solution (see Figures 4.4 and 4.5).Hence, the problem of whether the delay super-bound is optimal will be studied in a forthcoming paper.

Conclusions
In this paper, we have analyzed a system composed of multiple neurons with time-varying delays in detail.We first obtained the global asymptotically stable criteria dependent on delays for the equilibrium by employing the approach of Lyapunov functional.Our results are delay dependent.Then, we also derived the delay-dependent criteria for local asymptotic stability.Hence, our work complements and generalizes that is reported in [7,8,[11][12][13].In the meantime, we note also that, if the neural system starts with a stable equilibrium, but then becomes unstable due to delays, it will likely be destabilized by means of a Hopf bifurcation leading to periodic solutions with small amplitudes [14,15,18].The analysis of such a bifurcation to find out its bifurcation direction and stability of the periodic solutions is very complicated and lengthy.It is worth studying whether there are other dynamical behaviors, such as codimension-two bifurcations, periodic-doubling bifurcations, phase-locking and quasiperiodic dynamics, and so forth, These works will be studied in the near future.

A. Stability proof
We construct the following Lyapunov function: then its upper right Dini-derivative is Xiaofeng Liao et al. 19 where We note that Note that for sufficiently large t, 20 Delay-dependent asymptotic stability w τ i j q jε τ i j (t) where Furthermore, by (H 1 ), we have for t ≥ T + Δ, Therefore, μ jε (ξ)dξ. (A.11) Hence, where

B. Stability proof
By Appendix A, we can obtain

2
and 4.3.16 Delay-dependent asymptotic stability
Similar to Corollary 3.5, for system (3.22),we have the following.