Stability of the dynamics of an asymmetric neural network

We study the stability of the dynamics of a network of n neurons intercting linearly through a random gaussian matrix of excitatory and inhibitory type. Using the aproach developed in a previous paper we show some interesting properties of the dynamic of this system for large values of n. We got sufficient conditions for getting diverging synchronized behavior or stability.


Introduction
The dynamic of a system of neurons described by their electric potentials x i (t), i = 1, . . . , n interacting linearly through a random matrix has been extensively studied in the past literature and received increased attention in the last times, see for example ([1], [2], [5]), [3]). The first statement about the stability of the solution of the system was enunciated by May ([9]). Here J ′ is a real symmetric n × n matrix with independent gaussian elements and EJ ′ ij = 0, EJ ′ 2 ij = 1/n. The conjecture was that if κ > λ max , where λ max is the maximum eigenvalue of J ′ , then the solutions of the system (1.1) were stable. This conjecture has been proved by us many years later in the paper ( [7]) where the self-averaging property of the system have been used. In particular we introduced the random counting measure where θ(x) is the standard Heaviside function. This function N n counts the fraction of the electric potentials x i (t) which are less than a given threshold λ. The self-averaging property of N n means that N n (t) → EN n (t) in the L 2 norm with respect to the probability measure of the gaussian matrix, E being the expectation with respect to the probability of all the random entries of the matrix J ′ . In ( [7]) we were able to proof this property, so the random measure becomes asymptotically a gaussian distribution function with mean value a(t) and dispersion σ(t): e −(x−a(t)) 2 /2σ(t) 2πσ(t) . (1. 3) The results of the calculations were that a(t) = e −κt and σ(t) = e −2κt J 0 (wt) where J 0 is the Bessel function of zero order. From the asymptotic behavior of J 0 we get that if κ > w σ(t) goes to zero and we get a stable solution. It is a quite remarkable coincidence that this nice result depends on the self-averaging property of the system, this shows the real power of such property if one reminds all the rigorous properties which have been possible to show in the field of Statistical Mechanics of disordered systems. In this paper we look for analogous results when a different matrix J ′ is considered. The elements of J ′ are still independent but each row of the matrix have mean values depending on the column index: ij } = a · µ I /n 1/2 , j = 1, . . . [f n], µ E /n 1/2 , j = [f n] + 1, . . . , n. (1.4) The first [f n] columns represent inhibitory interaction (µ I < 0) while the other n − [f n] are excitatory interaction (µ E > 0), thus each neuron i receives [f n] inhibitory inputs of the same type from the other neurons and n − [f n] excitatory inputs form the other neurons, the excitation and the inhibition do not depending on the particular neuron i. With this choice the matrix J ′ is asymmetric and the variances of the matrix elements of J ′ also follow the same choice: Thus we look at the same property as before in the case of this new matrix which includes inhibitory and excitatory inputs which is nearer to realistic neural interactions. In order to understand better the new kind of stability properties that we obtain let us introduce some more definitions. Let m = (m 1 , . . . , m n ) be the vector defined by and M be the matrix with all rows equal to m. Then in the paper, following the ideas of (([10]), we introduce the decomposition in order to have all the eigenvalues included in the circle with radius one in the complex plane. The two theorems shown in this paper describe in detail the stability and asymptotic properties of the dynamic associated to the matrix J ′ and in particular these properties are very sensible to the choice of the initial conditions for the x i (t). We give here some hint since the complete definitions will be given in the next section. So suppose that the initial conditions can be written in the following way: where {ξ i } are independent random variables with distributions {ν i }, satisfying the conditions and the initial constants depend on the neuron in the same way as the In this situation the theorems proved in the paper establish that the contribution to the dynamic of the matrix J is stable if κ > σ * = f σ I + (1− f )σ E . Remark that since the matrix J is asymmetric the stability of the dynamic does not depend on the maximum eigenvalue so it is reasonable that the stability results for the dynamic generated by this matrix is different from the one enunciated above. The matrix M also contributes to the dynamic and, due to its particular form, gives some unexpected result, namely if c I = c E the average of the contribution of M to the dynamic goes like t √ n so it is divergent for large n but it goes in any case to zero due to the multiplication with the exponential e −κt . Thus we expect in this case to have large coherent motions which are then dumped by the exponential factor. If c I = c E the situation is completely different because the term of the order t √ n is multiplied by a constant equal to zero and disappears. In this case the contribution of M converges to a gaussian random variable with zero mean and variance of the type of a constant +J 0 ( √ σ * t) and so we get the usual stability result. These are the meaning of the theorems demonstrated in the paper.
2 Notations and formulations of the results Consider a dynamical system with random interactions (so-called a complex system in [9]) defined by where x ∈ R n , κ is a real number and J ′ is an n × n real random matrix. Following Rajan and Abbott (see ( [10])) we consider the case, when J ij are independent Gaussian variables with mean values Here 0 < f < 1 and a > 0 are fixed parameters, and µ I and µ E are chosen so that the vector m = (m 1 , . . . , m n ) The variances of J ij are chosen as follows It is easy to see that in this case the matrix J ′ could be represented in the form where the matrix M is a rang one matrix all rows equal to m, so that 8) and the matrix J has the form where W a Gaussian matrix with independent entries satisfying conditions with σ j defined in (2.6).
As it was shown numerically in the paper [10], the matrix J ′ under conditions (2.2)-(2.6) has a spectrum which is not localized in some fixed domain of C and so it is difficult to expect that the dynamics of the system (2.1) will be stable. But if we introduce the additional equilibrium conditions Ju = 0 ⇔ n j=1 W ij = 0 (i = 1, . . . , n), (2.11) then the spectrum J ′ coincides with the spectrum of J, which is well localized according to the results of [?, ?].
Hence e t(J+M) = e tJ + a t 0 dsMe sJ (2.13) and the solution of the system (2.1) could be represented in the form where x(0) is a vector of initial conditions. Thus to study the dynamics (2.1) it suffices to study the dynamics of the system with a matrix J of the form (2.9), and W, satisfying conditions (2.10) and (2.11).
Supply the system with the initial conditions where {ξ i } are independent random variables with distributions {ν i }, satisfying the conditions and (2.18) Define the normalized counting function of x i , solutions of the system (2.15), where θ(x) is the standard Heaviside function. N n (λ, t) is a random measure on the real line which counts the fraction of the variables x 1 , . . . , x n which are less then λ at time t. Thus it characterizes the distribution of x i (t) on the real line.
Theorem 1 Consider the system (2.15) with a matrix J of the form (2.9) under conditions (2.10) and (2.11), and supply this system by the initial conditions (2.16)-(2.18). Then for any t > 0, where N I (λ, t) and N E (λ, t) are the convolutions of the initial distribution ν I and ν E with normal distributions N (c I ,σ(t)) and N (c E ,σ(t) respectively and the varianceσ(t) has the formσ (2.23) Theorem 2 Consider the system (2.15) with matrix J of the form (2.9) under conditions (2.10) and (2.11), and supply this system by the initial conditions (2.16) with (2.17). Set If c I = c E , then w n (t) for each fixed t converges in distribution to a Gaussian random variable with zero mean and varianceσ where A and σ * are defined in (2.23).

Proofs
First of all we need to compute the expectations of E{W ij } and E{W ij W kl } under conditions (2.11) Lemma 1 (i) Under conditions (2.11) and (2.10) (ii) Consider the random variable of the form

3)
where the coefficients d k do not depend on {W 1k }. Then z is a normal variable with zero mean and the variance Proof of Lemma 1 The first equality in (3.1 is evident, because conditions (2.11) are symmetric with respect to the change W ik → −W ik . Besides, since different lines of the matrix W have independent entries it is evident that for i = k E{W ij W kl } = 0, and for i = k E{W kj W kl } do not depend on k. Hence, To prove the assertion (ii) of Lemma 1 we compute by the same way the characteristic function of z Lemma 1 is proved.
Below it will be convenient to consider the matrix J in the new orthonormal basis. Denote where {e 1 , . . . , e n } is the basis in which we consider the system (2.15) initially, so that Then define in E 1 and E 2 the orthonormal systems {u 3 , . . . , u [f n]+1 } and {u [f n]+2 , . . . , u n } which are orthogonal to the vectors e 1 + · · · + e [f n] , and e [f n]+1 + · · · + e n respectively. If we denote then, according to (2.5), (2.4) and our choice of u 3 , . . . , u n , the system {u i } n i=1 forms an orthonormal basis in R n . Let (u 1i , . . . , u ni ) be the components of the vector u i in the basis {e 1 , . . . , e n }. Then the matrix . . , n, σ I σ E /σ * , j = 2.
Using the result of [?], according to which under condition matrixJ 1 with i.i.d.
and the fact that ||J || = ||J||, we obtain now that
Proof of Lemma 2. It is evident that {W ki } n k,i=1 have joint Gaussian distribution, so to prove Lemma 2 it is enough to compute their covariances. To this aim we use relatioñ and Lemma 1. Then from the first equality of 3.1 we derive that the mean values of {W ki } n k,i=1 are equal to zero.
Now we use the fact that for different k 1 and k 2 W k 1 l 1 W k 2 l 2 are independent and for k 1 = k 2 E{W k 1 l 1 W k 1 l 2 } does not depend on k 1 (see (3.1)). Substituting 3.1) in (3.15), summing with respect to k 1 and using the orthogonality of u i 1 and u i 2 , we get Now if j 1 = 1, then u l 1 j 1 = n −1/2 and summation with respect to l 1 gives us zero because of (3.2). If j 1 = 3, . . . , [f n] + 1, then, since u l 1 j 1 = 0 for l 1 ≥ [f n] + 1, we have that in the r.h.s. of (3.16) σ l 1 = σ I , and so, using the orthogonality of u j 1 and u j 2 , we get the second line of (3.11). If j 1 = [f n] + 2, . . . , n the proof is the same. Now we are left to prove the last line of (3.11). Using (3.16) and (2.4), which gives us we obtain Proof of Theorem 1 Let us consider the system (2.15) from the second equation to the last one as a system of equations for x 2 (1), . . . , x n (t), where x 1 (t) is a known function. Then and J (1) is the matrix which we obtain from J replacing the first line and the first column by zeros. Substituting this expressions in the first equation of (2.15), we get wherer (1)  Using Lemma 1, it is easy to see that Indeed, according to (3.22), Hence, if we take the eighth power of (3.22) and take the expectation with respect to {W 1j }, we get E{(r (1) n (t)) 8 } = (3.25) where J (1)T means the transposed matrix of J (1)T , D is a diagonal matrix such that and here and below we denote by C(t) function of t (different in different formulas), such that C(t) ≤ Ce ct with some n-independent C and c. The relation (3.25) and a trivial crude bound where L is defined by (3.13). Then, by a standard argument, we get This bound allows us to write (3.23) as Now we can apply Lemma 1, which gives us that the sum in the r.h.s. of (3.29) is a normal random variable with the variancẽ Lemma 3 Under conditions of Theorem 1 R n (t, s) Besides,
Repeating (3.44) for all terms in (3.41) with different i, j, we obtain the inequality Iterating this inequality M = [log n] times, we get (3.33). The proof of the inequality (3.34) follows from the representation (3.42) immediately, if we use the independence of x 1 (0) and x 2 (0). Lemma 3 is proved.
Now we are ready to prove the self averaging property of N n (λ, t), as n → ∞, i.e. we prove that for any real λ and t > 0 According to the standard theory of measure, for this aim it is enough to prove that g n (z, t) -the Stieltjes transform of the distribution N n (λ, t) Then repeating the arguments (3.41)-(3.43), we obtain whereσ n is defined by (3.36) and we denote Since evidently we get from (3.49) Now the assertion of Lemma 4 follows from (3.38).
Using (3.35), the s.a. properties (3.33), and the fact that the system (2.15) is symmetric with respect to x 1 , . . . , x [f n] and with respect x [f n]+1 , . . . , x n , we obtain Repeating our conclusions for x n (t), we get i (s)ds (i = 1, . . . , n − 1), = (e tJ (n) x(0)) i and the matrix J (n) is obtained from J by replacing the last line and the last column by zeros. Then, applying Lemma 1, we obtain that the second sum in (3.52) is a Gaussian random variable with the same varianceσ n (t) (see (3.51)).
Equations (3.31) and (3.52) combined with (2.16) give us we obtain from (3.31) and (3.52) the system of equations (3.56) Then we obtain that the function satisfies the equation As it was proved in [7] this equation has the unique solution Then we can easily find that Using (3.45), and the symmetry of the problem we obtain that N n (t, λ) converges in probability to But by the above arguments

Theorem 1 follows.
Proof of Theorem 2. To prove Theorem 2 it is convenient to consider the system (2.15) in the basis {u i } n i=1 defined above (see (3.7)-(3.9). Let Then the system (2.15) takes the form y ′ =Jy, (3.64) whereJ is defined by (3.10). The question of interest is the behavior of y 2 (t). Repeating for y 2 (t) the arguments (3.19)-(3.24) we get the representation y 2 (t) = y 2 (0) +J 22 where andJ (2) is the matrix which we obtain fromJ replacing the first and the second lines and the first and the second columns by zeros. Taking the expectation in (3.65) we get the first statement of Theorem 2. Now assume that c I = c E = c. Then, repeating arguments (3.22)-(3.28), we get that y 2 (t) = y 2 (0) + n j=3W 1j n 1/2 d j (t) +ε n (t), (3.67) and E{(ε (2) n (t)) 2 } ≤ C(t)n −1 . (3.68) Applying Central Limit Theorem to the r.h.s. of (3.66) it is easy to obtain that y 2 (0) converges in distribution to a Gaussian random variable with zero mean and the variance Besides, since {W 2,j } are independent Gaussian random variables, the sum in the r.h.s. of (3.65) is a gaussian random variable with the variancẽ u kj u k ′ j . .
Finally we getσ (1) (t) = (3.70) Since the sum of independent Gaussian variables is a Gaussian random variable with the variance equal to the sum of variances, we obtain that w n (t) converge in distribution to a Gaussian random variable with zero mean and the variancẽ σ (0) =σ (1) (t) + σ y .
The second assertion of Theorem 2 follows.