Stochastic Resonance with Colored Noise for Neural Signal Detection

We analyze signal detection with nonlinear test statistics in the presence of colored noise. In the limits of small signal and weak noise correlation, the optimal test statistic and its performance are derived under general conditions, especially concerning the type of noise. We also analyze, for a threshold nonlinearity–a key component of a neural model, the conditions for noise-enhanced performance, establishing that colored noise is superior to white noise for detection. For a parallel array of nonlinear elements, approximating neurons, we demonstrate even broader conditions allowing noise-enhanced detection, via a form of suprathreshold stochastic resonance.


Introduction
Stochastic resonance has emerged as a significant statistical phenomenon where the presence of noise is beneficial for signal and information processing in both man-made and natural systems [1][2][3][4][5][6][7][8][9][10][11]. The excitable FitzHugh-Nagumo (FHN) neuron model has been discussed for exploring the functional role of noise in neural coding of sensory information [12]. Following this, the milestone concept of aperiodic stochastic resonance using the FHN neuron model [13] stimulated a number of interesting investigations in sensory biology [3,7,14,15] and physiological experiments [6,8,[16][17][18][19][20]. Due to the character of activity in the nervous system, the neuron coding strategy based on stochastic resonance is also found in threshold (level-crossing) [21][22][23] and threshold-free [24][25][26][27][28] neurons. Since there are large numbers of neurons in the nervous system of animals and humans with variations in structure, function and size [2,4,[7][8][9], then the potential exploitation of stochastic resonance in a neuron bundle becomes an interesting open question in neuroscience. In a general summing neural network, Collins et al. [4] reported that the noise intrinsic to each neuron could be used to extend the operating range of the sensitivity of the overall system. This, however, is not a unique case. In the summing multi-threshold network, suprathreshold stochastic resonance discovered by Stocks [29] overcomes the restriction of subthreshold signals, and appears to offer a possible explantation of dc adaptation in sensory neurons [9,30]. One-dimensional coupling [31] and spatio-temporal stochastic resonance [7,32] show that not only an optimal noise intensity but also an optimal coupling strength exists. Recent stochastic resonance research in complex networks [33][34][35][36][37][38][39][40] also demonstrates that an interconnected network configuration, as well as the non-zero noise level, can be optimized to achieve the best system performance.
In many practical situations, the idealization of white noise is never exactly realized [2,5]. Consequently, the effect of colored noise on stochastic resonance has been investigated using the measure of output signal-to-noise ratio of a periodic signal [2,5,16,[41][42][43]. Although the suppression of stochastic resonance with increasing noise correlation time was demonstrated [2,5,[41][42][43], it is interesting to note that, under certain circumstances, colored noise can be superior to white noise for enhancing the response of a nonlinear system to a weak signal [16,44]. In the field of signal detection, the employment of noise to enhance signal detectability also becomes a possible option [45][46][47][48][49][50][51][52][53][54][55]. However, in most of these studies, the observed noise samples are often assumed to be independent. Colored noise for signal detection [56][57][58][59][60] is not adequately investigated in the context of stochastic resonance. In this article, we focus on the weak signal detection problem with the beneficial role of additive colored noise in threshold neurons. Because of the ''alland-none'' character of nerve activity [61], the problem of thresholdbased neural signal detection can be considered as a statistical binary hypothesis test [7,23,27]. In this situation, explicit expressions for the maximum asymptotic detection efficacy are derived for a given transfer function of neuron model. We prove that colored noise that arises from a moving-average model is superior to white noise in improving the detection efficacy of neurons. It is illustratively shown that, for a single neuron with a signum threshold nonlinearity, the possibility of noise-enhanced detection only holds for non-scaled noise. For scaled noise, the effect of noise-enhanced detection does not occur in a single neuron model. However, when we tune the internal noise components of a parallel array of threshold neurons, it is observed that the constructive role of noise comes into play again in improving the signal detection efficacy, wherein suprathreshold stochastic resonance manifests its potentiality.

Detection model
Consider the detection problem formulated as a binary hypothesis test [23,60,62] H 0 : x~z, Under hypothesis H 0 , the observation vector x~½x 1 ,x 2 , Á Á Á ,x N T consists of noise z~½z 1 ,z 2 , Á Á Á ,z N T only, and under hypothesis H 1 it consists of noise z and known signal s~½s 1 ,s 2 , Á Á Á ,s N T with its strength h. There exists a finite bound U s such that 0ƒjs n jƒU s , and the asymptotic average signal power satisfies 0vP s~l im N?? s T s=Nv? [56][57][58]60,62]. Next, the test statistic T(x) is compared with a decision threshold c to decide the hypotheses, as where the coefficient vector c~½c 1 ,c 2 , Á Á Á ,c N T is associated with the function g(x) to form T(x). Assume the N-dimensional probability distribution f (z) of noise z and zero-mean vector of E½g(z)~Ð g(z)f (z)dz (for a shift in mean) [58,60]. Then, for a large sample size N of observation vector x, the test statistic T(x) has zero-mean and asymptotic variance under hypothesis H 0 . Furthermore, for weak signals (h?0) and under hypothesis H 1 , g(x) can be expanded to the first-order Then, the characteristics of T(x) under H 1 , up to the first-order in h, can be calculated as Under both hypotheses H 0 and H 1 , the test statistic T(x), according to the central limit theorem, converges to a Gaussian distribution. Thus, the binary hypothesis test of Eq. (1) becomes a Gaussian mean-shift detection problem [60,62]. Given the false probability, the detection probability is a monotonically increasing function of the detection efficacy j(T) [60,62] given by where the Cauchy-Schwarz inequality yields with the Fisher information matrix J~E½ Note that the equality of Eq. (6) is satisfied by the locally optimum nonlinearity for an arbitrary constant C. However, a complete closed-form noise distribution f (z) may be unavailable, especially in unknown noisy circumstances [56][57][58]60,62], which makes the nonlinearity of Eq. (8) difficult or too complex to implement. Thus, there may be compelling reasons for considering the given function g with an easily implemented feature. In this case, the detection efficacy in Eq. (6) can be maximized as with the Cholesky decomposition of the variance matrix V~E g(z)g(z) T Â Ã LL T and by optimally choosing the coefficient s for an arbitrary constant k.

Colored noise
Consider a useful colored noise model of the first-order movingaverage [56,59] as where the correlation coefficients are r 1,2 and w~½w 1 ,w 2 , Á Á Á ,w N T is an independent identically distributed (i.i.d.) random vector. For small values of r 1,2 (jr 1,2 j%1), the dependence among noise samples z i will be weak [56,59]. Here, we assume w i have an univariate distribution f w (w) that is symmetric about the origin. We also assume the memoryless nonlinearity g(z)~½g(z 1 ),g(z 2 ), Á Á Á ,g(z N ) T to be odd symmetric about the origin. Then, up to first order in small correlation coefficients r 1,2 , we can expand g(z i ) as and obtain expectations E½g Therefore, we have the expectation matrix with the unit matrix I, and the variance matrix V~E g(z)g(z) T Â Ã has elements for i~1,2, Á Á Á ,N. Then, based on Eq. (9), the normalized detection efficacy e(g) can be computed as Here, when the equality of Eq. (19) is achieved, the signal s is the corresponding eigenvector to the minimum eigenvalue l min of the matrix V. It is known that the eigenvalues of the matrix V are [62] l i~Vi,i z2V i,iz1 cos pi Nz1 corresponding to eigenvectors v i~s in (pik=(Nz1)) for k~1,2, Á Á Á ,N. Here, as the nonlinearity g is assumed to be odd, it is then found that E½wg(w) §0 and E½g 0 (w) §0. Therefore, if r 1 zr 2 v0 and for a large sample size N, we take i~1 and l min~l1 &E½g 2 (w)z2(r 1 zr 2 )E½wg(w)E½g 0 (w). Otherwise, we choose l min~lN &E½g 2 (w){2(r 1 zr 2 )E½wg(w)E½g 0 (w). An illustration of the eigenvector v N is shown in Fig. 1 for N~200. In this way, by optimally choosing the input signal (eigenvector) s~v 1 (v N ), the maximum efficacy e(g,r) can be calculated as e max (g,r)~max s e(g,r) Since, from its definition in Eq. (6), the efficacy e(g,r) is nonnegative, the denominator in Eq. (21) must satisfy E½g 2 (w){2jr 1 zr 2 jE½wg(w)E½g 0 (w)w0: In order to validate Eq. (22), we use the Cauchy-Schwarz inequality to yield with the Fisher information quantity J(f w )~E½(f 0 w =f w ) 2 and the variance s 2 w of noise distribution of f w [56]. Thus, we find Substituting Eq. (25) into Eq. (22) and noting we have Since we assume jr 1,2 j%1, the inequalities of Eqs. (27) and (22) can be satisfied, and the detector efficacy in Eq. (21) will be theoretically analyzed in the following. For white noise vector z with zero correlation coefficients r 1,2~0 , the detection efficacy e(g,r) in Eq. (21) satisfies e(g,r)~E 2 ½g 0 (w) E½g 2 (w) ƒe max (g,r): Thus, for a given function g, colored noise is superior to white noise in enhancing the detection efficacy, at a cost of optimally matching the input signal with the eigenvector v i of covariance matrix V.

Stochastic resonance in threshold-based neurons
We will illustratively show the possibilities of noise-enhanced detection in threshold-based neurons. The classical McCulloch-Pitts threshold neuron has the form with the response threshold '. It is seen that g can be expressed as a function of x in terms of the signum (sign) function as g(x)~1 2 sign(x{')z 1 2 . Since the constant factor 1=2 does not affect the detection efficacy of the transfer function g, then we focus on the signum function with response threshold '~0 in the following parts. Here, the signum function g is not continuous at x~0, but has its derivative g 0 (x)~2d(x) for any x [60].
For the colored noise model of Eq. (10), the correlation coefficient r 2~0 indicates the noise sequence z i is a causal process that can be physically realized. Here, we assume r 1~r (jrj%1) and r 2~0 , and show the possibility of stochastic resonance in the physically realizable noise environment. First, consider scaled noise w(t)~s w w 0 (t) that has the distribution f w (w)f w0 (w=s w )=s w [23,60]. Here, w 0 (t) has a standardized distribution f w0 with unity variance s 2 w0~1 . Thus, based on Eq. (21), the absolute moment is where the operator E w 0 ½ : ~Ð : f w 0 (w 0 )dw 0 . Thus, for the signum function g, the detection efficacy of Eq. (21) can be expressed as It is seen in Eq. (32) that e max (g,r) is a monotonically decreasing function of noise variance s 2 w , and no noise-enhanced detection effect will occur in such a single neuron model for scaled noise.
We further consider non-scaled Gaussian mixture distribution [5,47,48,60] where the variance s 2 w~m 2 zE 2 and parameters m,E §0. Then, for the signum function g in Eq. (30), the detection efficacy of Eq. (21) can be computed as where the error function erf(x)~2= ffiffiffi p p Ð x 0 exp ({t 2 )dt. In Fig. 2, for the correlation coefficient jrj~0:2 and different values of m~0:3,0:5 and 1, we show the detection efficacy of Eq. (34) as a function of noise variance s 2 w . For a given non-zero value m and as E?0, the noise distribution model of Eq. (33) indicates the dichotomous noise [5,48,53]. In this situation, as the signal strength h?0 and jhs n j%m, the signum function g will not change its output whether the signal appears or not. Therefore, the test statistics T(x)~c T g(x) in Eq. (2) will be the same value under hypotheses H 0 and H 1 , and the detection efficacy e max (g,r) in Eq. (34) starts from zero. This explantation can be also validated by Eq. (34) as E?0 and m being fixed, as illustrated in Fig. 2. However, it is clearly seen that, upon increasing the noise variance s 2 w~E 2 zm 2 (actually increasing E), the noise-enhanced detection effect appears. The smaller the parameter m is, the more pronounced the resonant peak of e max (g,r) becomes, as shown in Fig. 2.
Next, an interesting problem is that, for scaled noise, can we observe the noise-enhanced detection effect in threshold-based neurons? The answer to this question is affirmative. Here, we will resort to the constructive role of internal noise for improving the performance of an array of threshold neurons. Let For the colored noise model of Eq. (10) with r 1~r and r 2~0 , we have where the composite noise v i~wi zy i has the convolved distribution f v (v)~Ð f w (v{u)f y (u)du. Then, we have expectations and with the operator E v ½ : Then, based on Eqs. (38), (39) and (40), the maximum efficacy e max (g,r) can be computed by Eq. (21) as e max (g,r)~max For instance, we assume the initial Gaussian noise components w i have the distribution of f w (x)~1 ffiffiffiffiffiffiffi ffi ) and the given variance s 2 w . The internal noise components of each neuron is assumed to be the uniform random variable y i with its distribution f y (x)~1=(2b) for {bƒxƒb and zero otherwise. The composite random variables v i are distributed by For a given Gaussian noise level s w~0 :3, it is shown in Figs. 3 (a) and (b) that the maximum detection efficacy e max (g,r) varies as a function of internal uniform noise level b for different array sizes M and correlation coefficients r. It is noted that, at the uniform noise level b~0, the detection efficacy in Eq. (41) is just the expression of Eq. (32) for a single neuron. Thus, e max (g,r)~9:49 and 11:45 for jrj~0:2 (see Fig. 3 (a)) and jrj~0:3 (see Fig. 3 (b)), respectively. By comparing Fig. 3 (a) with Fig. 3 (b), it is seen that the maximum detection efficacy e max (g,r) can be further enhanced for a higher value of r. For the array size M~1 and upon increasing uniform noise level b, it is seen in Fig. 3 that there is no noise-enhanced effect in a single neuron. However, as M §2, it is illustrated in Fig. 3 that the internal uniform noise can enhance the detection efficacy e max (g,r), and the noise-enhanced effect does occur. Moreover, as the array size M increases, the noise-induced enhancement becomes more visible by adopting an appropriate amount of uniform noise of the neuron array, as shown in Fig. 3. As the detection problem so far is confined to the weak signal with its strength h?0 but hw0, and the response threshold ' of all neurons is zero, thus Fig. 3 shows the potential capability of suprathreshold stochastic resonance in improving the detection efficacy of a parallel array threshold-based neurons.

Conclusion
In this paper, we study the performance enhancement of threshold-based neurons for detecting weak signals in the presence of colored noise. For a given transfer function, we maximize the detection efficacy by optimally choosing the signal waveform. We prove that colored noise is superior to white noise in enhancing the  detection efficacy, at a cost of optimally matching the input signal with the eigenvector of the covariance matrix. Furthermore, we illustrate that, for a single threshold neuron, the possibility of noise-enhanced detection cannot occur in scaled noise, but does appear in a non-scaled Gaussian mixture noise model. Furthermore, for scaled noise, we can test a parallel bundle of neurons with the same response threshold, and recover the positive role of internal noise in enhancing the detection efficacy of the neuron array via the mechanism of suprathreshold stochastic resonance. These results demonstrate that the strategy of exploiting stochastic resonance is still interesting in the case of improving the nonlinear system performance by adding more noise to the signal corrupted by colored noise.
Here, we mainly consider the first-order moving-average noise model of Eq. (10) which is, as we show, amenable to analytical treatment. It is possible to extend the present approach to higherorder moving-average noise models. However, the same analytical treatment maybe no longer feasible. It is also interesting to consider yet other models of colored noise to enhance the detectability of the neuron array. This subject is very promising and currently under study.
It is noted that the detection efficacy of Eqs. (6) and (9) are established under the assumption of weak signal strength h?0. We only consider the first-order Taylor expansion of nonlinearities in Eq. (4), because it makes an analytical treatment possible and the corresponding results are rigorous. In practice, most noise distributions are symmetric and the nonlinear characteristics are odd symmetric about the origin. In this case, we can expand the nonlinearity to the second-order terms. The expectation of the second-order term of Taylor expansion of Eq. (4) vanishes and does not affect the conclusion of this paper. However, for unsymmetrical noise distributions and nonlinearities, the highorder terms of Taylor expansion of Eq. (4) are not exactly zero. For this case, we need to numerically observe the effect of highorder terms on the detector performance. It is interesting to compare the present theoretical results of first-order expansion with the numerical results in the further studies.
We also note that these equations of Eqs. (4)-(9) are the extension of white noise [54,[56][57][58]60] to the case of colored noise. Then, we consider a model of colored noise allowing for an analytical evaluation of the detection efficacy in Eqs. (6) and (9). The detection efficacy can also be numerically computed to address other models of colored noise, or to explore broader conditions beyond the weak signal limit. As the signal strength h increases, the Taylor expansion of Eq. (4) and the upper bound of Eq. (6) gradually cease to apply. However, based on the present results on weak signal in colored noise, and on [25,26,63] on nonweak signal in Gaussian white noise, it can be expected that noise benefit as reported here will persist with colored noise beyond the small-signal limit.