Extraction of Signals Buried in Noise: Non-ergodic Processes

In this paper, we propose extraction of signals buried in non-ergodic processes. It is shown that the proposed method extracts signals defined in a non-ergodic framework without averaging or smoothing in the direct time or frequency domain. Extraction is achieved independently of the nature of noise, correlated or not with the signal, colored or white, Gaussian or not, and locations of its spectral extent. Performances of the proposed extraction method and comparative results with other methods are demonstrated via experimental Doppler velocimetry measurements.


Introduction
Quality of signal information in different areas of science is degraded by encountered various natures of noises.This degradation may take different forms and evolves to observations where the time-averaged correlation function of a process is different from the ensemble-averaged function [1,2].Non-linear filtering and their corresponding asymptotic stability are generally proposed to handle such processes [3][4][5].
In situations where signals are totally buried in noise and for which no a-priori information is available, ergodic hypothesis cannot be validated.Moreover, none of the above methods is suitable for extraction of such buried signals.In real world situations, non-ergodic processes in which a desired signal is buried may occur as shown by Doppler velocimetry measurements [6][7][8][9][10][11] used in this work.
It is crucial to notice that by the terms extraction of signals, we mean extraction of clean spectra of buried signals in noise and by buried signals, we mean signals defined for low or extreme low signal-to-noise ratio.In [12,13], we proposed two equivalent extraction methods, called respectively "modified frequency extent denoising (MFED)" and "constant frequency extent denoising (CFED)".For easy reference, let us recall that the frequency extent is the interval   0, e f where e f is the sampling frequency of the buried continuous-time signal to be extracted.The first procedure (MFED) is based on modifying the sampling frequency and the second one (CFED) is suited to a collection of available sample noisy processes for which the sampling frequency is constant.Clearly, CFED offers implementation simplicity since in most applications, signals are defined for a constant sampling frequency.Proposed extraction does not use any a-priori information on the signal to be extracted, works without averaging or smoothing in the direct time or dual frequency space, and it is achieved independently of the nature of noise (colored or white, Gaussian or not) and locations of its spectral extent.Extraction methods in [12,13] are based on the theory of ergodic stationary processes.Extension of these results to extraction of signals correlated with noise in which they are buried are reported in [14].We have shown that extraction of signals correlated with noise is achieved without averaging and independently of the nature of noise (correlated or not, correlated or not with the signal, colored or white, Gaussian or not) and locations of its spectral extent.
In this work, we extend results of one of the aforementioned extraction methods (CFED), to a more general case where ergodic hypothesis, given experimental conditions, cannot be validated, which is a fact and an issue.

N. Y. BEY
In Section 2, we recall principal definitions and results of the CFED extraction method since this method offers implementation simplicity as mentioned above.In Section 3, expression of extracted spectra of buried signals from non-ergodic processes is derived without any sort of averaging or smoothing in the time or frequency domain and without assuming the signal uncorrelated with noise.Extraction performances and comparative results with other methods [15][16][17][18] are discussed and illustrated in Section 4. It is shown that CFED extracts the Doppler mean frequency from experimental Doppler velocimetry measurements for which ergodic hypothesis is not validated or satisfied.

Signal Representation
Let us consider a finite observation of z t , a process containing a band-limited signal buried in zero-mean wide sense stationary noise , in the interval of length with m .The process , available at the output of a low-pass filter of cut-off frequency max f , is given by, where and

The Sample Power Spectral Density (SPSD) [12]
Given , we can form the estimate, where DFT denotes Discrete Fourier Transform of .Here the estimate depends on the frequency, f , the sampling frequency, e f , and the length of the observation interval, .
T It is crucial to notice that (2) is not a power spectral density in the usual sense.In [12], (2) is defined as the "Sample" power spectral density or the sample spectrum of the process .In the following, we recall for easy reference the CFED extraction method.

 
N z n

CFED Extraction Method
Here, we have a collection of  realizations of dura- tion of a noisy process so that the length of the total observation interval is T T  .These  realizations de- noted where

Sample Spectrum of Noise
We found that the sample spectrum of noise obtained by Fourier transformation of (3) is given by [12], where  are translated copies of the original sample spectrum of noise whose components are spaced with the mutual distance 1/ on the frequency axis.
For the sake of simplicity and without loss of generality, we let, Notice that a justification of (6) is found in [14].Factors   p   reduce indifferently translated copies the original sample spectrum of noise independently of their nature (white or colored, Gaussian or not) and act indifferently at all frequencies. .Since  zeros are distributed in  frequency locations created in each interval of length 1 T (see [12]) then,

Spectral Distribution
depicted by (10), is given by,  

Extraction Procedure
Extraction of the sample spectrum of the buried signal is obtained by decimation.This decimation by the factor  is applied in the frequency domain to the Fourier transformation of (3).The signal-to-noise ratio   of the decimated spectrum written as a function of the signal-to-noise ratio of the original spectrum is given by, represent the amplitude spectra of respectively the signal and noise.Here Now, let us find the optimal form under which expression of the CFED sample spectrum is written only as a function of the sample spectrum of the signal and noise independently of any correlation between the signal and noise and without averaging in the time or frequency domain.
We have shown in [12] that increasing  increases the signal-to-noise ratio of the original noisy spectrum in which the desired signal is buried.Moreover, the variance of sample spectral estimates of noise tends to zero as  increases.

Non-Ergodic Processes
Expression of the sample spectrum of noise of a non-ergodic process is obtained by using the above expression (4) derived under the ergodic assumption.In (4), one finds translated copies of the original sample spectrum of noise, denoted in the ergodic case by Here, we extend above results obtained for ergodic stationary processes to processes for which ergodic hypothesis cannot be validated or satisfied.We consider therefore that we have a collection of  sequences whose probability density functions that describe noise affecting them are different from sequence to an other one.This means that any sample process can be put under the form, Now, in the non-ergodic framework, as depicted by (10), it is crucial to notice that we have  realizations defined for different and unknown noise distributions.As in (10) the index (p) identifies each noise distribution, here, we introduce the index (p) in order to identify their corresponding  original and different sample spectra.
We can therefore rewrite (4) by identifying translated copies of original sample spectra of noise by their corre- where is the signal defined above and is the additive noise specified for the realization (p).Moreover, probability density functions describing for are assumed unknown.Clearly the process from which samples are taken is stationary and not ergodic.Covariances of the are dependent on the sample process (see p. 89 of [19], for the definition of non-ergodic processes).For implementation simplicity, we use hereafter the CFED method, recalled above, where those where, , , = , , .

Decimated CFED Sample Spectrum
The decimated N-point sample spectrum applied to
  is the  -decimation applied to  .
where   

CFED Sample Spectrum
Notice that since the power spectral density of the signal is assumed constant in collected  sequences then decimated sample spectrum of the signal yields the same The sample spectrum of the process of duration T  , as e e e e f f T result as (7).
On the other hand, since we have  translated and different copies of original sample spectra of noise then decimation in the frequency domain yields a spectrum in which contribute coefficients of those  copies of original spectra of noise.This means that the coeffi-cients of the decimated spectrum described by (12) are those of the translated copies . Notice that this applies also to the amplitude spectra of noise  , , e  f f T   .This means that by setting  , the decimated sample spectrum of noise in (14) becomes, , , = , , = 1 , , ; , , , where the set of the copies is given by , , ; , , where 2 min  and 2 max  are respectively the smallest and the greatest noise variances contained in the collection of  sequences.
It is crucial to notice that ( 16) is at the heart of this work.According to (16), we can assume, for the sake of simplicity and without loss of generality, that the decimated sample spectrum of noise approaches the average sample spectra of the translated copies of noise.Under this assumption, justified by (16), we can write the variance of the decimated sample spectrum of noise


are Fourier coefficients of the translated copies  .Now, according to (17), the sample spectrum written as a function of its Fourier coefficients, yields, , , ; , , = ,  . Notice that the ergodic case [12,13] can be obtained from ( 18) by setting    

Optimal CFED Sample Spectrum
Let us in the following find a condition under which the decimated sample spectrum as depicted by ( 19) can be written without rectangular terms (cross-products).The optimal form is that for which the sample spectrum is described only as function of the spectra of the signal and noise.
where k c is defined in (18).By using ( 22), the sample spectrum, as given by ( 19), yields therefore explicitly,

The Optimal Reduction Factor
In the following, we derive the optimal reduction factor or the optimal number of concatenated sample processes  under which contribution of the cross-products in ( 19) is made negligible.Coefficients of the last right-hand side of (23) can be put under the form, where Now, let us find the condition that defines the minimum value of  under which ( 27) is smaller than unity, i.e., We propose to find  as a function of the mean signal-to-noise ratio of the  collection of different processes.Let us define the mean signal-to-noise ratio of the collected  sample processes by, is the mean variance of noise of the collected  sample processes.Here   where k  and k c are arbitrary chosen coefficients and, where and   represent respectively the number of components of the signal and noise.
It is easy to see that I  is bounded by,

  
, min max , where   denote respectively the minimum and the maximum values of the set formed by s k   , for and = 0,1, , Since k  is an arbitrary chosen coefficient and according to (31), we can consider that, Similarly, = c I  .The signal-to-noise ratio, as depicted by (29), becomes, = .
For a useful interpretation of (34), we express the optimal reduction factor  only as a function of the mean signal-to-noise ratio  .By setting Here (36) depicts the optimal reduction factor  as a function of the mean signal-to-noise ratio  of the collection of  non-ergodic sample processes.This opti- mal reduction factor represents therefore the number of concatenated sample processes.


 represents the number of concatenated sample processes under which the decimated sample spectrum of the process becomes optimal or be written under the form (39).
It can be seen that under the condition (36) contribution of the cross-products is made negligible.Expression of the extracted spectrum of the buried signal from non-ergodic processes, as depicted by (39), can be obtained without the requirement based on ensemble averaging and without assuming the signal uncorrelated with noise.Moreover, at the limit of large values of  , (39) becomes, The sample spectrum of noise, independently of its nature, vanishes.The decimated spectrum is identical to the original deterministic spectrum of the signal.Results (40) and ( 39) achieve extraction of buried spectra of signals from non-ergodic processes.

Experimental Signals: Doppler Velocimetry
It is well known that Ultrasonic Doppler velocimetry provides a non-invasive method for measuring direction and speed of fluids.Information of interest on Doppler signals may be found, for example, in [6][7][8][9][10][11]. Signals used here are issued from a flow measurement apparatus that uses pulsated emitting source with a constant Pulse Repetition Frequency (sampling frequency) PRF = 8 kHz.A fluid runs at constant or quasi-constant speed and the concern here is to measure its mean speed [6,9].The Doppler mean frequency 3.3 f  is related to the mean velocity of the flow by (see [10] among above references), where is the frequency of the pulsated emitting source, is the velocity of the ultrasound wave and

Non-Ergodicity of Doppler Sequences
In Figure 1, zero-mean four realizations (a1)-(a4) of Doppler velocimetry signals carrying information on the speed of the fluid are depicted.Here we assume that the nature of noise (white or colored, correlated or not) affecting the Doppler mean frequency is unknown.
Moreover, to accentuate non-ergodicity effect, let us add to the first Doppler sequence, (a1), the third one, (a3), and the last one (a4), three different colored noise sequences   where   u n is a random signum function (logical function which extracts the sign of a uniformly distributed random number) and   e n is white Gaussian noise sequence.The length of these sequences is .N We assume that signal-to-noise ratios of these Doppler sequences are unknown.However, mean powers of collected Doppler sequences can be computed.These are variances of noise since the Doppler signal carrying the Doppler mean frequency is very weak.We have (1.14 dB), (-2.2 dB), (-1.55 dB) and a (1.5 dB).This creates variations of the signal-to-noise ratio.According to (39), we have


, the mean signal-to-noise ratio for which an extraction from noise is possible is > 0.25  (-6 dB).If no extraction is obtained, this means that the mean signal-to-noise ratio of Doppler sequences is lower than -6 dB.
In Figure 1, histograms (b1), (b2), (b3) and (b4) of the four Doppler sequences are shown.It can be seen that we have four different and deformed noise distributions with varying amplitudes.In (c1), (c2), (c3) and (c4), covariance functions of the Doppler sequences are depicted in logarithmic scales for better visibility (low correlation lags) and for comparison purposes.These covariance functions are clearly dependent on the sample Doppler process.We have therefore non-ergodic Doppler velocimetry measurements (see p. 89 of [19]).

Extraction Results
In Figure 2, we propose extraction of the Doppler mean frequency by using CFED method, Welch PSD estimation [15] and the Thomson's multi-window method (MTM) [17].We recall that MTM uses a bank of windows that compute several periodograms of the entire ignal and then averaging the resulting periodograms to s  construct a spectral estimate.In order to minimize the bias and variance in each window, theses windows are chosen orthogonal.Optimal windows that satisfy these requirements are Slepian sequences or discrete prolate spheroidal sequences [18].Now, these four realizations  are concatenated and the obtained process, as given by (10), is analyzed by the above methods.The spectrum of CFED concatenated realizations and its decimated version are respectively shown in Figures 2(a It can be seen that independently of the nature of noise (white or colored, correlated or not) affecting experimental signals and variation of the signal-to-noise ratio, the Doppler mean frequency is clearly extracted by CFED from non-ergodic process with an excellent signal-to-noise ratio without any averaging in the time or frequency domain and without using any a-priori information on the signal (Doppler mean frequency) and the nature of noise in which the signal is buried.

Conclusions
In this work, extraction theory of signals buried in non-ergodic processes is proposed.We have shown that no a-priori information on the signal to be extracted is used and no averaging in the direct time or frequency domain is performed.Observed results on experimental Doppler velocimetry measurements buried in non-ergodic processes show that extraction of the Doppler mean frequency is achieved independently of the nature of noise, correlated or not with the signal, colored or white, Gaussian or not, and locations of its spectral extent.Observed results are in accordance with theoretical predictions.
respectively the additive noise (white or colored) and the signal observed in the time interval of length .T By considering the instants = Spectral lines of each copy of the original sample speclines of noise separated by the mutual distance 1 T are now distributed in new  frequency locations created in each frequency interval of length 1 T .On the other hand, the spectrum of the signal   T s t as given by the transformation of concatenated realizations (3), is specified by   S f f T (21)then according to (20) and (21), we have, represents the mean power of the signal and

p 2 
is noise variance of the sample process  

≫
in accordance with (34), then by choosing  exceeding the upper bound of the variation interval of min  condition (28) is fulfilled.

Figure 1 .
Figure 1.Four Doppler velocimetry measurements (a1), (a2), (a3) and (a4) in which the Doppler mean frequency is buried in white and colored noise.Non-ergodic character is shown by different histograms and corresponding different covariance functions (in logarithmic scale).

Figure 2 .
Figure 2. CFED spectrum and its decimated version are shown in (a) and (b) where the Doppler frequency is extracted (7.55 kHz).Comparison with Welch PSD and the Thomson's multitaper method is provided in (c) and (d).

4   = 4 
) and (b).The frequency of the depicted spectral line (Doppler mean frequency) is   =  f  = 7.55 kHz.The obtained varia-tion with respect to the above expected Doppler frequency is 0.13%.The Welch periodogram estimation shows however a widened peak located at 7.4 kHz with a weak signal-to-noise ratio whereas the MTM method depicts with a much weaker signal-to-noise ratio a spectral line far from the expected Doppler mean frequency.