Rapid parameter estimation of discrete decaying signals using autoencoder networks

In this work we demonstrate the use of neural networks for rapid extraction of signal parameters of discretely sampled signals. In particular, we use dense autoencoder networks to extract the parameters of interest from exponentially decaying signals and decaying oscillations. By using a three-stage training method and careful choice of the neural network size, we are able to retrieve the relevant signal parameters directly from the latent space of the autoencoder network at significantly improved rates compared to traditional algorithmic signal-analysis approaches. We show that the achievable precision and accuracy of this method of analysis is similar to conventional algorithm-based signal analysis methods, by demonstrating that the extracted signal parameters are approaching their fundamental parameter estimation limit as provided by the Cram\'er-Rao bound. Furthermore, we demonstrate that autoencoder networks are able to achieve signal analysis, and, hence, parameter extraction, at rates of 75 kHz, orders-of-magnitude faster than conventional techniques with similar precision. Finally, we explore the limitations of our approach, demonstrating that analysis rates of $>$200 kHz are feasible with further optimization of the transfer rate between the data-acquisition system and data-analysis system.


I. Introduction
Machine learning (ML) is becoming a widespread method for the generation, and analysis of big data. ML uses networks of interconnected neurons (i.e., neural networks) that, much like real brains, recognize patterns in data structures [1,2]. Generally, these neural networks are first trained in situations where the desired action/output of the network is known and subsequently used in similar, real world situations. Research fields where a precise mathematical description of the problem is difficult -if not impossible-stand much to gain from implementing ML solutions. Areas such as image processing [3,4] or text generation and analysis [5] have seen innovations that would not have been possible without the implementation of ML.
In physical sciences, ML-based techniques are becoming more and more widespread (see Ref. [6] for an extended review). Besides unlocking completely new innovations, ML techniques have been utilized in situations where finding solutions to physical problems requires a lot of computing effort using conventional methods. For example, in fluid simulations where normally one is required to solve the Navier-Stokes equations, ML can be used to accurately predict the evolution of a fluid simulation [7], while reducing the amount of computation time of these complex simulations significantly. Furthermore, neural networks have been used to simulate light scattering by multi-layer nanoparticles and the subsequent design of these nanoparticles using backpropagation [8]. In spectroscopy, machine learning is used to accurately classify physical objects based on noisy/complex spectroscopic data [9][10][11]. Another field where ML techniques find their way is in experiments with extremely high inference rates and low latency constraints, such as in highperformance detector triggers [12] (e.g. the ATLAS experiment at the LHC at CERN). In these experiments, field-programmable gate array (FPGA) implementations of ML techniques are used to create complex hardware triggers for events with sub-microsecond lifetimes.
However, ML techniques are notoriously opaque, meaning that, although they deliver desired results, our understanding of how the results are obtained and how patterns in data are detected by the underlying neural network is very limited [13]. Often, little or no information on the physical system that is solved/simulated can be gained by studying the neural network itself, which is a significant drawback of these techniques. Furthermore, basic questions regarding the minimal amount of training data, or the optimal size of the underlying neural network required to obtain good precision for a pre-defined problem remain unanswered [6].
Signal-parameter estimation is an integral part of both fundamental and applied research, with precision, accuracy and speed being crucial for the real-time observation and control of physical and chemical dynamic processes. A multitude of research fields rely on determination of time constants and frequencies of decaying signals, examples of which include: nuclear magnetic resonance (NMR) [14] where molecular structures are analysed from precise determination of the frequency and decay time of the Larmor precession of nuclear spins in a magnetic field; free-induction-decay (FID) optical magnetometry [15][16][17][18][19] where measurement sensitivities depend on the precision of the measurement of the precession of magnetized spins about a magnetic field; and cavity ring-down spectroscopy (CRDS) [20][21][22][23][24] that relies on the detection of variations of the photon lifetime in optical cavities to identify trace gasses, measure absorption cross-sections or observe chemical reactions in real time. Other cavityenhanced methods, such as cavity ring-down polarimetry (CRDP) [25][26][27][28][29][30][31] and ellipsometry (CRDE) [32][33][34], measure birefringence and/or dichorism of an optical medium through the precise estimation of the signal-decay time and the polarization beat oscillations superimposed on such decaying signals.
All the aforementioned methods require some form of data processing to determine their relevant signal parameters. Optimizing sensitivity and minimizing computational costs is usually done by averaging multiple, consecutive measurements, sacrificing temporal resolution. Therefore, it is imperative that one considers the sampling and acquisition rates of an instrument and balances that against the computational costs (calculation time) to analyze the acquired data, the time scales of relevant dynamics to be observed, and the requirement to observe those dynamics in real time. In cavity-enhanced spectroscopy, CRDP/CRDE techniques in particular, the decay times of interest are typically in the 10 −7 − 10 −5 s range, a few orders of magnitude smaller than the typical decay times of NMR experiments (10 −2 − 10 1 s) or FID optical magnetometry techniques (10 −3 − 10 1 s). As such, analysis methods that are sufficiently fast for the real-time analysis of single measurements of NMR or FID magnetometry experiments do not have the capability to FIG. 1. Example of a dense neural network with L = 2 layers. The first layer, l = 0 is the input layer, l = 1 a hidden layer and l = 2 is the output layer. χ x y specifies neuron x in layer y. Each neuron contains a bias and layer specific activation function (not illustrated). ωy,x,z denotes the weight (strength) between neuron x in layer y and neuron z in layer x − 1. For illustrative purposes, some connections have been highlighted and are shown with their corresponding weight. The output of the neuron depends on its inputs, weights and bias put through the activation function (Eq. 1).
offer real-time analysis in situations where the relevant time scales are much smaller than 1 ms. Different timeand frequency-based computational methods for rapid parameter estimation have been demonstrated, and evaluated, for cavity ring-down (CRD) spectroscopy methods [35][36][37]. Notably, Fourier-transform implementations on FPGAs have demonstrated data analysis rates as high as 4.4 kHz [38]. Several works discuss time-and frequencydomain analysis algorithms for damped sinusoidal signals [39,40] and a recently published comparative study includes three analysis methods of discretely sampled damped sinusoidal signals in terms of their speed, and attainable accuracy and precision [41].
In this work we demonstrate a ML based approach to extract the relevant signal parameters from experimentally relevant signals with well-defined functional form. More specifically, we use dense autoencoder networks to encode, extract parameters from, and subsequently reconstruct two types of discretely sampled, decaying signals: (1) exponentially decaying signals and (2) decaying oscillations. We evaluate the autoencoder network on its precision and accuracy in parameter extraction and compare its performance to the fundamental estimation limits of such signals given by the Cramér-Rao lower bound (CRLB). We show that the dense autoencoder network is able to reach analysis rates of 75 kHz of 1000 sample signals with cost-effective computational facilities. This makes the ML method ideal for implementations where computational capabilities come at a premium, such as fast, portable cavity-enhanced sensing instruments.

A. Dense autoencoder neural networks
A neural network is an ordered group of neurons that, much like the neurons in brains, communicate by signaling to each other. Each neuron has a number of incoming and outgoing connections from and to other neurons. Each connection has a strength (weight) and can be stimulating or inhibiting the response of the receiving neuron. To calculate the activity (value) of a neuron, one compares the sum of its weighted inputs to a neuron specific reference (bias). The difference is passed through an activation function which produces the output of the neuron [1,2]. Initially, the weights and biases within a neural network start off randomly. As the neural network is trained, these weights and biases are altered to optimize the performance of the network with respect to the task it is asked to perform. The manner in which neurons are ordered and connected influence the capabilities of the overall network and a variety of different network configurations with different purposes have been demonstrated [42]. A dense neural network is a network where the neurons are arranged in sequential layers and all the layers are fully connected. This means that every single neuron in one layer is connected to all neurons in the next layer. There are no neural connections within a layer or connections spanning multiple layers. An example of a dense neural network is shown in Fig. 1. For such a neural network, one can calculate its output as: where L is the number of layers in the neural network (excluding the input layer), χ l is the output of the network at layer l, ω l is the weights matrix, and b l is the bias vector for the corresponding neurons of layer l. Finally, F l is the activation function of the network layer that acts piece-wise on each neuron in layer l. The input layer (l = 0) has no bias or activation function. Autoencoders are symmetric, hourglass-shaped neural networks [43][44][45][46][47] (Fig. 2). This means that the number of neurons in the middle layer of the network is small compared to the number of neurons in the input and output layer of the neural network. The goal of the autoencoder network is to recreate its input signal. In the first half, the autoencoder network learns to encode the input data into a lower dimensional representation. In the second half, the autoencoder network learns to subsequently decode that lower dimensional representation and reconstruct the original signal with minimal losses. The middle layer of the network, in which the maximal compression of the signal takes place, is called the latent space. The latent representation of the original signal can be extracted from the latent space. Autoencoder networks are typically used in data compression [46] and de-noising [47].
The choice of dimensionality of the latent space is "problematic" [44]. In order to achieve maximum data compression, the number of neurons in the latent space of the autoencoder network should be as small as possible. However one would also like to be able to reconstruct the data with minimal error. If the inherent dimensionality of the data is not known a priori (as is typical) [44,46,47] but important, sparse, or variational autoencoder networks with Bayesian regularization methods should be used [48]. Furthermore, inference from the latent space representation is nontrivial because the way a network "learns" to encode a signal in training is not necessarily unique. Another way of structuring the latent space of an autoencoder network is by including the Kullback-Leibler (KL) divergence during training. However, KL divergence inevitably introduces a trade-off between the ability of the autoencoder network to reconstruct the original signal and a structured latent space [49].
In this work, we examine how an autoencoder network consisting solely of dense layers can be employed for direct extraction of the parameters of a signal with well-defined functional form from its latent signalrepresentation, as this becomes relevant in real-time signal analysis. To achieve such a task, we must make sure that (1) the dimensionality of the latent space matches the dimensionality of the input signal, and (2) that the autoencoder network encodes the input signal in a specific way where the latent representation of the signal matches the independent parameters of the signal.
We are able to satisfy the first requirement because the number of parameters in our signal's functional form are known, which allows us to match the number of neurons in the latent space of the autoencoder network to the number of parameters of the function (Secs. II B & III B). Thereby we achieve efficient encoding without redundant neurons in the latent space, or redundant values in the latent representation of the signal. We are able to satisfy the second requirement by training the autoencoder network in a specific, three stage training method (Sec. III D).

B. Model signals
We use autoencoder networks to estimate the parameters of two types of experimentally relevant model signals: (i) exponentially decaying signals and (ii) decaying oscillations.

Exponentially decaying signals
We start by investigating purely exponentially decaying signals. Such signals are encountered in a wide range of spectroscopic techniques, such as cavity ring down spectroscopy and cavity-enhanced sensing methods [22,23,50] and time-resolved fluorescence spectroscopy [51,52]. An exponentially decaying signal can be characterized in terms of a model function as: where A 0 is the initial amplitude and τ is the decay constant of the signal, y 0 (t) is the signal offset, and t is the independent (time) variable that is discreetly sampled. Here, for simplicity and without loss of generality, we assume that A 0 = 1 and y 0 (t) = 0. Furthermore we restrict the investigation of noise contributions to the global offset parameter, i.e. y 0 (t). We model the noise to be normally distributed, i.e., y 0 (t) = 0, y 2 0 (t) = σ 2 y0 and define the signal-to-noise ratio as: y0 . Under realistic experimental conditions, signalamplitude fluctuations can be incorporated into the SNR through A 0 . This way, the dimensionality of the exponentially decaying signal is reduced to a single parameter that in most cases of interest carries the valuable information: the decay constant τ .

Decaying oscillations
The second type of model signals we investigate are decaying oscillations, as these become relevant in a wide range of experimental techniques, particularly within nuclear magnetic resonance (NMR) [14], optical magnetometry [15][16][17][18][19] and CRDP [25][26][27][28][29][30][31]53] and CRDE [32][33][34]54]. An exponentially decaying oscillation can be characterized in terms of a model function as: were t is the discretely sampled independent variable of the signal (time), A 0 is the oscillation's initial amplitude, τ is the decay constant of the signal's envelope, f is the frequency of the oscillation, and φ its phase. Finally y 0 (t) is the global signal offset. Again, we assume the signals amplitude to be normalized (A 0 = 1) and no global signal offset ( y 0 (t) = 0), and SNR is defined as SNR = σ −2 y0 identical to the exponentially decaying signals. The experimentally relevant parameters τ , f and φ, are the free parameters we extract from the latent representation. We recognize that, similar to the case of pure exponential decays, the amplitude parameter A 0 could potentially be an experimentally relevant parameter for inspection purposes, and could be considered as a free parameter of the model signal.

Cramér-Rao lower bound
The fundamental limits for the statistical uncertainties of determining the decaying time-constant and oscillation frequency of pure and oscillating decaying signals are described by the Cramér-Rao lower bound (CRLB). For the case of a decaying (oscillating) signal (Eq. 2 & Eq. 3) the CRLB [16,55] sets the lower limit on the variance of both the decay constant estimator σ 2 τ and the frequency estimator σ 2 f . In general, the CRLB limit can be defined for any parameter extracted by an unbiased estimator.
The relation between the variance limit of the decaytime estimator and the lower limit of the frequency estimator is given by: σ 2 τ = 2π σ 2 f . The CRLB for the frequency estimator σ 2 f is given by [16,55]: where SNR is the signal-to-noise ratio of the decaying oscillating signal; f BW is the sampling-rate-limited bandwidth of the measurement; T m is the measurement time window and, ξ (τ /T m ) is a correction factor that takes into account the signal decay, which is given by [16,55]: .
The factor ξ (τ /T m ) serves as a compensation factor in Eq. 4 that penalizes measurement of the tails of the exponential decay when the signal has effectively died out. The validity of Eq. 4 hinges on the assumption that the period of the oscillation is shorter than the typical decay time (τ ) of the signal envelope, and that a sufficient number of oscillations occur in the measurement time window. Moreover, Eq. 4 dictates that any noise sources affecting the signal detection are contributing to the fundamental CRLB limit through their effect on the SNR of signal. In Ref. [30], it was demonstrated that the CRLB limit is the appropriate estimator of the fundamental sensitivity of frequency-based measurements within the context of cavity-enhanced spectro-polarimetric techniques [25-34, 53, 54].

A. Simulated signals and data-sets
We use simulated signals generated using the model functions presented in Eqs. 2 & 3 to train and demonstrate the parameter-extraction capabilities of dense autoencoder networks. For our simulated signals -both pure exponentially decaying and decaying oscillating ones -we assume a total signal duration of 5 µs and an effective bandwidth-limited sampling rate of 200 MHz, resulting in discretely sampled signals with a length of 1000 samples. Such signals are frequently encountered in CRDP experiments [30]. We generate training data consisting out of multiple signals with varying experimentally relevant parameters. From the training data the autoencoder network learns the relationship between the shape of the signal and the values of the relevant parameters. For exponentially decaying signals (Eq. 2), the only parameter of interest is the decay constant τ , which we vary between simulated signals as follows: where µ τ and ζ τ are the average and standard deviation of a normal distribution, from which we take the absolute value as a negative decay constant would lead to exponentially increasing signal. For the decaying oscillations (Eq. 3) we vary the decay constant using the same process, but also vary the other two experimentally relevant parameters, the frequency and phase of the oscillation, f and φ respectively, using: For the exponentially decaying signals a training data set consists of 200 signals, while for the decaying oscillations, a training data set consists of 1000 signals. These signals have a SNR = 2 20 , to allow the autoencoder network to learn the features of each model signal and the signals variance under changing underlying parameters of interest without being hindered by noise artefacts.

B. Dense autoencoder networks
The autoencoder networks that we use to reconstruct and extract the parameters from decaying signals, have an input layer and output layer of 1000 neurons, equal to the number of samples in a signal. However the number of neurons in the latent space of the autoencoder networks are different between autoencoder networks that analyze exponentially decaying signals and decaying oscillations. In the case of the exponentially decaying signals, we use a network with only a single neuron in the latent space, corresponding with the single parameter of interest that we wish to extract from the signal, i.e., τ . In the case of the decaying oscillations, we need three neurons in the latent space of the network in order to be able to extract the three interesting parameters of that signal, i.e., τ, f, φ. Furthermore, the signal complexity of a decaying oscillation with three parameters is higher than a exponentially decaying signal with only one parameter of interest. For this reason, to analyze decaying oscillations we increase the number of neurons in the layers of the encoder and decoder part of the autoencoder network (Fig. 2). The network we use to analyse the exponentially decaying signal has 5 total layers with 1000 -50 -1 -50 -1000 neurons per layer respectively. For the decaying oscillations, we use a network with 7 layers and 1000 -50 -10 -3 -10 -50 -1000 neurons per layer respectively. We choose to use a hyperbolic tangent as activation function for all relevant layers. This choice allows the neural network to express values between -1 and 1, which matches the maximal amplitude of the input and output signals. We create the aforementioned neural networks using the Tensorflow and Keras [56] libraries in a homemade Python script.

C. Signal reconstruction and parameter extraction
Autoencoder networks are used to reconstruct the original input data. However, our focus here is to use autoencoder networks for the rapid extraction of the signal parameters from exponentially decaying signals and decaying oscillations from the latent space of the autoencoder network. To do this we map the signal parameters of interest to a number between -1 and 1 that the autoencoder network can express in the latent space: where x lat is the latent representation of the value of parameter x (τ , f, and φ). µ x and ζ x are the mean and standard deviation of the parameter variation defined in Eqs. 6-8. We use Eq. 9 to create a desired latent representation of signal parameters during training, and to convert the latent representation back into the signal parameter when using the autoencoder network to analyse signals. For a trained neural network, a signal only needs to be passed through the encoding part of the network, up to the latent space, in order to find its latent representation. To investigate the analysis speed of the neural networks, we only take into account the time it takes to encode the original signal into the latent representation.

D. Training protocol
The autoencoder networks are trained using a stochastic gradient descent (SGD) algorithm [56] and following a three-stage training scheme. Even though the sizes of the autoencoder networks are different given the two different model signals, the training method for the autoencoder networks is the same: 1. The complete autoencoder network is trained to recreate simulated input signals for 100 epochs.
2. The encoder part of the autoencoder network, up to the latent space (Fig. 2), is trained to generate the desired latent-signal representation from simulated input signals.
3. Finally, the decoder part of the autoencoder network is trained with the desired latent-signal representation as input and the desired signals as output for 100 epochs.
We repeat these steps 10 times after which a new training data set is generated to avoid over-fitting a single data set. In total, each network is trained on 10 different data sets. By implementing three distinct training steps we are able to train the neural network to not only recreate the original signal, but also to encode the signal in a specific way that allows extraction of the signal parameters from the latent space. Figure 3 compares the prediction error (loss) of two autoencoder networks during training. We measure the loss of each neuron in the output layer and the latent space using the mean squared error (MSE) [56]. The smaller the loss of the autoencoder network, the better the network is in recreating the desired output. The first network (red and black dots) is trained using our three-stage training method. The second network (dark-green line) is only trained on replicating the input data (first step). Both networks encode and reconstruct exponentially decaying signals described by Eq. 2. We observe that both training methods converge to similar losses per neuron at the end of training for the complete autoencoder networks (black dots versus dark-green line), meaning that both networks, at the end of training, are able to reproduce the input data with a similar level of precision and accuracy. However, in the autoencoder network that is trained using our three-stage training method, we see a decrease in losses from the encoder part of the autoencoder network (red dots), indicating that by using the three-stage training method we can achieve a desired latent signal representation without sacrifice of the capability of the autoencoder network to reconstruct the original signal.

IV. Results
A. Exponentially decaying signals Figure 4 shows the results for a trained autoencoder network reconstructing and extracting parameters from exponentially decaying signals. The ability of the autoencoder network to extract the signal parameters from the latent space is compared to the performance of an ordinary least squares algorithm [30]. In Fig. 4(a) we show an example of a generated exponential decaying signal with SNR = 2 5 and a randomly selected time constant (τ = 1.81 µs). We use a trained autoencoder network (Sec. III D) to reconstruct this generated signal. In addition, we use a least-squares algorithm to analyze the generated signal, from which we extract the expected decay constant τ = 1.81(2) µs, with a precision in accord to the CRLB limit (see related discussion in Ref. [41]). In Fig. 4(b) we show a histogram of 500 estimates of the decay constant extracted from the latent space of the autoencoder network. By subtracting the least squares decay constant estimate (τ est ) and fitting the histogram to a Gaussian distribution we show that there is no difference in accuracy between the two analysis methods as the center of the distribution is not significant from zero. The width of the distribution of the decay constant estimates coincides with the uncertainty of the least squares estimation method indicating that also the precision of the two methods is equal. In Fig. 4(c) we demonstrate that the trained autoencoder network is able to, accurately and without loss in precision, follow signal changes. In particular, we simulate a spectral absorption feature having a characteristic dispersive (Lorentzian) profile. In this case, the decay constant changes by 30% and both the autoencoder network and the least squares algorithm are able to accurately detect such a change. . This situation is representative of a spectroscopy experiment investigating, for instance, absorption from gaseous species using CRDS [24,57]. In Fig. 4(d) we show that the precision of the neural network is able to match the precision of the least squares algorithm and that both are limited by the CRLB over several orders of magnitude in SNR. It is crucial to emphasize that the network is able to accurately and precisely extract the correct signal parameters over a wide range of SNR values, despite the fact that we do not vary the SNR of the input training data during the training of the network. Histogram of 500 decay-constant estimates extracted from the latent space of the trained autoencoder network. Analysed signals have decay constants and SNR equal to the signal shown in (a). The value of the decay constant estimated by the least squares method, τest, is subtracted from the decay constant extracted from the latent space of the autoencoder network, τenc. The full width at half maximum (FWHM) of the fitted Gaussian distribution coincides with the uncertainty of the least squares method while the center of the distribution is not significantly different from 0. (c) Accurate and precise signal parameter extraction using an autoencoder network is possible in the case of (relative) changes in the decay constant through a simulated absorption feature (dotted line) as one would expect in, for instance, cavity ring-down absorption spectroscopy when a laser is detuned from the resonance frequency of the absorption feature. The feature is followed by both the trained autoencoder network (blue dots) and a least squares algorithm (red dots). The error bars represent the standard deviation of 100 parameter estimations from the latent space of the trained autoencoder network and the least squares algorithm respectively. (d) Precision of a least squares algorithm and a trained autoencoder network on estimating the decay constant from the latent space over multiple orders of magnitude in SNR. The Cramér-Rao lower bound (CRLB) giving the fundamental estimation limit on the decay constant is also shown (dotted line) and the area below the fundamental estimation limit is greyed out.

B. Decaying oscillations
In Fig. 5 we present the results of a trained autoencoder network reconstructing and extracting parameters from decaying oscillations. In Fig. 5(a) we show an example of a decaying oscillating signal with SNR = 2 5 , τ = 1.28 µs, φ = −0.243, and f= 2.972 MHz. We fit the signal to Eq. 3 using a least squares algorithm and use a trained autoencoder (Sec. III D) to reconstruct the signal. Figure 5(b) shows the histograms of 500 parameter estimates of τ , φ (c) Accurate and precise signal parameter extraction using an autoencoder network is possible in the case of (relative) changes in the decay constant through a simulated disperive absorption feature, or cotton effect, (dotted line) as one would expect in a cavity enhanced polarimetry experiment when a laser is detuned from the resonance frequency of the absorption feature. The feature is followed by both the trained autoencoder network (blue dots) and the least squares algorithm (red dots).The error bars represent the standard deviation of 100 parameter estimations from the latent space of the trained autoencoder network and the least squares algorithm respectively. (d) Precision of the trained autoencoder network (blue dots) and the least squares algorithm (red dots) in estimating both the frequency (f) and decay constant (τ ) over several orders of magnitude in SNR. The fundamental estimation limits of these parameters, given by the Cramér-Rao lower bound (CRLB) is also shown (dotted line) and the area below the fundamental estimation limit is greyed out. and f, extracted from the latent space of the autoencoder network. We extract the latent parameters τ lat , φ lat , f lat from the latent space of the autoencoder network and use Eq. 9 to reconstruct the estimate of the signal parameters found by the autoencoder τ enc , φ enc , f enc . Following the same procedure described for the case of pure exponentially decaying signals, for each parameter we subtract the parameter's estimated value obtained through the least squares method and fit the histograms to a Gaussian distribution. Each distribution has a center that is not significant from zero and a width equal to the uncertainty of the fit parameters found by the least squares fit indicating that there is no difference in accuracy or precision between the two analysis methods. In Fig. 5(c) we demonstrate that the trained autoencoder network is able to, accurately and without loss in precision, follow signal changes. In particular, in Fig. 5(c) we simulate a spectral feature having a characteristic change in both its absorption and dispersion (e.g., Cotton effect). This results in signals whose frequency and decay constant change in a correlated way. Finally, in Fig. 5(d) we show that the precision of the parameter estimates from the latent space of the trained autoencoder network follows the precision of the least squares method over five orders of magnitude in SNR. Moreover, the precision of the parameters extracted from the latent space of the autoencoder network approaches the fundamental estimation limit as given by the CRLB. The lengths of these layers range from 10 to 500 neurons, where the smallest network investigated has a single layer with 10 neurons, and the largest network has 3 layers with 500, 200 and 100 neurons between the input layer and the latent space respectively. The dashed and dotted horizontal lines represent the calculation times required to analyse the same signal using a least squares algorithm and non-iterative Fast Fourier transform method, respectively [41].
Analysis speed is crucial for the real-time investigation and control of fast processes. Faster methods of analysis allow for large data stream to be analysed quicker, driving down computational costs.
For our particular cases of interest, we recently showed (Ref. [41]) that non-iterative FFT methods require ∼ 300 µs of calculation time whereas a least-squares fitting algorithms requires > 1 ms of calculation time under identical signal conditions as we present in this work. Similarly, other works have shown that FPGA-based systems running FFT algorithms are able to determine the decay constant of exponentially decaying signals with analysis rates of 4.4 kHz [38]. Here, we show that trained autoencoder networks are able to analyse both exponentially decaying signals and decaying oscillations significantly faster than any previously reported method. We compare the calculation speed of the encoder part of trained autoencoder networks of different sizes on different systems: a central processing unit (CPU-) and a graphics-processing unit (GPU-) based system. The CPU-based system is based around an Intel(R) Xeon(R) W-2123, 3.6 GHz processing unit with access to 16 GB of random-access memory (RAM) with a frequency of 1330 MHz, while the GPU system is based on a NVIDIA Quadro p5000 graphics board.
In Fig. 6 we present the calculation time required by the encoder part of an autoencoder network to analyse a single decaying oscillation signal for varying sizes of networks. For our estimation of the overall CPU calculation time (Fig. 6) we count the matrix multiplications, the adding/subtracting of the network biases and the application of the activation function. Other prepossessing operations for the CPU-based system, such as data collection, take between 3.6 -4.9 µs per signal and are not included. For our estimation of the overall GPU calculation time we include the aforementioned operations (such as matrix multiplications, the adding/subtracting of the network biases, and the application of the activation function) and the transfer time of the results of the encoder onto the RAM of the computer. We do not include, however, the time it takes for the original signal to be transferred from the RAM onto the GPU memory (∼ 5 µs) or any other pre/post-processing operations conducted by the CPU of the system (∼ 6 µs). For the GPU-based system there is a clear difference in calculation time between neural networks with different numbers of layers in the encoder network, something we do not observe for the same neural networks on the CPUbased system. The results we present in Figs. 4 -5 are obtained with autoencoder networks that require the calculation of ∼100,000 floating point operations (FLOPS) to reach the latent space representation of the signal. Using the trained autoencoder networks we are able to achieve analysis rates upwards of 75 kHz using the GPUbased system. This includes the data transfer time, the pre/post processing operations conducted by the CPU, and the actual calculation time required of the autoencoder network itself. By optimizing the transfer rate be-tween the data-acquisition system and data-analysis system and minimizing the prepossessing operations, down to a combined ∼ 3 µs, the method of analysis we propose in this paper has the capabilities to reach analysis rates of >200 kHz, in which we include the <2 µs calculation time required by the system.

V. Outlook and conclusion
In summary we have shown that we can accurately and precisely extract the signal parameters of decaying signals using simple autoencoder networks. We demonstrate that our approach is orders of magnitude faster than conventional algorithmic methods (e.g., least-squares or FFT), regardless of CPU-or GPU implementation of the neural network. We demonstrate analysis rates upwards of 75 kHz for signals with 1000 samples, and illustrate that analysis rates of >200 KHz are feasible with optimization of data transfer speed between a dataacquisition and data-analysis device, which would allow for real time signal analysis rates of >200 kHz using state-of-the-art GSa/s sampling rates. Such capabilities can enable the real-time signal analysis of, for instance, CRDP signals at > 200 KHz rates using state-of-the-art acquisition modules that have GSa/s sampling rates.
Concluding, we wish to note that the methodology of signal parameter extraction directly from the latent space of dense autoencoder networks could be applicable to other signal types that currently use fitting models for parameter extraction. Presently, neural networks are used for the classification of spectroscopic data [9][10][11], however, our approach can be directly implemented to analyse spectroscopic data for quantitative rather than qualitative results. Signals of higher complexity, such as signals with additional decay constants or frequency components, will require larger networks and larger training data sets for the network. If the number of extra parameters is known, a similar technique to what we demonstrate can be employed. However, if the number of additional parameters is unknown, a regularization method should be used to adequately choose the number of latent space parameters of the network to analyse the signal.