Deep learning-based noise reduction for seismic data

An improved noise reduction algorithm based on feedforward denoising neural network (DnCNN) is proposed for the noise removal problem of noisy seismic data. The previous DnCNN originally used for noise reduction of seismic data had the problem of large network depth and thus reduced training efficiency. The improved DnCNN algorithm was first proposed for the noise reduction of natural data sets, and this paper applies the algorithm to the noise reduction of seismic data after adjusting the relevant parameters. The analysis and comparison of the experimental results show that the DUDnCNN algorithm can remove noise with high efficiency, and the algorithm has certain feasibility and significance for further research in seismic data noise reduction.


Introduction
With the increasing difficulty of exploration, seismic data fidelity and noise reduction is still a key technical problem to be solved. As the basis of resource exploration, the accuracy of seismic data is especially important in the subsequent exploration, and the complex geological conditions often interfere with the data acquisition. Therefore, noise reduction of seismic data is a key step to improve the signal-to-noise ratio of seismic data.
As a classical problem in the field of image processing, many noise reduction algorithms have been proposed in the early years. Initially, noise reduction algorithms similar to median filtering [1] and mean filtering [2], which present poor adaptive ability in performing image noise reduction and lack in extracting edge information, were also gradually abandoned. In addition, traditional noise reduction algorithms include f-x domain [3], least square filtering [4], and block matching based threedimensional filter transform (BM3D) algorithm [5]. For noise reduction of seismic data, David Bonar and Mauricio Sacchi [6] proposed a nonlocal mean algorithm to attenuate random noise in seismic data, which has the advantage of noise reduction for all pixels in the image and avoids the possible limitations of noise reduction.CAO et al [7] proposed a second generation wavelet transform algorithm for noise reduction of seismic data, which is a flexible construction algorithm based on the wavelet transform algorithm further improved. Mousavi et al [8] proposed a non-diagonal seismic noise reduction algorithm based on continuous wavelet transform and hybrid block thresholding. Li et al [9] applied the full-variance regularized nonlocal mean algorithm to seismic data noise reduction, which effectively removes the noise while preserving the edges well. Wang Wei et al [10] proposed a noise reduction algorithm based on dynamic clustering with singular value decomposition. Cheng Hao et al [11] applied the adaptive thresholding algorithm based on shearlet transform to seismic data noise reduction. Although many of the above algorithms have achieved good results in seismic data noise 2 reduction processing, further exploration is needed to improve the efficiency and accuracy of noise reduction processing, and it is of great importance to study more accurate and efficient noise reduction algorithms for seismic data.
With the rapid development of computer networks in recent years, more and more deep learning algorithms have been applied to the research of seismic data noise reduction, and good noise reduction results have been achieved. For example, Tang J. et al [12] combined K-SVD denoising algorithm with deep learning network to study the seismic random noise suppression method based on the sparse representation of overcomplete dictionary signals by deep learning. Wang Qiqi et al [13] proposed a noise reduction algorithm for seismic data using multilayer perceptron (MLP), and obtained a noise reduction model with better efficacy after effective model training and parameter tuning. The development of convolutional neural networks (CNN) based on artificial neural networks has also been explored by many scholars in seismic data noise reduction. Han Weixue et al [14] proposed a random noise removal algorithm for seismic data based on CNN and compared it with traditional denoising algorithms such as wavelet transform and curvilinear wavelet transform, and CNN demonstrated superior denoising effect. Mandelli et al [15] also studied a convolutional neural network structure called U-Net to achieve noise reduction and interpolation of seismic data. It can be seen that CNN with strong feature learning capability is widely loved by scholars in seismic data noise reduction research. CNN has many kinds of structures, and DnCNN is a classical feedforward denoising convolutional neural network, which is a more advanced denoising algorithm in the field of deep learning at present. Some scholars have previously taken the lead in applying DnCNN to seismic data noise reduction research and achieved good application results, but it is more difficult to train due to the deeper network structure.
The DuDnCNN network framework used in this paper is a fusion of two networks, DnCNN and U-Net. The network initially showed good noise reduction when used in the noise reduction processing of the natural dataset BSDS300, and the network structure is relatively simple with small depth, which can improve the training efficiency to a certain extent. Therefore, the network is applied to the seismic data under study after adjusting the relevant parameters. The experimental dataset is a set of underground random seismic data provided by the official website of the kaggle competition, and we add noise to the clean data to form a noisy dataset and a clean dataset. The test set and result analysis can verify the feasibility of DUDnCNN algorithm in the field of seismic data noise reduction processing.

2.1.Principle of noise reduction
Seismic data subject to noise interference can be simply defined by the following equation： Where x is the original noise-free seismic data, y is the noisy seismic data, and n is the noise, usually n is the additive Gaussian noise obeying the normal distribution. Based on the meaning expressed in the above equation, the main purpose of noise reduction is to recover x from y as much as possible, i.e., the ultimate goal is to remove the noise from the noisy data by noise reduction processing, so that the obtained seismic data is as close as possible to the original seismic data without noise, and the seismic data used in the subsequent experiments in the paper are all seismic images.
Using neural networks to further improve the denoising model into a learnable process, the following equation can be obtained: Where i x is the original seismic profile image without noise,   i n x is the noisy image after noise is the forward propagation process of the neural network, and  refers to the weights.

2.2.Original DnCNN network
The network structure of DnCNN is shown in Figure 1 below. There are three main parts of the network, the first part C1 layer consists of Conv and ReLu activation layers, Conv is mainly used for the extraction of data features in the learning process, ReLu is a common activation function that can effectively prevent gradient explosion and zero out all negative values. The second part of the C2 layer contains 6 layers, each consisting of Conv, BN and ReLu activation layers, with one more BN layer than the C1 layer. The main role of the BN layer is to readjust the data after convolution and perform data normalization. The third part of the convolutional layer is used for image reconstruction, where noisy images are learned after multi-layer convolution to obtain noisy images. A global jump is constructed between the input and output of the network, and the noise-bearing image is reoperated with the learned output noise image to obtain a clean denoised image, which is the characteristic of DnCNN residual learning. low-frequency information of the data is gradually perceived as downsampling progresses, upsampling is a process of decoding and recovery, and the connection is cleverly added in the middle to fully combine the different levels of information obtained, so that the network can learn more comprehensive information about the data.
(2) Expanded convolution: The DUDnCNN network uses the expanded convolution operator in the design of the convolution kernel. Downsampling is the size of the convolution kernel in the left half of Figure 2 after adding the interval. The size is 3 3  , 7 7  and 15 15  , the gradually increasing receptive field can better extract the low-frequency information of the image data, and the convolution kernel size design of the up-sampling process in the second half is just the opposite of the left half, which is also the process of image restoration.

Experiment
In this section, the details related to this noise reduction experiment are introduced, including the preparation of the dataset and the training and testing of the noise reduction network. In the test and result analysis section at the end of this section, the noise reduction effect plots of DUDnCNN with different parameters are compared and analyzed, and the result outputs of both DnCNN and DUDnCNN are also compared to demonstrate the advantages of DUDnCNN in seismic data noise reduction.

4.1.Data preparation
(a) Clean image (b)Noise image Figure.3 Comparison of images before and after adding noise The dataset used for the experiment is the dataset provided by Kaggle's official website in the salt body segmentation competition. The data is a set of randomly selected images at various locations in the underground, and the images are all 101 101  pixels. A total of 3100 images were selected from the dataset, of which 3000 images were used as the training set and 100 images formed the test set, all of which were clean images without noise. In the data processing, Gaussian noise is first added to the 3000 images of the training set to create noisy images, so the processed training set consists of 3000 clean and 3000 noisy data, and the test set is operated in the same way. Figure 3 shows the clean images in the original data set on the left and the noisy images on the right after adding Gaussian noise. Figure 4 shows a flow chart of the training and testing process in the experiment, with the training process in the upper half. The image size in the training process is101 101  , and the output image is also the same size. The training process is to input the noisy data from the training set into the improved DnCNN noise reduction network in batches, extract and remove the noise features to obtain a clean image, and then calculate the error between the clean image and the clean image in the training set before adding noise, and then back propagate to update various parameters in the network, and loop this process until the error is minimized and stop iteration, at which time the model should reach the optimal convergence. The model should reach the optimal convergence effect, and the parameters of the network training process are set as shown in Table 1.

4.3.Test and Result Analysis
In this section, we mainly input seismic data from the test set into the network for testing and compare the noise reduction effect of the DUDnCNN algorithm with different parameters and optimizer settings. The noise reduction results of the two optimizers are shown in Figure 5. (a) shows the noise reduction effect of the Adam optimizer, which combines the advantages of both AdaGrad and RMSprop and is popular for its excellent convergence speed. (b) shows the noise reduction effect of AdamW optimizer, which is a variant of Adam, and its weight decay and L2 regularization can effectively alleviate the overfitting problem in training, and the generalization ability of Adamw is also stronger. We can also see from the training loss curve at the bottom of Figure 5 that AdamW can reduce the loss to a smaller size and converge faster at the same epoch. However, the noise reduction effect of both optimizers has the same problem, that is, the effective information is corrupted during the noise reduction process, and we need to adjust the learning rate and epoch to improve the problem. Figure 6 shows the noise reduction effect of the two algorithms DnCNN and DUDnCNN. For the training and testing of the same data set, DnCNN shows the phenomenon of smoothing over and overfitting. And the loss change curve of DUDnCNN shows that the training loss remains stable at 20 iterations has basically converged, and the peak signal-to-noise ratio has also improved significantly before and after iteration, but although we try to alleviate the problem of effective information being removed by adjusting the learning rate and Batch size several times.The noise reduction effect of the DUDnCNN algorithm in Figure 7 still loses more effective information than the original seismic data figure (a). Therefore, the fidelity of DUDnCNN while ensuring the noise reduction efficiency is the next step to be solved.

Conclusion
To address the problem that the excessive depth of the noise reduction network affects the training efficiency, this paper proposes to apply the DUDnCNN with smaller depth to seismic data noise reduction. DUDnCNN combines the advantages of DnCNN batch normalization with end-to-end U-Net neural network, and the skip-connection operation of the network avoids the loss of shallow information in the feature learning process, which can effectively extract feature information, and the convolution The dilation convolution operator is added to the kernel, which makes it possible to increase the perceptual field in the convolution process. Six thousand 101 101  seismic data are input to the network as the training data set, which contains 3000 noise-free data and 3000 noise-containing data. The error between the network output and the initial noise-free data is back-propagated and tuned to minimize the error, and the optimized network model shows a good noise reduction effect after tested by test samples. Experimental results show that DUDnCNN algorithm has better noise reduction efficiency, but still has shortcomings in fidelity. Although the algorithm has presented near-optimal noise reduction in the BSD300 dataset, further network adjustment and better parameter selection to achieve optimal noise reduction in the application of noise reduction in seismic datasets are urgent problems to be solved.