Research on the Application of Super Resolution Reconstruction Algorithm for Underwater Image

: Underwater imaging is widely used in ocean, river and lake exploration, but it is affected by properties of water and the optics. In order to solve the lower-resolution underwater image formed by the influence of water and light, the image super-resolution reconstruction technique is applied to the underwater image processing. This paper addresses the problem of generating super-resolution underwater images by convolutional neural network framework technology. We research the degradation model of underwater images, and analyze the lower-resolution factors of underwater images in different situations, and compare different traditional super-resolution image reconstruction algorithms. We further show that the algorithm of super-resolution using deep convolution networks (SRCNN) which applied to super-resolution underwater images achieves good results.


Introduction
The Ocean is the heart of the planet. Water covers more than two-thirds of the Earth's surface. Sea plants, like posidonia, produce 70% of the oxygen we breathe, and the deep waters are home to wildlife and some of the biggest creatures on earth. It provides us with food, jobs, life, entertainment, and sailing. The ocean is also the mainstay of transportation, providing the most economical and convenient transportation route for human exchanges across the sea. And with the continuous development of the international situation, Ocean has gradually become the new strategic center of the world. Therefore, the ocean has become an important field for human beings to explore the mysteries of nature and develop advanced technologies. Since seawater is a special medium, it is difficult to observe underwater targets. People have used sonar technology to detect underwater targets, but the results are not satisfactory. Due to the special transmission characteristics of light in water, direct underwater images quality is often poor. On the one hand, different wavelengths of light have different attestations when transported in water, which causes degradation of the original color of the object in underwater imaging. In general, the transmission performance of the red light is the worst in water, and that of blue light is the best. On the other hand, due to the scattering effect of water molecules and suspended particles on the light, much stray light is scattered into the camera lens, which is often referred to as background light. Background light can fog the image, causing a reduction in image contrast. Therefore, the contrast of the water absorption and back-scattering winter solstice is reduced, and the forward scatter causes the image detail to be blurred, and the resolution of the sensor is limited, resulting in low-resolution imaging with blurred image and serious noise, and the characteristics of blue or green hue. There are many ways to improve image quality, and we can start with hardware devices, such as to upgrade the camera or camcorder equipment, but the cost is higher, and the effect is not obvious. It is necessary to use image processing technology for image denoising, enhancement, restoration and super-resolution creation to obtain high-quality images, enhance imaging capabilities and underwater target recognition without increasing cost, and improves the efficiency of underwater missions. And then the super-resolution reconstruction of underwater image is an efficient method to improve the quality of underwater image. The super-resolution reconstruction of images is a research hot spot in the field of image processing this year. It is not only theoretically significant, but also gets good effect and benefit in application. The super-resolution reconstruction of the image is to restore the resolution loss of the image to a certain extent as much as possible, that is to say, to restore the low-frequency information of the image in the transmission bands and to recover the high-frequency information above the cut-off frequency obtain more details and information of the reconstructed image, which is closer to the ideal image. Superresolution reconstruction removes the sharpening of the imaging system and restores spatial frequency information beyond the diffraction limit of the optical system.

Related works
Due to the effects of attenuation and scattering, underwater visibility in clear seawater is 20 m, while visibility in turbid seawater is only 5 m. The absorption and scattering of light in water severely limits the performance of underwater imaging systems. Forward scatter causes blurring of image detail, and back-scattering results in reduced image contrast and a fog effect on the image surface. In addition, due to the wavelength dependence of light attenuation, as the distance increases, color loss and distortion become more and more serious. Due to the shorter wavelength, the blue light travels farthest in the water, so the underwater image tends to have a blue hue. In short, there are some problems with the imaging of underwater images, such as when shooting underwater, the imaging distance is relatively close, then underwater images have low contrast, uneven illumination, blurred details, color degradation and more noise. It is found that the reduction in horizontal contrast in underwater imaging is directly dependent on the attenuation coefficient of light [Zaneveld and Pegau (2003)]. The computer simulation and analysis of underwater imaging is based on the establishment of underwater imaging optical model. The computer model of the classical underwater imaging system was proposed by McGlamery in 1979[McGlamery (1979], which states that the optical radiation received by the imaging system consists of three parts: direct attenuated scene radiation, scattered scene radiation, and back-scattered background light. The input parameters of the model include the coordinate relationship of the imaging system, the light source parameters, optical parameters of water, and so on. Based on the model, the article proposed a computational model for underwater imaging simulation [Jaffe (1990)], which takes into account the intrinsic optical parameters and the table sightseeing parameters of the water. In order to study the influence of illumination conditions on underwater imaging, proposed a computer model of underwater active illumination imaging [Lin, Seet and He (2005)]. At the same time, Boffetyet al. proposed a simulation calculation model of underwater RGB images under artificial illumination, analyzing camera response function and selecting the number of discrete points, and the influence of optical parameters of water on imaging results [Boffety, Galland and Allais (2012)]. The complete underwater scene imaging system design mainly includes Funk' model [Funk, Bryant and Heckman (1972)] and Bonin system [Bonin, Burguera and Oliver (2011)]. Since the degradation of underwater images is mainly caused by light scattering effects, many scholars have specifically studied the effects of light scattering on imaging. Wang et al. studied laser active illumination imaging technology using near-infrared laser as illumination [Wang, Jin and Yang. (2005)]. Yang et al. used the atmospheric backscattering geometric model to derive the approximate expression of the cumulative effect of back-scattering [Yang, Sun and Fu (2009)]. The calculation results showed that the back-scattered light in the range of 50 m accounts for 80% of the energy of the backscattered light in the optical system. Wu et al. started from the illumination model of active optical imaging system, and researched the target and scene imaging characteristics in the case of large field of view. Also they analyzed the attenuation effect of the transmission medium on the illumination and imaging beam, and obtained the image illumination of the optical imaging system [Wu, Zhang and Song (2000)]. The mathematical model was analyzed in detail for its contrast-transfer characteristics and the effect of the field of view on imaging. Huang Jin proposed an underwater forward scatter imaging model based on spatial broadening. The spatial broadening was represented by the modulation transfer function proposed by Wells, and the contribution of water reflected light to the illumination of the imaging surface was considered. Super-resolution reconstruction (abbreviated SR) is a technique for obtaining highresolution images from low-resolution images or image sequences. The complementary information between image sequences is used to recover the cutoff frequency information caused by diffraction limit. The idea of super-resolution reconstruction appeared in the 1960s, but the formal research began in the 1980s. Almost all existing single-frame super-resolution algorithms rely on offline databases and use specific operators to extract information from the image as an input feature of the model. Yang et al. used the sparse expression method to extract the high-frequency information of the low-resolution and high-resolution training sample pairs and compiled them into a dictionary [Yang, John, Huang et al. (2010)]. Then, the same method was used to extract the high-frequency information in the test image, and the corresponding high-resolution image was reconstructed. On that basis, the article performed principal component analysis on the high-frequency information to reduce the dimension and then perform sparse coding, which improved the training speed of the algorithm and reduced the influence of noise on the algorithm [Zeyde, Elad and Protter (2010)]. However, using the dictionary method to describe the manifold space where the natural image was located, and was still not accurate enough. Therefore, in the method of Timofte' model, the atoms in each dictionary are selected as the "anchor point", and the surrounding area was L2 constrained. Linear regression had achieved better results than the dictionary. It used random forests to characterize the manifold, and it could also achieve good results as a nonlinear regression. The RAISR model proposed by Romano et al. [Romano, Isidoro and Milanfar (2017)], simplifies the coding process and adds many engineering techniques to greatly improve the reconstruction speed while sacrificing the reconstruction accuracy. Other methods based on deep neural network models could achieve the best reconstruction quality, but because of the many parameters of the model, its training was extremely dependent on a good hardware platform and sample set. Therefore, it was not suitable for some low-end imaging devices such as mobile phones. It should be noted that the iterative back projection algorithm proposed by Yang et al. [Yang, John, Huang et al. (2010)]. It was widely used in the post-processing of the above-mentioned super-resolution reconstruction algorithm and generally improves the reconstruction effect, and did not require training. It could be said that it was a simple super-resolution algorithm. Leding's algorithm based on confrontation learning could obtain super-resolution images closer to natural pictures, but due to the large number of high-frequency information being fabricated, the peak signal-to-noise ratio was relatively low, more similar to image generation problems than super-resolution problems. Applying image super-resolution reconstruction technology to underwater imaging had not been studied by many scholars. Firstly the image resolution reconstruction technology itself is in the development stage. Secondly, the resolution parameters of underwater imaging images are not important for image sharpness and contrast, and underwater image restoration is still in the mainstream research position. Since resolution is also an important parameter of underwater images, and super-resolution reconstruction does not conflict with image restoration or even organically combines, therefore, this topic first focuses on the introduction of classical and deep convolutional neural network algorithm image super-resolution reconstruction technology. So under the water the point spread function obtained by underwater modeling is used as the prior knowledge of image superresolution reconstruction, which makes the reconstruction more specific and effective.

Underwater image and degradation model
Because of the complexity of the composition of water, the process of light transmission in water is very complicated, and the absorption and scattering are much stronger than those in atmospheric medium. The mathematical model of underwater channel (as shown in Fig. 1) is obtained by studying the physical mechanism of underwater imaging. There are two kinds of reasons for image degradation, then one is the influence of the system transmission process caused by water absorption and effective scattering, and the other is the noise caused by the invalid scattering and background light. The system H (x, y) is superimposed with additive noise n (x, y) to form the degraded image g (x, y), that is, the actual image. The mathematical expression for this process is: g(x,y)=H[f(x,y)]+n(x,y) (1) H [•] can be understood as a function that synthesizes all degenerate factors. If H [•] is a linear operator, it is satisfied that: (2) where k 1 and k 2 are constants, with: of which h(x, y; α, β) = H[δ(x − α, y − β)] Called the impulse response of h(x,y). In the optical process of image formation, the impulse is a spot of light. And so it is h(x, y; α, β) called the point extension function in the degenerate process. If H(x, y) is at the same time position invariant, that is, H[•]satisfies: Therefore, the image restoration process can be regarded as a priori knowledge of g (x, y) and h (x, y), n (x, y), and the problem of f (x, y) can be solved.

Application of super-resolution image algorithm in underwater image
At present, only a few scholars have studied the application of advanced image super-resolution reconstruction algorithm in underwater images to obtain high-quality underwater image targets. The main reason is that the image super-resolution reconstruction technology itself is in the stage of development. Secondly, the degradation model of underwater image needs to be analyzed in detail and the resolution parameters are not clear, so the contrast is important. Because the resolution is also an important parameter of underwater image, the super-resolution reconstruction can be compared with the advanced neural network method to obtain the best scheme. Traditional methods include frequency domain method, non-uniform image interpolation method, convex set projection method and maximum posterior probability method. With the development of computer technology, especially the extensive application of machine learning and deep learning in recent years, convolutional neural networks are applied to image super-resolution reconstruction (abbreviated SRCNN). In Dong's paper Dong et al. [Dong, Change, He et al. (2014)], it proposes a deep learning method for single image super resolution by learning an end-to-end mapping between the low/high-resolution image. The method is that the mapping is represented as a deep convolutional neural network which takes the low resolution image as the input and outputs the highresolution one. The method jointly optimizes all layers, unlike traditional method. In terms of network structure, the network structure of the method is simple in design, and the sparse coding process is regarded as a convolution operation. Three convolutional layers are used to complete image patch extraction, nonlinear mapping and image reconstruction. In the optimization method, the traditional shallow learning SR method represented by SCSR separately designs and independently optimizes the three stages, focusing on dictionary learning optimization and neglecting the optimization of the overall framework. In the training phase, in order to avoid the problem of increased computation caused by overlapping image block post-processing, this paper randomly clips high-resolution samples into several sub-images, and then passes Gaussian kernel filtering, down sampling and up-sampling sub-images. In the training strategy, the method uses the mean square error as the loss function, and the stochastic gradient descent (abbreviated SGD) minimizes the loss function, and the training speed is further improved. The method is characterized by the association of traditional sparse coding with deep learning SR methods (as shown in Fig. 3). Although this method shows a better reconstruction effect, SRCNN does not achieve better results while deepening the number of network layers.

The data set
The images of underwater are collected from the internet. These images were crawled using a web crawler program. A total of more than 200 images were crawled.

Loss function
The loss function we used in our experiments is the same as the loss function proposed in. Given a set of high-resolution images {Xi} and their corresponding low-resolution images {Yi}, we use Mean Squared Error (abbreviated MSE) as the loss function: where n is the number of training samples, the loss is minimized by using stochastic gradient descent with the standard back propagation.

Experimental results
We firstly apply the SRCNN to simple images including underwater images of signal thing, all using the same sampled from training images. We will further demonstrate the method to handle complicates textures images, with the same sampled from training.

Conclusion
The problem of underwater images is those of detail and color distortion because of attenuation of light transmission through water. We researched the degradation model of underwater images, and presented super-resolution under images by convolutional neural network framework technology to remove noise. We get some underwater images using a web crawler program, and some of them are trains data set, and others are testing data set. Results showed that the proposed method is better than traditional methods.