Restoration of degraded underwater images with attenuation coefficients as a clue

Underwater image restoration methods require the knowledge of wideband attenuation coefficients per color channel. In this paper, a Generative adversarial network(GAN) framework is proposed for underwater image restoration using the attenuation coefficients as an input. Attenuation coefficients like atmospheric light and transmission map are also estimated in order to restore an underwater image. For atmospheric light estimation, a combination of Res-net and Dense-net architecture is modelled and making it a learning based Residual-Dense Network. Through literature it is evident that, the transmission map describes about the portion of the light that is not scattered and reaches the camera. Transmission map is generated using the underwater image formation model. Further, these attenuation coefficients are given as an input to the proposed GAN architecture for underwater image restoration. The GAN architecture is having a generator block and a discriminator block. The result is demonstrated on synthetic and real underwater data-set and then compared with state-of-the-art method.


Introduction
Underwater image restoration is one of the demanded areas of research due to the challenges involved. Underwater images are more prone to various distortions like poor contrast, colour deviation or refractions of light. The main reasons for the degrading of an underwater image are light scattering and absorption of light. Scattering of visible light gradually reduces the sharpness of the image whereas the varying degrees of light attenuation travelling in water results in the change of colour. There are various techniques and methodologies are being used to process and restore an underwater image. As the visible light energy propagates deeper and deeper, the colour drops gradually depending upon the different wavelengths. And as the visible light coming from the image travels towards the camera, a part of this is visible light energy encounters the suspended particles in water. Therefore, results in scattering of light energy by these particles. Simultaneously, some kind of uneven illumination is caused, leading to spots that are brighter at the centre of the underwater image.
According to the Jaffe-McGlamery model of linear superposition, an underwater image can be represented by few parameters namely forward scattering, direct scattering as well as backward scattering parameters. The Jaffe-McGlamery model will cover a multiple range of imaging conditions and also which possesses few of the complex numerical methodologies, hence cannot be utilized for a single image restoration design. The underwater image is split into two segments IOP Publishing doi:10.1088/1757-899X/1187/1/012014 2 namely the light transmitted which is undeviating and the transmission of light by the hazy aqua medium and also the minute particles, which is namely the veiling light. The mathematical equation is expressed as, The RGB model is defined using the above variables, where I(a,b) represent the image captured through the camera, J(a,b) represent the haze-free image, t(a,b) will represent the transmission of the medium across the beam which will describe the part of light which is never scattered back and will reach the camera and A will represent global atmospheric light. The idiom J(a,b)t(a,b) will be a direct attenuation and the idiom A*(1-t(a,b)) represents water light luminescence. The t(a,b) which is the transmission of medium is a rapidly decaying activity as, where, β represent the attenuation coefficient and d(x) represents the distance from the scene to the camera, also called the scene depth.
In recent years, the deep learning frameworks for underwater image restoration have developed rapidly, especially the CNN (convolutional neural network) , mainly used in classification of image, object detection, and motion detection, and they perform much better compared to the traditional methods. But in the case of underwater image restoration using CNN is limited due to the lack of real-world underwater scenes. Therefore, using a synthetic underwater dataset would be an effective approach. Some generative adversarial networks (GAN) are used for generating real underwater images. In this paper, a GAN framework is put forward for Underwater image restoration using the attenuation coefficients like the atmospheric light A of the equation and the transmission map t(x) of the equation as an input. Estimation of atmospheric light is carried out by modeling a combination of Res-net and Dense-net architecture and proposing a Residual-Dense Network model. The transmission map is known as the portion of the light that is not scattered and reaches the camera. A transmission map is generated using the image formation model. The two attenuation coefficients that are, the atmospheric light and transmission map are used as an input for the proposed GAN underwater restoration model. The GAN architecture mainly has two blocks, a generator block; U-Net is used for the generator network, and a discriminator block; the proposed GAN having a discriminator block is executed as a Patch-GAN. The proposed technique successfully dehazes an underwater image when tested on quantitative and qualitative synthetic data and real data also surpass various state-of-the-art techniques of dehazing.

Related work
The attenuation coefficients for the ocean show many temporal variations as it depends on many organic concentrations and substances in water. The contrast will have much significance on underwater-computer-vision, here the details of the attenuation coefficients is used for the color restoration algorithm. Derya Akkaynak [1] has set a goal in the paper to grasp the base knowledge of visual hydrography in order to enhance the underwater-computer-vision techniques. Initially, the loci are derived that constitute an appropriate search space in underwater image restoration techniques, along with the reference for realistic computer-generated images. Through the wideband coefficients, it is shown that it relies on both the properties of scene and the water, a possession which is overlooked on the recently utilized for the image formation. Then verify the preferred loci model using in-situ analyses using 2 non-identical optical water bodies. The wideband coefficient becomes sensitive to the real color and the range of the object. This relatively invalidates the image formation model and hence the error is analyzed from this discrepancy. Phillip Isola [2] investigates that the image-to-image translation problem can be 3 addressed using a conditional adversarial network as a general-purpose solution. Here the network not only learns how to map from input image to output image but also learns a corresponding loss function to train the mapping. In the manner, GAN will attain generative model and a conditional GAN will attain a conditional generative model of data, hence making the conditional GAN fit the image to image interpretation. The author demonstrates that this approach is efficient at synthesizing images from label maps, image colorizing, and reconstructing objects.
The key global atmospheric light is ignored as the prsesent learning techniques which will anticipate only transmission medium using the CNN(convolutional-neural-network). Chongyi Li [3] proposes an adaptive cascaded CNN for single image dehazing, by considering the attenuation coefficients, that is atmospheric light globally and transmission medium together by 2 task based sub-networks. Moreover, a common feature is shared between the two cascaded subnetworks. Ultimately, using the parameters which will be estimated by the model, a restored image is attained. More accurate and effective restoration performance is achieved by the atmospheric scattering model.

Methodology
In this section, an architecture is been proposed for underwater image restoration. The framework mainly consists of 2 parts,namely Discriminator and Generator. A Generator network will learn a function to map from an input image to an output image. A generative adversarial network will be an appeal to train generator, which will be used to generate images using generator. The Discriminator network classifies if the image generated is Fake or Real. We have used a synthetic data-set which is been prepared using the NYU dataset. We have used this synthetic data-set for training the architectures. generative adversarial networks

Atmospheric light estimation
A learning-based approach for the estimation of atmospheric light is proposed. Initially, the input image is passed through the convolutional layer and further passed through the dense blocks to estimate the atmospheric light like AR, AG, AB for the respective red, green and blue channels. The main block of our dense residual network is dense blocks. Here each and every layer of the network is connected to each and every other layer and we have added residual connections in-between the dense blocks. Apart from adding more layers of CNN, adding residual networks will increase the performance and also accuracy. The residual block is adding output from the previous layer is added to the output of some forward layer. ResNet uses an additive method (+) that merges the previous layer (identity) with the future layer. Here we have added 2 dense blocks to our network. In CNN layers due to more layers, there may be chances of losing the information before reaching its destination. In DenseNet, each and every previous layer is connected to the next layers to avoid losing the information. DenseNet concatenates the previous layers with the future layers.

Transmission map estimation
We have estimated transmission map by using 2 methods. Firstly, by using underwater image formation model. • β is the attenuation coefficient.
• d(a, b) is the distance from camera to the scene.
Here beta value varies from [0.4,1.6]. Secondly by using CNN layer. The architecture of the CNN layer is shown below.
We have proposed our model on estimation of transmission map as shown above. Here the input image is passed through several layer layer of CNN to train on the transmission map data set which is estimated using the equation as stated above and we have tested on the testing data set which is added in the result section.

Generator :
For generator architecture, we have used U-Net architecture for the model to learn and generate images. We have given a hazy image, transmission maps, and atmospheric light as input for the generator network. A Generator network uses the U-Net architecture which is an encoder and decoder model. The network generates the restored image by down-sampling or the input image is encoded into the bottle-neck layer and then up-sampling or the output image is decoded from the bottle-neck layer. The skip connections are introduced in the middle of encoder and the decoder layers, which gives a 'U' shape to the architecture.   Discriminator : The discriminator model takes a generated input and a real image from the data-set and along with transmission map and atmospheric light as a hint and predicts whether the image is a real or generated image. The Patch-GAN will be implemented in this proposed discriminator architecture. Here the discriminator architecture will generally try to categorise whether each of the MxM patch is fake or real for a given image. In this model a Patch-GAN of size 16×16 is applied, thus says that the output image will be mapped with the input image of 16x16 size. As a result, the 16x16 Patch-GAN will categorise 16x16 patches as fake or real in an input image.  In Fig.6, we have estimated the atmospheric light on the testing images from the synthetic data-set. We infer from the Fig.9, that the atmospheric light estimated from the proposed Residual-Dense network works well. We can conclude from the test images that the prediction and target values are almost near. In the Fig.7, we have estimated the atmospheric light values on the Jerlov water type 7C and 9C of UIEBD data-set. In Fig.8, we have estimated atmospheric light for the Jerlov water type -1C of UIEBD data-set.
In Fig.9, We have tested our model on Jerlov water type -1B. We can see that the predicted values are different from the actual values. Thereby, we can conclude that for different Jerlov water types we need to train different model for the estimation of atmospheric light.

Conclusion
In this project, the GAN framework is proposed which uses the attenuation coefficients for restoring an underwater image. The attenuation coefficients are estimated individually for the same. Here, a learning based estimation of atmospheric light using residual-dense Further demonstrated the results on the benchmark data set and compared with state-of-theart methods. The proposed GAN restoration framework performs well on the degraded images and the results obtained are comparatively better.