Enhancing underwater optical imaging by using a low-pass polarization filter

: Object identification in highly turbid optical media depends mainly on the quality of collected images. Underwater images acquired in a turbid environment are generally of very poor quality. Attenuation and backscattering of light by water, by materials dissolved in the water, and by particulate material are the main causes of the degradation of underwater images. It is therefore essential to improve the quality of such images to facilitate object identification. The focus of this paper is to report the principle and validation of a fast and effective method of improving the quality of underwater images. On the one hand, this method uses a polarimetric imaging optical system to reduce the effect of diffusion on the image acquisition. On the other hand, it is based on an optimized version of the dark channel prior (DCP) method that has received a great deal of attention for image dehazing. Results derived from images obtained in a controlled laboratory water tank environment with different turbidity conditions and images from tests using the proposed method at sea demonstrate an ability to significantly improve visibility and reduce runtime by a factor of about 50 for a 4K image when compared to conventional DCP methods.


Background
Most underwater (scientific, industrial or military) vehicles are equipped with at least one embedded camera to observe and analyze the seabed. Typically, these remote operated vehicles (ROV) are often operated by crew operators aboard a research vessel. The ROV has an umbilical cable that contains electrical cables and optical fibers to carry electrical power, data and video signals [1]. Tethering systems to ROVs impose speed and depth restrictions that limit maneuverability. In addition, human intervention implies that the treatment is often long, tedious and dangerous. Such issues can be reduced by the use of autonomous underwater vehicles (AUV). The AUV is an untethered underwater robot that carries out a mission fully autonomously even without communication with crew operators. However, the treatment and the automatic analysis of acquired images are limited by the very bad quality of underwater images which decreases the performance of the detection and object recognition algorithms. Absorption and scattering of light by water, by materials dissolved in the water, and by particulate material are the main causes of the degradation of these images. Morel [2], give excellent reviews of the associated physics and mathematics. Both phenomena are highly wavelength dependent and depend also on the size of the particles within the water which also governs the type of scattering that occurs, namely Mie or Rayleigh scattering [3]. Blue light penetrates deeper than red light (Fig. 1). One of the major difficulties inherent in the treatment of the collected underwater images comes from the exponential attenuation of light which limits visibility to around 20 m in clear water and to less than 5m in turbid water [4]. A useful background underwater image model of formation has been proposed by Jaffe [5]. This assumes that the underwater image I received by the camera is the sum of three components, i.e. direct attenuation A I , forward scattering F I and backscattering B I such as: where x denotes image coordinates. According to Schechner and Karpel [6], backscattering is the main reason for deterioration of the underwater visibility, thus forward scattering can be neglected. This assumption is valid when the distance between the scene and the camera is large. Equation (1) can be written as: forward scattering causes a blur in the image while backscattering produces a light veil which reduces the contrast and visibility of the observed scene [4]. Additionally, using artificial lighting to improve visibility introduces shadows and illuminates the scene in a non-uniform way. Both phenomena make it difficult, even impossible, to detect and/or identify objects in strongly turbid optical media.

State-of-the-art
Many techniques of restoration and improvement of the quality of the underwater images have been proposed in the archival literature and this continues to be a timely subject of research [8]. Restoration techniques are designed to assess the degradation of the image during its acquisition and then to apply an inverse transform of the model to recover the original image. Deconvolution of the water transfer function is in principle possible for restoration of underwater images [9]. Hou and associates [10] incorporated optical properties of water in the transfer function of an underwater camera system. Trucco and Olmos-Antillon [11] presented a self-adjusting restoration filter based on a simplified model of underwater image formation. Ouyang and associates [12] used pulsed laser light and a bilateral filtering method based on deconvolution for underwater image enhancement. Schechner and Treibitz [13] suggested an effective algorithm that requires two polarized images obtained in two orthogonal directions to restore the original scene. Although these approaches were being introduced for improving the physical contrast of simple images, they require many unknown parameters (such as the attenuation coefficients of the medium and depth of the scene) which can be difficult of estimating with precision. In addition, these methods use relatively expensive and complex optical components such as e.g. a range-gated laser. Furthermore, solutions that require many image entries are usually difficult to apply because the acquisition of multiple images of the same scene is difficult in underwater conditions. In recent years, He and associates [7] suggested the DCP that allows estimating the depth of a scene from a single image and restoring the image without fog. Several studies used DCP to remove the luminous veil and adapted it to underwater conditions [14,15]. In contrast to restoration analysis, methods to improve quality do not rely on physical models and do not require an a priori knowledge of the environment. These methods use qualitative subjective criteria to improve the visual quality of the image, to reduce the noise and to correct color. To overcome the issue of non-uniform illumination in underwater images, Garcia and associates [16] proposed applying a homomorphic filtering. Hitam and associates [17] applied adaptive histogram equalization for underwater image enhancement. Ghani and Isa [18] suggested dividing the histogram of the image in the middle and stretching the two histograms obtained over the entire dynamic range following the Rayleigh distribution. Ancuti and associates [19] adopted another strategy by merging images derived from the original degraded image. Bazeille and associates [20] suggested an automatic algorithm consisting of several independent steps of treatment allowing correcting non-uniform lighting, eliminating noise, improving contrast and equalizing colors. Despite the simplicity and the multitude of digital methods designed for improving quality, their efficiency decreases as turbidity is increased. The combination of several numerical methods increases the runtime which is not desirable for any real-time application. For this specific reason, two approaches will be combined in this study. On the one hand, our method uses a polarimetric imaging optical system to reduce the effect of diffusion on the image acquisition. On the other hand, the method is based on an optimized version of the dark channel prior (DCP) method that has received a great deal of attention for image dehazing.

Motivation and outline
A question that often appears throughout the literature asks is that of the extent to which quality of underwater images can be improved while reducing algorithm runtime in order to facilitate real-time object identification?. This challenge is of importance when targeting underwater objects. In this paper, computational and experimental investigations show the effect of turbidity on visibility and algorithm runtime of collected images. The successful performance of the method developed in the paper is illustrated by considering images obtained in underwater field experiments either in a controlled laboratory water tank environment with different turbidity conditions and or from tests at sea.

Imaging polarimetry
In this section, we offer a brief background as well as a set of definition of terms that are used in the paper. Then, preliminary tests document the benefit of incorporating polarization information into the algorithm.

State of the art
Polarimetry has been widely as a useful technique that removes degradation effects in underwater vision [6,[21][22][23][24]. The propagation of polarized light in water has led to the development of active and passive polarimetric imaging systems. Passive imaging uses the ambient light from the sun that is unpolarized. The light that does not reach the object becomes partially polarized by scattering and can be filtered by an analyzer placed in front of the detector [6]. However, natural light illumination is inapplicable at great depth where active scene irradiance is required. Active imaging requires artificial light illumination and appropriate detection scheme. Several studies have proposed to exploit the differences in polarimetric responses between target and backscattering [25,26]. A comprehensive study of polarimetric backscatterd light variations was considered in [25]. The backscattered Mueller matrix for various concentrations of spherical scatterers with and without painted metal target was measured. However, such measurements are time consuming because they often require considerable data acquisition. More recent studies proved that the strong backscatter created by artificial lighting can be reduced by performing a single acquisition with adapted input and output polarization states. The evolution of circularly and linearly polarized states of light as they scatters throughout an optical medium composed of particles in suspension in water (e.g. randomly scattering media of a polystyrene-microsphere solution), numerical simulation, analytical theory and experimental data have shown that circularly polarized light exhibits superior persistence (compared to linearly polarized light) for forward-scattering particulate medium [27][28][29][30][31][32][33][34]. Assuming that the target is totally depolarizing and handedness of the circularly polarized state change after reflection by the scattering particles, Gilbert and associates [27] showed that using circularly polarized light permits reduction of backscatter and increase by a factor of two of the visibility range compared to intensity measurements. Mullen and associates [34] provided a technique using polarization discrimination for analyzing the contribution of backscattered light, arising from multiple scattering of the interrogating laser beam from water particulate matter in both the forward and backward directions, and improving the contrast of the retro-reflected signal. These authors evidence that both the light forward scattered on its path to and from an underwater object and the light that is scattered back to the detector without reaching the object have a high degree of polarization when linearly polarized light is transmitted.

Characterization of underwater object
The detection of an underwater target by use of polarization discrimination depends on the different polarimetric responses between the target and scattering medium which separates the object from the camera. Several studies have focused on characterizing the transport of polarized light through different scattering media by means of the Stokes-Mueller formalism [25]. This enables to completely describe the effect of the scattering medium or the target on an incident polarized light via Mueller matrices. A Stokes vector S is a four-component vector defined as: Where I represents the total intensity of the detected light, and , , Q U V defined by x y x y x y where 0 x E and 0 y E denote the amplitudes of the parallel and perpendicular to the scattering plane electric field components and φ is the relative phase difference. The polarization state of the scattered light S′ can be written as: 11 where the 16 ij M elements define the Mueller matrix. In pure water containing Rayleigh spherical scatters, single scattering events dominate over multiple scattering and absorption can be neglected. Within these assumptions, the normalized Mueller matrix is [26,33] ( ) For backscattering conditions ( 180 linear polarization is preserved and circular helicity is reversed. This backscattering conservation property is widely used for underwater target detection by controlling the polarization state of the light source and the camera through suitable combination of polarizers. Indeed, imaging with cross-polarization state using linearly polarized light and co-polarized state using circularly polarized light better discriminates the useful signal from backscattering. However, in practice this method can be effective only if the target is sufficiently depolarizing. Other studies have demonstrated that performing measurements with two orthogonal states permits decreasing the backscattering effect. Treibitz and Schechner [6] presented an approach for recovering the object signal using polychromatic polarized light illumination. The camera is fitted with a polarization analyzer. Two frames of the scene are taken, with different states of the analyzer or polarizer. A recovery algorithm follows the acquisition. This method allows both the backscatter and the object reflection to be partially polarized. The algorithm assumes that the underwater image is made up of the target signal and the backscattered signal. We will write the measured image in the following typical form: where ( ) , S x y is the signal from target and ( , ) B x y corresponds to the backscatter. Two images min I and max I are acquired, corresponding respectively to the linear cross-polarization image (co-circular) and the linear-parallel image (cross-circular). The linear crosspolarization (co-circular) image min I is obtained when the analyzer is placed in the orthogonal (resp. same) state of the light source polarizer. The linear parallel polarization (cross-circular) image max I is obtained when the analyzer is placed in the same (resp. orthogonal) state of the light source polarizer. As a large fraction of the linear (circular) polarization is preserved (reversed) under wide-angle backscattering, this technique in the cross-state (co-circular) yields an image with less backscatter and target signal. Thus, the images are expressed as: The degree of polarization (DOP) of the target and backscatter are written, respectively, as: and The target and the backscatter signals can be expressed as: and Equations (18) and (19) can be used to provide an estimate of the target S  and backscattering B  as respectively: The target estimate can thus be inferred from measures of min I and max I , for a given value of scat P , since arg t P does contribute only to a scale factor for the reconstruction of the signal. The target estimate can be done on the basis of Eq. (18) assuming that targ P 0 = . It should be noted that the value of scat P is crucial and must be assessed with precision. We note that this method has been used in [22], and it gives good results. However, it is difficult to use it for the current application at least for two reasons. Firstly, the acquisition of two images of the same scene is difficult without human intervention because of the rapid changes of the 3D scene structure and light illumination conditions. Second, the estimate of scat P requires a precise selection of an image area where the target does not appear, which is difficult to implement automatically. This is the reason why this study considers a single polarization measure that minimizes the backscattering effect in the image.

Experimental setup
Since the nature of the object to be identified is unknown, it remains difficult to estimate its depolarization properties. Consequently, no optimized state of polarization can be guessed. In this work, we try to use polarization states allowing us to get images with the best quality. Two polarization options are considered, either a linearly co-polarized or a cross-polarized scheme. The experimental set-up used is depicted in Fig. 2. Thus, the Stokes vector 5 S that characterizes the polarization state of light received by the CCD camera can be written as: And (iii) the Mueller matrix of water (source-target 1 w M / target-camera 2 w M ) are also unknown because we cannot specify the effect of turbidity of the seabed during image acquisition. However, based on the fact that backscattering conserves linear polarization and reverse circular helicity, we assume that: Knowing that the Mueller matrix of a linear polarizer is: Where ϕ is the polarization angle, the parallel polarization image \\ Similarly, the orthogonal polarization image I ⊥ is obtained when the polarization angles for

Tests in small water tank
In this part we present our first tests that were carried out in a controlled laboratory water tank (15 L of natural water) environment with different turbidity conditions by adding milk [35].
Milk is a complex biological fluid composed of water, fat, protein, lactose, organic acids, and inorganic compounds. These different kinds of particles in milk change its scattering properties based on their differences in size, concentration and optical properties. Skim milk contains casein micelles with size polydispersity in the range between 0.04 and 0.3 mm (Rayleigh scattering regime). Skim milk appears slightly blue because the small casein micelles predominantly scatter the shorter wavelengths of visible light. In addition, low-fat milk contains fat droplets with size in the range 1-20 mm, corresponding to Mie scattering regime. Thus, for a concentration c (in %) of milk in a certain volume of water, an estimate for the diffusion coefficient is respectively  Figure 3, shows the experimental set-up used to acquire tests images. As can be seen in Fig. 3(b), the target (capital letter "A") is placed at the bottom of the water tank at a distance of 34 cm of a digital camera Sealife DC1400 video pro Set digital camera [37]: high resolution 14 megapixel CCD sensor, 26 mm lens wide angle, 20 MB memory internal, underwater vision field of 51  and can operate up to 60 m depth. The target is illuminated by a light source made by three white LEDs on which a linear polarizer is set. A linear analyzer is placed in front of the camera Fig. 3(a). The image obtained for the linear parallel polarization state ( \\ I ) (resp. ( I ⊥ )) is obtained when the analyzer is placed in the same (resp. orthogonal) state of polarization as the light source. . We can see from these images that the target is not visible in images taken without the use of the analyzer (Fig. 4(a) and Fig. 4(d)) and also in the image in the linear parallel analyzed polarization (Fig. 4(b) and 4(e)). Visibility is significantly improved when the linear cross-polarization arrangement is used (compare Fig. 4(c) and 4(f)) in accordance with the fact that polarization of the backscatter is mainly linear and in the same orientation of the incident light.

Tests in larger water tank
To validate the interest of using crossed linear polarization to eliminate backscatter effects, we extend our experiments to more realistic conditions. Tests were performed in a large water tank ( Fig. 5(a)) of dimension [ 3 2 m m × ] filled with natural water (5000L) to which we add skim-milk to mimic the turbidity conditions. Figure 5(b) shows the waterproof imaging system designed by FORSSEA Robotics Company [38]. This system contains a polarized light source and two cameras. The first is equipped with a polarizer fixed in the orthogonal state with respect to the state of the source. No polarizer is fixed on the second camera, this allows us to acquire two images of the scene at the same time (without and with polarization) in order to compare them. The linear crossed polarimetric imaging system was precisely calibrated in our laboratory (Fig. 5(c)). On an optical bench, two polarizers were placed oppositely to each other and separated by a 30 cm distance. A linear polarization state is generated from the first polarizer illuminated by an He-Ne Laser source emitting at 632.8nm. The second polarizer (analyzer) can be rotated. Several markers were placed at the bottom of a large water tank at different distances, as shown in Fig. 5(d).  As can be seen from results, images taken using cross linear polarization have fewer reflections compared to raw images taken without polarized light. Consequently, letters on markers are clearly observed and easy to be distinguished in the cross linear polarization images. Figure 7 shows a comparison of the marker (framed in red) in Fig. 6(c) and Fig. 6(j). Capital letter P is very blurred in the unpolarized image ( Fig. 7(a)). A significant improvement in quality is observed when cross linear polarization is used (Fig. 7(b)). This clearly demonstrates the benefit of using polarization for improving the visibility and detection of markers. However, the visibility of the markers (especially the most distant ones) decreases when the concentration of milk in water is increased (Fig. 6(d)). In this case, polarization fails to totally remove the backscattering effect ( Fig. 6(h)). Moreover, in real conditions, water depth and turbidity which mainly determine the light penetration will strongly limit the estimation of parameters of interest from the acquired image in test at sea. In the next sections, we introduce an innovative method that uses a combination of optical imaging polarimetry and dark channel prior (DCP) methods which enhance contrast in images of objects in turbid environments.

Optimized DCP-based method and experimental analysis
The dark channel prior (DCP) method has been heavily utilized in applications involving haze removal (dehazing) because it can significantly improve the quality of images of outdoor scenes. Basically, DCP relies on the fact that the intensity value of at least one color channel within a local window is close to zero. Based on the DCP, dehazing is accomplished through four steps: atmospheric light estimation, transmission map estimation, transmission map refinement, and image reconstruction [39].

Brief survey of DCP method
DCP is based on the following observation on outdoor haze-free images that in most of the non-sky patches at least one color (R, G, B) channel contains some pixels whose intensity are very low . To render observable this phenomenology, He and associates [7] introduced the concept of a dark channel for an arbitrary image J as: ( ) , , where c J denotes a color channel of image and ( )

Validation tests for underwater grayscale images
To explore the performance of our algorithm tests are conducted on unpolarized images. Figure 9(a) shows the target image in clear water. Figures 9 (b) and 9(c) show the corresponding images for two turbidity cases, i.e. As is expected the intensity of the veil increases with water turbidity. In Fig. 10, we present a qualitative comparison for the improvement of quality of the images by making use of three methods: DCP, DCP with refinement of the transmission map by soft matting [7], and our approach. When viewing the results shown in Fig. 10, we observe that the DCP and our approach significantly reduced the veil and improved the target visibility. To quantitatively evaluate this improvement for the images shown in Fig. 10 where an image is divided into 1 2 k k blocks, and , , ⊗ Θ ⊗ are the PLIP operations [41]. As can be seen in Table 1, a better contrast is obtained with our approach.  Our overall estimate of the transmission map allows us to avoid the effects of windowing as is evidenced for the conventional DCP method (visible in Fig. 10(b)) which requires a map refinement by another algorithm such as soft matting (Fig. 10(c)). To illustrate further this point, Fig. 11 shows a comparison of the transmission maps obtained by the conventional DCP method (Fig. 11(b)) and our approach (Fig. 11(c)). In addition to gain in quality, an added benefit is that it is less computationally expensive than the DCP model, which allows for fast detection and recognition of objects. Our approach based on global filtering is much faster than the local calculation of dark channel. The runtime results in Table 2 show that our algorithm is much faster than the DCP algorithm. Hence, the proposed method significantly reduces runtime by a factor of about 50 for a 4K image compared to DCP. The low computational burden is important for optical-imaging applications where we desire real-time, high-quality color images. For this purpose, we now extend our approach to deal with color image processing. This method has been tested in experiment to determine sensitivity to turbidity. The results are shown to compare favorably with DCP data.

Validation tests for underwater color images
In this section, we present the details for how we execute the proposed algorithm shown in Fig. 12. Building on the algorithm described in Fig. 8 Figure 13 shows the results of validation tests of our approach for color images. To compare our approach to the conventional DCP method and its extended version for color images, we consider data similar to those presented in Fig. 9. When viewing the example of Fig. 13, our method allows us to get a better contrast better than that obtained by the DCP and DCP with soft matting.

Validation tests for underwater grayscale polarized images
The examples in Fig. 10 and 13 show that our algorithm is able to significantly improve visibility of the images acquired in turbid optical media. In Fig. 14 we compare images of the target in turbid water acquired without polarization, with linear cross-polarization and linear cross-polarization using our processing method, respectively. It can be seen from Fig. 14(b) and Fig. 14(e) that the veil is not completely filtered out and the contours of the target are not well defined by using linear cross-polarization. However, numerical improvement of these images (see Fig. 14(c) and Fig. 14(f)) by our method allows removing the remaining backscattering contribution and permits to significantly enhance the target contours.

Preliminary results from tests at sea
We show consider experimentally determined images obtained by tests at sea (bay area of Brest, France) in collaboration with FORSSEA Robotics which designed an autonomous docking underwater technology for AUVs based on our proposed method. This system was the subject of a patent (N/Ref: BG/EBU/DD-FR 1763336: DISPOSITIF D'AMELIORATION D'IMAGE SOUS-MARINE). This imaging system contains a polarized light source and two cameras. One is equipped with a polarizer oriented perpendicularly to the source polarization in order to acquire the linear cross-polarization image. No polarization filter is attached to the other camera. The system is connected to our boat in real time (Fig. 15). The tests were performed in the port of Brest at night (http://www.google.fr/place/Brest). Preliminary results at 8 meters depth are shown in Fig. 16. Two targets are used to acquire test images. A swimmer adjusts the target-camera distance and stir the sand of the seabed to increase turbidity. 1379 [728 × 1288] image size are acquired during this experiment. As can be seen from Fig. 16, a significant gain in quality is observed when the linear crosspolarization arrangement is used ( Fig. 16(b)). The acquired images are much less affected by the veil of backscattering compared to those obtained without polarization ( Fig. 16(a)). The additional processing of polarized images to fully remove the remaining backscattering contribution has the effect of significantly enhancing the contours of the target images ( Fig.  16(c)).

Concluding remarks and directions for further work
The novelty of this paper is to propose and study a relatively simple and computationally inexpensive yet effective algorithm for improving the quality of images in automating detection and recognition of objects under the surface of the water. The paper describes how to use a system of linear cross-polarization to filter out most of backscattering contribution during image acquisition and combining it to our numerical dehazing approach to these images. The latter assumes that backscattering is characterized by a low-frequency component which affects the image: this property is used as a prior (or knowledge) to estimate the transmission of the scene and restore the image from a simplified model of underwater image formation. The proposed method was tested in simulation and experiment over a range of conditions. It performs better than previously published dehazing algorithms in optical imaging applications. Based on its simplicity, consistency, and improvement in quality and algorithm runtime, this method is recommended for use in future real-time automating image processing technique designed for detection and recognition of objects under the surface of the water. Future investigations of the effects of turbidity on the predictive method proposed here will explore approaches to circumvent current limitations in underwater imaging applications due to the blurring of the collected images. A complete description of the robustness of this approach for imaging in optically scattering media is under investigation. In this work, we focus on detecting relatively simple objects. Any future work will include more testing and validation with more complicated objects, e.g. underwater mines. Another continuing challenge in underwater optical imaging is the interpretation of the physical meaning of the Mueller matrix of an acquired image of a depolarizing object present in a turbid medium.