Single image dehazing with a physical model and dark channel prior
Introduction
The intensification of environmental pollution has seen levels of SO2, NOX, and particulate matter increase sharply in recent years. The first two are gaseous pollutants, but particulate matter is the most important factor in the appearance of haze. When combined with fog, the sky appears gray. The quality of a photograph taken in hazy weather is severely reduced because a large number of small particles refract and reflect the light before it reaches the camera lens. Therefore, the contrast of the photo is reduced and color images lose a lot of detail, especially in terms of the depth of objects. Thus, the information contained in these pictures is greatly reduced, as shown in Fig. 1(a) and (c). In practical applications such as military technology, traffic, forensics, meteorology, and astronomy, it is often necessary to extract image features from a collection of outdoor video sequences. Thus, the dehazing of images has become an urgent and practical research topic. The dehazed image is more visually pleasing, contains more information, and can be widely used in many fields. For example, dehazed images are an effective source of data in computer vision, as shown in Fig. 1(b) and (d). In image processing, image dehazing is used as a pre-processing method. For instance, Gibson et al. [7] investigated the dehazing effects on image and video coding for surveillance systems. Wang et al. [24], [25], [26], [27], [28] improved the accuracy of image relevance ranking and image or video annotation using dehazed images. Further, Gao et al. [4], [5], [6] used dehazed images for 3D object retrieval and recognition. Restoring the true image from the fog-impaired version has important academic and practical significance.
Early approaches to image dehazing used a physical model for haze removal. This mainly relies on additional depth information or multiple observations of the same scene. Kopf et al. [9] proposed a method to recover hazy images using a 3D model or scene depth information, which is directly accessible from digital terrain maps or Google Earth. This method has limited practical applications. Schechner et al. [18], [20] noted that the light scattered by atmospheric particles is partially polarized. Using this, they proposed a quick method for removing haze by taking two images through a polarizer at different angles. However, this method does not conform to the real physical model. Narasimhan et al. [13], [14], [15], [16], [17] proposed a physics-based scattering model based on the binary scattering deduced from the RGB color space. The haze-free scene structure can be recovered from two or more weather images by determining the 3D structure of the hazy scene. However, they assume that the atmospheric scattering coefficient does not change with the wavelength of light. This assumption is only an approximation in foggy weather conditions. Therefore, we cannot produce good results by processing hazy regions in a similar way as areas of sky in the binary scattering model. By considering fog and haze as a kind of noise, hazy images can also be processed using the methods in [1], [11]. However, the formation of fog and haze are varied, so the visual appearance of the dehazed images are not very accurate.
Significant progress has been made on single image dehazing in recent years. However, the increasingly detrimental effect of haze, smoke, and fog means that less scene structure information can be used, so single image dehazing has become more challenging. Fattal et al. [2] proposed a mathematical model for image dehazing. This model describes the surface shading of the objects and the scene albedo. By assuming that the two functions are locally statistically uncorrelated, hazy images can be divided into regions of constant albedo. These can be used to infer the actual scene. The algorithm is based on local statistics and requires sufficient color information. A larger haze concentration means that the scene loses more energy. This gives the scene a gray appearance, and a relatively small local variance. So this method cannot effectively estimate the transmission coefficient. Therefore, only the local transmission coefficient can be estimated. When objects are far from the camera, there will be a certain degree of mist. Tan et al. [23] enhanced hazy images by maximizing their local contrast. Although this method was successful in regions with very dense haziness, the color of the haze-free image is often oversaturated. This phenomenon is caused by the haze concentration being overestimated in the process of contrast maximization, a result of using image enhancement technology instead of a physics-based method. Kratz et al. [10] proposed describing an image as a factorial Markov random field, in which the scene albedo and depth are two statistically independent latent layers. A canonical expectation maximization algorithm is implemented to factorize the image, recovering haze-free images with fine edge details, but sometimes the output images are over-enhanced. He et al. [8] proposed a dark channel prior algorithm. The dark channel prior is the result of an observation of outdoor haze-free images, in which most of the non-sky patches have at least one color channel containing low intensity pixels. Statistical results show that most of the outdoor pictures satisfy this requirement. Combined with soft matting, the dark channel prior method achieves outstanding results with hazy images and obtains the corresponding depth image. This is currently one of the most effective dehazing methods. The dark channel prior may be invalid when the scene object is inherently close to the airlight. Further, this method cannot process gray areas of the image very well and is somewhat time-consuming. Sun et al. [21], [12], [19], [29] also obtained good results with algorithms based on the dark channel prior.
This article proposes a kind of atmospheric scattering model and dark channel prior principle based on the work of He [8] and others. First, we improve the transmission map, which reduces the processing time. Second, as the atmospheric light plays a very important role in the dehazing effect, we derive a new method to select this value. This method overcomes the deficiency of the dark channel prior, and reduces the influence of white objects or sky areas on the whole image. Finally, we obtain a clear image without fog.
Section snippets
Physical model
Fattal et al. [2] studied the physical model of atmospheric scattering and optical reflectance imaging. In computer vision and computer graphics, the model widely used to describe the formation of a hazy image iswhere is the observed image, is the scene radiance, is the global atmospheric light, and is the scene transmission (which describes the portion of light that is not scattered and reaches the camera). The aim of haze removal is to
Estimating the scene transmission
He et al. [8] used the atmospheric scattering model of Eq. (1). The atmospheric light is a given constant value. We normalize this haze imaging equation using , i.e.,
The normalization operation is independently applied to the three color channels.
We further assume that the transmission in a local patch is constant. We denote this transmission as . We then apply the two minimum operators to both sides of Eq. (7):
Experimental results and analysis
We implemented the proposed algorithm on a Windows 7 PC with an Intel (R) Core (TM) [email protected] GHz processor, running Microsoft Visual Studio 10.0 and OpenCV 2.4.8. We used 300+ images downloaded from the Internet public image library, as well as our own pictures. We categorized the experimental hazy images into remote sensing images, building images with areas of sky, and other natural scene images.
In the first experiment, we selected hazy photos without sky areas for processing. The
Quantitative evaluation
This section contains the cross validation results that compared the existing and proposed methods. To objectively determine the quality of the degraded images, we can evaluate the image quality using three indicators: MSE (mean squared error), PSNR (peak signal to noise ratio), and average gradient (AveGrad).
The MSE and PSNR are defined as follows:andwhere is the original image, is the dahazed image, and
Conclusions
The selection of atmospheric light values has a very important effect on dehazing, because they are directly related to the image without fog. We propose a method for estimating the atmospheric light value using the variogram concept. Our algorithm uses dark channel prior knowledge to perform the dehazing. We also propose a fast estimation algorithm for the transmittance, and a method for selecting rational atmospheric light values. This improves the operational efficiency.
In addition to our
Acknowledgments
This work was supported by the Natural Science Foundation of China (Grant nos. 61370138, 61271435, 61103130, U1301251); National Program on Key Basic Research Projects (973 programs) (Grant nos. 2010CB731804-1, 2011CB706901-4); Project of Construction of Innovative Teams and Teacher Career Development for Universities and Colleges under Beijing Municipality (Nos. IDHT20130513, CIT&TCD20130513), and Beijing Municipal Natural Science Foundation (No. 4141003).
Jin-Bao Wang received the B.S. degree from Electronic Information Science and Technology, Physics Department, Hebei University, in 2013. He is currently working toward the M.S. degree in the Beijing Key Laboratory of Information Service Engineering, College of Information Technology, Beijing Union University. He is interested in digital image processing.
References (29)
- et al.
Smooth incomplete matrix factorization and its applications in image/video denoising
Neurocomputing
(2013) - et al.
Adaptive kernel regression and probabilistic self-organizing maps for JPEG image deblocking
Neurocomputing
(2013) - et al.
Sparse coding for image denoising using spike and slab prior
Neurocomputing
(2013) - et al.
Weighted haze removal method with halo prevention
J. Vis. Commun. Image Represent.
(2014) A new single-image fog removal algorithm based on physical model
Opt. – Int. J. Light Electron Opt.
(2013)- et al.
Fast single haze image enhancement
Comput. Electr. Eng.
(2014) Single image dehazing
ACM Trans. Graph.
(2008)- et al.
3-d object retrieval and recognition with hypergraph analysis
IEEE Trans. Image Process.
(2012) - et al.
Visual–textual joint relevance learning for tag-based social image search
IEEE Trans. Image Process.
(2013) - et al.
Less is more: efficient 3-D object retrieval with query view selection
IEEE Trans. Multimed.
(2011)
An investigation of dehazing effects on image and video coding
IEEE Trans. Image Process.
Single image haze removal using dark channel prior
IEEE Trans. Pattern Anal. Mach. Intell.
Deep photo: model-based photograph enhancement and viewing
ACM Trans. Graph.
Cited by (161)
Visibility enhancement and dehazing: Research contribution challenges and direction
2022, Computer Science ReviewDesign of a deep learning visual system for the thickness measurement of each coating layer of TRISO-coated fuel particles
2022, Measurement: Journal of the International Measurement ConfederationSingle image haze removal using contrast limited adaptive histogram equalization based multiscale fusion technique
2024, Multimedia Tools and ApplicationsComputational Imaging and Artificial Intelligence: The Next Revolution of Mobile Vision
2023, Proceedings of the IEEE
Jin-Bao Wang received the B.S. degree from Electronic Information Science and Technology, Physics Department, Hebei University, in 2013. He is currently working toward the M.S. degree in the Beijing Key Laboratory of Information Service Engineering, College of Information Technology, Beijing Union University. He is interested in digital image processing.
Ning He was born in Panjin, Liaoning Province on July 29, 1970. She graduated from Department of Mathematics at Ningxia University in July 1993. She received M.S. degree and Ph.D. degree in applied mathematics from Northwest University and Capital Normal University in July 2003 and July 2009, respectively. Currently she is a associate professor of Beijing Union University and master tutor. Her research interests include digital image processing, computer graphics.
Lulu Zhang, a master student in software engineering from Beijing Union University, Beijing Key Laboratory of Information Service Engineering. Presently her fields of interest are digital image processing and computer vision.
Ke Lu was born in Ningxia on March 13, 1971. He received his master degree and Ph.D. degree from the Department of Mathematics and Department of Computer Science at Northwest University in July 1998 and July 2003, respectively. He worked as a postdoctoral fellow in the Institute of Automation Chinese Academy of Sciences from July 2003 to April 2005. Currently he is a professor of the University of the Chinese Academy of Sciences. His current research areas focuses on computer vision, 3D image reconstruction and computer graphics.