A new method for automated driving image defogging based on improved dark channel prior

To tackle the problem of blurred image subjects due to haze in the images captured by the automatic driving system, which affects the safety of automated driving, a new, improved dark channel image defogging method based on adaptive domain dark channel calculation, fast bilateral filtering to optimize transmittance and automatic color equalization is proposed. First, map the original image based on the haze image degradation model and evaluate the atmospheric light intensity and transmittance based on the adaptive domain. Then, the atmospheric transmittance is optimized by combining the powerful value filtering with the fast bilateral filtering method. Finally, the image is further optimized by using the multi-channel automatic color gradation equalization method to solve the phenomenon of oversaturated color and dark brightness in the filtered image. The results show that the algorithm of this paper has high clarity and contrast, reduces the computation and running time, preserves the image edge information, has an excellent fog removal effect, and is highly adaptive for automatic driving image processing.


Introduction
Image detection is an essential component of autonomous driving, and the prerequisite is the acquisition of high-quality images. Despite the high resolution of cameras, the quality of images captured by cameras is severely degraded by foggy weather, which affects the accuracy of computer vision algorithms in detecting vehicles. Therefore, image defogging algorithms are of great practical value for research in autonomous driving.
Image defogging algorithms mainly contain two types: image enhancement-based defogging algorithms and image restoration-based defogging algorithms. Image enhancement algorithms are mainly based on removing image noise and improving image contrast to achieve the purpose of defogging. The image restoration algorithm is mainly based on atmospheric degradation model to reduce the disturbing information in the image, but there will be over-enhancement, resulting in image distortion. Deng [1] applied fuzzy logic control to statistically analyze the grayscale image characteristics and divided different images into mapping levels to develop corresponding defogging strategies for different images. Song [2] combined the grayscale characteristics of images with image enhancement methods to optimize the dark channel a priori defogging method. Yu [3] used the Otsu method to label different image blocks and differentiate the labeled blocks. Tan [4] found statistically that haze-free images have higher contrast than haze images, so they used local image contrast maximization to achieve haze removal. However, this method does not consider that light component often leads to excessive recovery of image color saturation and distortion. Fattal [5] estimated the albedo of a scene and medium transmission under the assumption that transmission is locally uncorrelated with surface shadows. This approach is physically sound and can produce impressive results. However, it does not handle severely blurred images well and may fail in the case of broken assumptions. He [6] first proposed a classical dark channel a priori image defogging algorithm with good results. The method is effective in defogging, but there are many color distortions at the edges of color blocks, and it is easy to lose some luminance information and image details.
Based on the previous research, this paper proposes a new method to improve dark channel image defogging based on adaptive domain dark channel calculation, fast bilateral filtering to optimize transmittance, and automatic color equalization. The fast bilateral filtering significantly preserves the image edge information and adjusts the contrast with the automatic color balance method to obtain a better-defogged image.

Haze image degradation model
In nature, the scattering phenomenon occurs due to the influence of aerosol systems composed of dust and other particles in the air, resulting in low color saturation and contrast in the image acquisition process. In order to reduce interference and fundamentally eliminate the influence of fog, an image degradation model is established as follows [7] .
Foggy day images collected at the observation site, ( ) J x is Defogging images, A is the atmospheric light intensity and ( ) t x is the transmittance associated with the depth of the image.
Understood from the perspective of mathematical modeling, the defogging recovery is the estimation of transmittance and atmospheric light intensity from the foggy sky image, and the defogging image is obtained after an algorithm with the following equation.
The lower limit 0 0.1 t = is taken to avoid the noise phenomenon caused by the transmittance of individual pixels tending to zero.

Improved algorithms
The algorithm flow in this paper is shown in Figure 1.  3 domain mean, the initial estimate of transmittance is obtained based on the dark channel a priori. It is refined using fast bilateral filtering. Finally, based on the statistical analysis of the color saturation of the preliminary image defogging, further adaptive color scale adjustment is performed to finally obtain the defogged image.

Dark primary color model for adaptive fields
In clear images that do not contain large bright areas such as the sky, where most local areas have at least one color channel with a very low gray value, the mathematical equation describes the dark primary color principle.
is the field centered on pixel x ; and dark ( ) J x is the dark primary color of the image in the field, which will tend to 0 if the image is clear and fog-free.
The domain window of ( ) x Ω in the dark channel algorithm is a fixed value15 15 pix pix × , and the size of the fixed window search is difficult to apply to scenic features at different resolutions, which will lead to local dark channel estimation failure for high-resolution images. In this paper, we propose an improved method for adaptive resolution, i.e., setting the domain window size according to four percent of the original image size. min( , ) 4% k m n = × (4) where m and n represent the width and height of the image, respectively.
When the image resolution is 600 400 ; when the image resolution is 1280 720 . More accurate atmospheric light intensity is obtained due to the chunking of the adaptive domain, which uses the average value of the area pixels instead of the individual maximum values.

Fast transmissivity optimization of bilateral filtering
The transmittance ( ) t x can be estimated from the atmospheric scattering model and the dark channel a priori model . He et al. show that the atmospheric light intensity is known in the domain of and the transmittance is constant. Transforming equation (1) and performing the minimal value operation on the left and right ends, we can substitute equation (3) to obtain The classical dark channel prior model performs better in processing images with slow gradient change.
However, the local transmittance is not a constant value in the region of abrupt gradient changes at the junction of the near and far views. If Equation (5) is directly substituted into Equation (2), a noticeable block effect and color distortion will appear in the defogging image. Therefore, He et al. used the soft matting algorithm to optimize it and later changed it into guided filtering for edge-preserving and smoothing the original transmission graph. Although the defogging image has been smoothed to a certain extent, when image smoothing and edge protection are applied simultaneously, the image edge may be inevitably smoothed due to fixed parameters, resulting in low smoothness, halo effect, and loss of image brightness.
To solve the problem of image smoothness and image brightness decrease after guided filtering. Based on the classical dark channel prior model, this paper proposes a method to judge the bright region and the unbright region of the image by using the idea of image region segmentation and then improve the calculation efficiency of transmittance through the effective combination of maximum filter and bilateral filter.

Judgment of bright area
When there is a bright area in the scene, the intensity of atmospheric light in the bright area will basically weaken, and the transmittance will also decrease, which will cause color distortion and a halo effect to some extent [8] . Therefore, it is necessary to weaken the influence of bright areas on defogging images. In order to distinguish the bright region from the dark channel region, a coefficient k is introduced here. When the difference between the pixel of the dark channel and the intensity of atmospheric light is more significant than, the region is a non-bright region; otherwise, it is a bright region. The determination formula of the bright area is as follows: Among them: ω is a regulating parameter in [0,1].In order to keep a small part of the fog in the distant view and make the contrast and saturation of the image more realistic, the value of 0.95 was determined after extensive experimental tests.

Fast bilateral filtering model
In processing the dark channel, smoothing the transmittance image after the bright area judgment is an important step, which has a significant impact on the subsequent image processing results. A fast algorithm based on the maximum operation is adopted in this paper. That is, the maximum operation is performed on Formula (7) : A fast bilateral filtering algorithm [9] is used to smooth the defogging image to further improve the block and halo effect in ~2 ( ) t x . The filtering algorithm comprises two weights of airspace and range, which can maintain nonlinear smoothing of edges. The weight in the neighborhood window in the filtering process is related to the pixel distance and gray difference of the center position, and the calculation formula is as follows: Where, S is the neighborhood window centered on pixel p ; p I and q I are the gray values of pixel p and q respectively. S σ and R σ are kernel functions of airspace S and range R respectively.

automatic color equalization (ACE) method
After optimizing the above algorithm, the edge distortion of the dark channel is significantly improved, and the value of transmittance is usually still small, leading to images with oversaturated colors and dark luminance. In order to solve the above problems, the use of automatic color equalization algorithm, this algorithm considering the color and brightness of the image space position relations, local characteristics of the adaptive filter, to realize the local and the nonlinear characteristics of the image with the adjustment of the brightness and color and contrast adjustment, at the same time satisfy the grey theory hypothesis and white spots assumptions. The ACE algorithm first adjusts the image color and corrects the image chromatic aberration to get a null domain reconstruction image; then dynamically expands the image to get a clear image.
is the difference between the brightness of point p and point j; ( ) r  is the relative brightness performance function; ( ) d  is the distance metric function between the two points.
( ) d  uses the less computationally intensive Manhattan distance and ( ) r  uses the saturation function. 1, where T is called the saturation threshold and is dynamically adjusted according to the actual application to obtain the desired effect.
The filtered intermediate results are dynamically expanded in the second step to obtain the image data output. 127 This equation ensures that both the actual gray and white spot hypotheses are met and combines the spatial domain model with the classical hypothesis.

Experimental results and analysis
In order to verify the effectiveness of the method in automated driving, pictures with fog in the driving scene were randomly selected. To verify the effectiveness of the improved algorithm, one is by comparing the variation between the step-by-step processing results, and the other is by comparing the variation between the multi-method processing results.

Step-by-step processing results
The scene image of foggy day driving is tested in Figure 2(a). This scene is a thin fog, the brightness is dark, and the observed vehicles slightly far away are no longer clear. Figure 2(b) shows the recovery results of the traditional dark channel algorithm, and it is observed that the brightness of the image after defogging is dark, and there are significant distortion blocks at the edges of the dark channel region. Figure 2(c) shows the recovery results of the fast bilateral filtering algorithm, and it is observed that this filtering algorithm optimizes the region edge distortion. Figure 2(d) shows the result of automatic color gradation equalization, which enhances the image's overall brightness and improves the dark details, resulting in a more vibrant color rendering overall.

Multi-method processing results
The image is processed with the traditional dark channel algorithm (Figure 3(a). Although it removes the haze, it makes the image brightness darker and becomes difficult to discern for black vehicles. The image processed with adaptive histogram equalization, as in Figure 3(b), the image as a whole is defogged for the near image, with significant brightness enhancement and sound color reproduction. However, with the increase of depth of field, there is the phenomenon of excessive contrast enhancement, a large amount of noise, and a significant color shift. The image processed by the multi-scale Retinex method, as shown in Figure 3(c), has relatively darker edges, color bias, loss of texture, and no significant defogging effect on the distant areas.
In a comprehensive comparison, the image quality after processing using the method in this paper is optimal, as in Figure 3(d), balancing the precise relationship between distant and near areas with enhanced contrast, minor color shift, and more precise edges. The sky halo phenomenon is optimized, the image's overall brightness is improved, and the clarity is significantly enhanced.
(a) Traditional dark channel method (b) Adaptive histogram equalization method (c) Multiscale Retinex method (d) Methodology of this paper Figure 3 Comparison of multi-method processing results

Objective parameter comparison and evaluation
Objective parameters are criteria for evaluating the images on the mathematical model. To demonstrate the effectiveness of this method, the traditional dark channel method, the adaptive histogram equalization method, and the multiscale Retinex method are applied to the randomly selected sample images. The objective parameter indicators are execution time, which indicates the speed of execution of the defogging algorithm; information entropy [10] , which indicates the amount of information contained in the image. The average value represents the brightness of the image.    From the statistics, we can see that the original image is in a foggy environment, the image is white, the mean value is high, and the brightness is brighter. Generally, the pixel mean value of the foggy image is higher, and after the fog is removed, the mean value of the image will generally decrease. However, with the traditional dark channel method, the mean value decreases too much relative to the original image, and the overall brightness is dark. The fog reduction increases the image details, and the information entropy generally increases. The application in automatic driving requires that the execution time be as short as possible.
In a comprehensive comparison, the method in this paper has a short execution time, the image mean value is between 120 and 180 [11] , which is most acceptable to the human eye, and the information entropy is the maximum value, indicating that the image processed by this paper shows more details and has better results in terms of image brightness and clarity.

Summary
Aiming at the problem of subject blur in the image collected by the automatic driving system, which affects vehicle detection accuracy by computer vision algorithm. In this paper, we propose a method for atmospheric light intensity valuation in an adaptive domain, which solves the poor defogging effect due to fixed atmospheric light intensity in classical dark channel theory. Then the combination of the extreme value filtering and fast bilateral filtering methods to optimize the atmospheric transmittance improves the efficiency of transmittance calculation. Finally, the multi-channel automatic color gradation equalization method is used to optimize further the image, which solves the problems of color oversaturation and brightness of the filtered image. The experimental results show that the results of this paper are better in terms of image clarity and overall brightness, the halo effect and block effect are also improved, the image contrast is good, and the color saturation and realism are effectively maintained, which can provide a better solution for the image detection and processing of automatic driving systems.