Enhancement of Asymmetrically Color-Cast Sandstorm Image Using Saturation-Based Color Correction and Hybrid Transmission Network

Abstract

The images discussed in this manuscript show atmospheric conditions of smog, sandstorm, and dust. Moreover, the images were taken in various environments and have features such as dimness or color cast. The smoggy image has a greenish or bluish color veil, and the sandstorm image has a yellowish or reddish color veil because of the various sand particles. Various methods have been used to enhance images containing dust. However, if the color-cast ingredients are not considered during image enhancement, then the enhanced image will have a new, artificial color veil that did not appear in the input image, as the color-veiled image does not have a uniform color channel. Certain channels are attenuated by sand particles. Therefore, this paper proposes a color-balancing method based on saturation to enhance asymmetrically cast colors due to the attenuation of the color channel by sand particles. Moreover, because the balanced image contains dust and the distribution of hazy ingredients is asymmetrical, a dehazing procedure is needed to enhance the image. This work used the original image and a reversed image to train the hybrid transmission network and generate the image’s transmission map. Moreover, an objective and subjective assessment procedure was used to compare the performance of the proposed method with that of other methods. Through the assessment, the performance of the proposed method was shown to be superior to other methods’ performance.

1. Introduction

The images discussed in this paper have diverse features with a hazy appearance or a color veil caused by various atmospheric circumstances. Hazy and dusty images are dim and unclear, and the sandstorm image contains a yellowish or reddish color veil because a certain color channel is attenuated by sand particles. Moreover, because the sandstorm image has low resolution and a rare color channel in certain environments, it presents a challenge in the areas of computer vision or image recognition. Therefore, a sandstorm image enhancement procedure is needed. Because sandstorm images and dusty images are obtained by a similar path, to enhance both images, a dehazing procedure is required. However, the existing dehazing methods have no color-balancing techniques; therefore, the enhanced image contains a new artificial color cast which was not visible in the input image with a color veil. Therefore, to enhance the sandstorm image naturally, a color-balancing procedure is needed.

The hazy-image-enhancement methods can be divided into two broad categories of machine-learning-based methods and non-machine-learning-based algorithms.

Numerous studies have been conducted on the use of non-machine-learning-based algorithms to enhance hazy images. He et al., proposed a dehazing algorithm using the dark channel prior (DCP) [1]. This method is usually applied for dehazing. However, because this method uses the constant kernel size to estimate the transmission map, the enhanced image has an artificial effect similar to the blocked effect. Meng et al., used the boundary constraint transmission map to enhance hazy images [2]. Their algorithm compensates for the DCP method using boundary constraints. Because this method has no color-balancing procedure, if the image has a cast color, the enhanced image will have an artificial color. Narasimhan et al., proposed a dehazing method using the image’s scene depth map, which is generated under different weather conditions [3]. Narasimhan et al., presented a hazy-image-enhancement algorithm according to changes in the scene color by atmospheric conditions [4]. Although this method enhances hazy images, the enhancement effect becomes low with the increase in depth. Zhao et al., enhanced hazy images using a transmission map with the pixel- and patch-wise method to compensate for the edge region of the existing transmission map [5]. In this method for enhancing hazy images, if the enhanced image is too dark, an exposure procedure is applied [5]. Tarel et al., proposed an image-enhancement method using white balance, atmospheric veil inference, and corner-preserving smoothing [6]. Nasseeba et al., enhanced hazy images using a depth estimation module to refine the transmission map with median filtering, a color analysis module using the gray world assumption, and a visibility restoration module to adjust the transmission map [7]. Schechner et al., proposed a hazy image enhancement algorithm using polarization [8]. Hong et al., enhanced hazy images using the adaptive gamma correction with the saturation increase [9]. However, in this method of hazy image enhancement, only the constant value that is not image-adaptive is used. Al-Ameen proposed a dusty-image-enhancement method using a tri-threshold with gamma correction [10]. Although this method enhances dusty images with a color cast, the constant value is not suitable for various other image conditions. Shi et al., enhanced dusty images using a normalized gamma transform and a contrast limit adaptive histogram equalization [11]. This method enhances color-cast dusty images; however, to balance the color components, this method uses a mean shift in color ingredients, and an artificial color cast can appear in the enhanced image. Cheng et al., enhanced images containing sand dust using blue channel compensation [12]. This method enhances the color-cast sandstorm image suitably; however, if the image’s color channel is too rare, a new artificial color veil can appear. Cheng et al., proposed a sandstorm-image-enhancement algorithm using the blue channel prior and white balance [13]. Gao et al., established a sand-dust-image-enhancement method using the blue channel prior and color-balancing method [14]. This method enhances the color-cast sand-dust image sufficiently; however, in the case of greatly attenuated images, because certain color channels are rare, the enhanced image could show a new color veil. Shi et al., proposed a sandstorm-image-enhancement algorithm using the compensated dark channel prior [15]. This method also uses the mean shift in color ingredients to enhance the color cast; however, a newly cast color can be seen in the enhanced image.

Furthermore, a great number of studies have focused on how to enhance hazy images based on machine learning. Zhu et al., enhanced hazy images using color attenuation prior and by training the depth map [16]. Ren et al., enhanced hazy images using two kinds of multi-scale convolutional neural networks: one generating a transmission map and the other generating a refining-of-transmission map [17]. Although their method enhances the hazy image suitably, because the training image is taken in the daytime, the nighttime image is not sufficiently enhanced. Wang et al., enhanced hazy images using atmospheric illumination prior and multiscale convolutional neural network [18]. This method effectively estimates the transmission map; however, in some images, the sky region of the transmission map is not well-estimated [18]. Lee enhanced the sandstorm image using image adaptive eigenvalue and brightness adaptive dark channel network [19]. Santra et al., improved hazy images using a transmittance map and environmental illumination [20]. This method enhances hazy images; however, for images taken at nighttime, the enhanced image has an artificial effect because the synthetic image does not include the nighttime environment. Yu et al., enhanced hazy images using ensemble learning with a two-branch neural network [21]. Zhou et al., improved the haze image using robust polarization and neural networks [22]. However, since this method is not able to estimate certain conditions of the atmosphere, the enhanced image has an artifact effect in some cases [22]. Zhang et al., enhanced the hazy image using a pyramid channel-based feature attention network, which has three modules with three-scale feature extraction, a pyramid channel-based feature attention, and a reconstruction module to extract diverse characteristics of the image [23]. Machine-learning-based dehazing methods are sometimes superior to non-machine-learning algorithms; however, creating a hazy dataset is a difficult task, and the synthetic image cannot contain the various circumstances required to train the neural network sufficiently.

The sandstorm image has a color veil due to the attenuation of color components. Therefore, to enhance the sandstorm image naturally, asymmetrically casted color needs a color-balancing procedure. Moreover, because the color-balanced image only appears to be a dusty image without a color veil, to enhance the image sufficiently, an image-adaptive dehazing procedure is needed because the distribution of hazy particles is asymmetrical. Therefore, this paper proposes an image-adaptive color-balancing method and a dehazing method, respectively.

2. Proposed Method

2.1. Saturation-Based Asymmetric Color-Channel Compensation

The sandstorm image has a certain color veil, which is either reddish or yellowish, due to the attenuation of the color channel. Moreover, the distribution of the color channel is asymmetrical. In order to balance the asymmetrical color components of an image naturally by attenuation on the color channel, it needs certain parameters that show the image’s characteristic. Hong et al. [9] enhanced hazy images using gamma correction with the value channel of the HSV domain and an increase in the saturation channel. As the hazy image only has dim characteristics, not a color cast, variations in saturation can lead to a new color cast. The saturation of image shows how the color is mixed and whether the color is dark or light. The sandstorm image has a yellowish or reddish color cast. To balance the image’s color, if the hue channel is controlled, then the image color is changed, leading to an artificial effect. However, because the image saturation shows how the color is mixed, to balance the color-casted image, the image saturation should be adaptively controlled.

Figure 1 provides an overview of the color-balancing procedure used in the proposed method. Figure 1a is an asymmetrically color-casted sandstorm image; Figure 1b shows a variation of saturation (variation in circle’s position: brown dotted arrow and circle are the saturation position of the yellowish casted sandstorm image; black dotted arrow and blue circle are the saturation position of the color-balanced sandstorm image). Figure 1c is a color-balanced image. As shown in Figure 1a–c, if the image saturation is changed, then the color veil of the image can be compensated. Because the color-casted image can be balanced by a change of saturation, this work proposes an image-adaptive color-balancing algorithm based on variations of saturation.

2.1.1. Color-Compensation Measure on Reddish or Orange Images

The sandstorm image has two types of color cast: reddish (yellowish) by sand particles or greenish (bluish) by smog particles. The reddish color-veiled image has the highest mean value for the red channel and a lower mean value for the blue or green channel than that of the red channel due to the color channel attenuation. The greenish color-veiled image has the highest mean value for the green channel and a lower mean value for the red and blue channel than that of the green color channel due to the color channel attenuation. Accordingly, to enhance the reddish and orange color-casted sandstorm images, this paper proposes the following color-balancing parameters:

$$rati{o}_{rb}=m\left(I^r\right)-m\left(I^b\right),$$

(1)

$$rati{o}_{rg}=m\left(I^r\right)-m\left(I^g\right),$$

(2)

where $rati{o}_{rb}$ and $rati{o}_{rg}$ are the average difference in values between red and blue color channels or red and green color channels, respectively, $m\left(·\right)$ is the average operator. If the sandstorm image has a reddish or yellowish color veil, then the mean value of the red channel of the image is bigger than that of other color channels; therefore, the $rati{o}_{rb}$ and $rati{o}_{rg}$ values are always more than zero. As the ratios are applied as follows:

$$rati{o}_{RY}=\left(rati{o}_{rb}+rati{o}_{rg}\right)·\omega ,$$

(3)

$$\omega =\underset{}{\mathit{max}}\{m(I^r), m\left(I^g\right), 1-m(I^b)\},$$

(4)

where $rati{o}_{RY}$ is the ratio of the reddish or yellowish sandstorm image, $\omega$ is the controlling parameter according to the image condition. If the image has a high level of color-casting and is reddish or yellowish, then the blue color channel of the sandstorm image is the rare condition, and the average value of the blue channel is lowest. However, because the average value of the reversed blue channel is the highest, the weight is somewhat uniform. Moreover, if the image has a light color veil because the average values of the color channels are somewhat uniform, the average value of the reversed color channel is uniform. Therefore, using Equation (4), the ratio can be changed, including adaptive image conditions.

2.1.2. Color-Compensation Measures for Greenish or Bluish Images

The reddish color-casted image has an imbalanced color channel, as the red channel is more abundant than other channels. Meanwhile, if the image has a greenish or bluish color veil, then the average value of the green channel is higher than other color channels. That is, the average value of the greenish or bluish image is higher than that of the red color channel. Therefore, if the $rati{o}_{rb}$ or $rati{o}_{rg}$ is lower than zero or equal zero, to enhance the greenish color-casted image, this paper uses the average value difference in color channels as follows:

$$rati{o}_{gr}=m\left(I^g\right)-m\left(I^r\right),$$

(5)

$$rati{o}_{gb}=m\left(I^g\right)-m\left(I^b\right),$$

(6)

where $rati{o}_{gr}$ and $rati{o}_{gb}$ show the average difference in values between the green channel and red or blue channel.

$$rati{o}_{GB}=\left(rati{o}_{gr}+rati{o}_{gb}\right)·\omega ,$$

(7)

where $rati{o}_{GB}$ is the ratio of the greenish or bluish color-casted image.

2.1.3. Color Compensation Using Image-Adaptive Measures

The ratios obtained by Equations (1)–(7) are applied to balance the image based on saturation, as follows:

$$S_p=S-rati{o}_{\phi \in \left\{RY,GB\right\}},$$

(8)

where $S_p$ is the saturation channel enhanced by the proposed method; $S$ is the saturation channel of the input image; $rati{o}_{\phi \in \left\{RY,GB\right\}}$ is the image-adaptive ratio of the greenish or reddish color-casted image obtained by Equations (1)–(7) where Hong et al. [9] enhances hazy image by increasing saturation of image with constant $rati{o}_{\phi \in \left\{RY, GB\right\}}$ value however it is not image adaptive. Using Equations (1)–(8), although the color channel of the image is rare, the enhanced image has a balanced natural color channel.

Figure 2 shows a balanced image, with the color-casted image and non-color-casted image. Figure 2b shows the color-balanced image obtained by Hong et al.’s [9] method. The image improved by Hong et al. [9] still has a shifted color because this method increases saturation with a constant value to enhance the hazy image. Meanwhile, the proposed color-balancing algorithm shown in Figure 2c shows a nice performance for both greatly color-casted images and non-color-casted images due to the image-adaptive saturation variations. Therefore, the proposed color-balancing algorithm is suitable for enhancing the sandstorm image.

2.2. Hybrid Transmission Network

The color-balanced image has hazy features, similar to dim images. Moreover, because haze particles are distributed asymmetrically, a dehazing procedure is required to enhance hazy images. The existing dehazing methods usually use dark channel prior (DCP) [1]. This method is useful for estimating the transmission map of a single image. However, when a transmission map is estimated because the constant kernel size is used to estimate the image’s dark region, the estimated image has a blocked area. Meanwhile, because the convolutional neural network (CNN) uses various kernel sizes, it can generate a transmission map naturally. Therefore, this work estimated the transmission map using CNN. When training the neural network, various training data are needed. Because acquiring the transmission map of images is a challenging task, a synthetic dataset is used. However, because the synthetic dataset does not contain various image circumstances, the enhanced image contains artifact phenomena. The transmission map is defined as [1]:

$$t\left(x\right)=ex{p}^{-\beta d\left(x\right)},$$

(9)

where $\beta$ is a scattering parameter; $d\left(x\right)$ is a depth map of the image. The transmission map is changed by the $\beta$ parameter. The transmission map has diverse features according to whether the $\beta$ parameter is low or high. Therefore, this work generated a suitable transmission map using variation in the $\beta$ parameter, and called this the ground truth image of the transmission map. The proposed transmission map was obtained as follows:

$$t_i\left(x\right)=ex{p}^{-{\beta }_i d\left(x\right),}$$

(10)

$$t_g\left(x\right)=\frac{1}{N}{{\displaystyle \sum }}_{i=1}^N{t}_i\left(x\right),$$

(11)

where $t_g\left(x\right)$ is the ground truth transmission map; $N$ is the length of ${\beta }_i$; $t_i\left(x\right)$ is $i^{th}$ transmission map; ${\beta }_i\in \left[0.5, 2\right)$ with 0.1 intervals. Obtained using Equations (10) and (11), the generated ground truth transmission map has diverse features. To generate the image-adaptive transmission map, this work used the hybrid transmission map applying the theories of dark channel prior (DCP) [1] and bright channel prior (BCP) [24]. The DCP estimates the dark region of images; however, if the image has bright regions, such as sky regions, the estimated image is still bright, not dark, and the enhanced image using the estimated DCP has an artificial effect. Additionally, because the BCP [24] estimates the bright region if DCP [1] and BCP [24] are hybridized, the estimated transmission map will be more natural. Therefore, this work used the hybrid transmission map with DCP [1] and BCP [24]. The DCP [1], BCP [24], and transmission map were obtained as follows:

$$I_d\left(x\right)=\underset{c}{\mathit{min}}\left(\underset{y\in \Omega \left(x\right)}{\mathit{min}}\left(\frac{I^c\left(y\right)}{A^c}\right)\right),$$

(12)

$$I_b\left(x\right)=\underset{c}{\mathit{max}}\left(\underset{y\in \Omega \left(x\right)}{\mathit{max}}\left(\frac{I^c\left(y\right)}{A^c}\right)\right),$$

(13)

$$t\left(x\right)=1-I_{d or b}\left(x\right),$$

(14)

where $I_d\left(x\right)$ is the dark channel, $I_b\left(x\right)$ is the bright channel, $\Omega \left(x\right)$ is the patch size used to estimate dark or bright region, $A^c$ is backscatter light, $c\in \left\{r,g,b\right\},$ and $t\left(x\right)$ is the transmission map obtained by reversing the dark channel or bright channel. By using Equations (12)–(14), to estimate dark or bright regions, a certain size kernel can be applied, and because the transmission map is estimated by reversing the dark channel or bright channel, the enhanced image can obtain a blocked effect by using a constant kernel size. Lee [19] designs neural network applying DCP theory [1]. Therefore, this work, aiming to estimate a transmission map without blocked phenomenon, used multi-scale CNN because CNN has a diverse kernel size with DCP [1] and BCP [24] theories. The brief design of neural network is as follows:

$$l_{t_d}\left(x\right)=1-l_d\left(x\right),$$

(15)

$$l_{t_b}\left(x\right)=1-l_b\left(x\right),$$

(16)

$$l_{t_p}\left(x\right)=cat\left(l_{t_d}\left(x\right), l_{t_b}\left(x\right)\right),$$

(17)

where $l_{t_d}\left(x\right)$ is the transmission layer obtained applying DCP theory [1], $l_{t_b}\left(x\right)$ is the transmission layer obtained applying BCP theory [24],$l_d\left(x\right)$ is the dark channel layer with minimum pooling, $l_b\left(x\right)$ is the bright channel layer with maximum pooling, cat($·)$ is the concatenate layer, and the rectified linear unit (ReLU) [25] is used as an activation function with each convolution layer and post-arithmetic operation. This work aims to apply the theories of DCP [1] and BCP [24] using minimum pooling and maximum pooling, respectively. Moreover, to obtain various image characteristics, multi-scale convolutional neural networks are applied.

Figure 3 provides an overview of the proposed neural networks and individual networks. Figure 3a provides an overview of the proposed method, while Figure 3b shows the networks of the dark channel: the brown color is the minimum pooling layer, sky blue color is the convolution layer, green color is the up-sampling layer, and the dark blue color is the concatenate layer. This network has 10 convolution layers, 2 minimum pooling layers, 3 up-sampling layers, and 4 concatenate layers. Figure 3c shows the networks of the multi-scale bright channel: the yellow color is the maximum pooling layer, sky blue color is the convolution layer, dark blue is the concatenate layer, and green color is the up-sampling layer. This network has 8 convolution layers, 2 maximum pooling layers, 2 up sampling layers, and 2 concatenate layers. Figure 3d shows a hybrid network: the sky blue color indicates the convolution layer, dark blue is the concatenate layer. This network has 2 convolution layers, and 1 concatenate layer. Moreover, the yellow rectangular shape shown in Figure 3b–d indicates the grouping of unit layers, where 1/2 and x2 indicate variation of size; downsize as 1/2, upsize as x2, the number below the layers indicates the channel size. The networks partially applied a U-net [26] architecture with a multi-scale resolution to obtain the various image characteristics.

Figure 4 shows ground truth transmission maps, the transmission map generated by the proposed algorithm, and the existing transmission map. In the existing methods established by He et al. [1], Santra et al. [20], Ren et al. [17], Meng et al. [2], and Zhao et al. [5], the bright region is too dark or too bright; however, the transmission map generated by the proposed algorithm estimates bright and dark regions suitably. Therefore, the proposed algorithm is competitive in terms of transmission map estimation.

2.3. The Training Environment Set

The color-balanced image has diverse features, such as dusty or hazy. Therefore, to enhance hazy images suitably, the training dataset should also be diverse. In order to train the neural network suitably, this work used a D hazy dataset [27], which has 1449 original images, synthetic hazy images, and depth map images. Moreover, during the training, 10% of 1449 images were used for validation, and 90% of 1449 images were used for training. Moreover, the hybrid loss function was applied for the loss function with mean squared error (MSE) and structural similarity index measure (SSIM) [28] as follows:

$$L_p=L_{mse}+L_{\alpha },$$

(18)

$$L_{\alpha }=\frac{L_{mse} · L_{ssim}}{L_{mse}+L_{ssim}},$$

(19)

$$L_{mse}=\frac{1}{N}{{\displaystyle \sum }}_{i=1}^N{e}_i^2,$$

(20)

$$L_{ssim}=\frac{\left(2·{\mu }_t·{\mu }_G+C_1\right)·\left(2·{\sigma }_{tG}+C_2\right)}{\left({\mu }_t^2+{\mu }_G^2+C_1\right)·\left({\sigma }_t^2+{\sigma }_G^2+C_2\right)},$$

(21)

where $L_{mse}$ is mse loss function; e is error; $L_{ssim}$ is ssim loss function; ${\mu }_t$ mis the average intensity of the target image; ${\mu }_G$ is the average intensity of the generated image; ${\sigma }_{tG}$ mis the correlation coefficient; ${\sigma }_t$ mis standard deviation of the target image; ${\sigma }_G$ is the standard deviation of the generated image. $C_1, C_2$ are constant values. Using Equation (18), the loss value can be adjusted more suitably because both SSIM [28] and MSE indicate the similarities between two objects in different ways, and Adam optimizer [29] is used. Moreover, the train batch size and validation batch size were set as 8, while the learning rate and weight decay were, respectively, 0.0001 and 20 epoch sets. While network is training, per 1 epoch, 163 iterations with 8 batches = 1304 images are used, and when 20 epochs, 163 iterations, and 8 batches = 26,080 images are trained. Moreover, during validation, per 1 epoch, 18 iterations with 8 batches = 144 images are used, and when 20 epochs, 18 iterations, and 8 batches = 2880 images are used for validation, approximately. Moreover, to show accuracy during the training, SSIM [28] measure is used. The hardware environment set was Intel ^® Core™ i7-8700 CPU @3.20 GHz, 32 GB RAM, 12 GB Geforce RTX 2060, and 6 GB Geforce GXT 1660 super.

Figure 5 shows the variation in loss function and accuracy of the training. The loss value gradually converges, and accuracy gradually converges.

Additionally, the detection in adverse weather nature (DAWN) dataset [30] was used, which has 323 natural sandstorm images to validate the trained data.

2.4. Image Recovery

The sandstorm image has color-casted characteristics due to the color of sand particles. In order to improve this phenomenon, this work proposed a color-balancing algorithm based on saturation. The balanced image seems hazy. Therefore, to enhance the image, this work used the CNN with a hybrid transmission map. This section, using the color-balanced image and generated transmission map, recovered the image as follows [1,4,31,32,33]:

$$J^c\left(x\right)=\frac{I_B^c\left(x\right)-A_B^c}{\max \left(t_p\left(x\right), t_0\right)}+A_B^c,$$

(22)

where $J^c\left(x\right)$ is the enhanced image; $x$ is the pixel location; $I_B^c\left(x\right)$ is the color-balanced image obtained using the proposed method; $t_p\left(x\right)$ is the generated transmission map; $t_0^{}$ sets 0.1 to prohibit divided 0; $A_B^c$ is the backscatter light of the balanced image obtained by He et al. [1] method. Moreover, to refine the enhanced image, this work applied a guided image filter [34] as follows:

$$J_G^c\left(x\right)=G_f\left\{J^c\left(x\right), K, eps\right\},$$

(23)

$$J_{en}^c\left(x\right)=\left\{J^c\left(x\right)-J_G^c\left(x\right)\right\}·ratio+J_G^c\left(x\right),$$

(24)

where $J_G^c\left(x\right)$ is guided filtered image; $G_f^{}\left\{·\right\}$ is the guided filter; $K$ is kernel size, set as 16; $eps$ was set as ${0.1}^2$; $J_{en}^c\left(x\right)$ is the refined enhanced image; $ratio$ was set as 5.

Figure 6 shows the color-balanced image, transmission map, and enhanced image obtained by the methods of He et al. [1] and Santra et al. [20]. Figure 6b shows the color-balanced image; Figure 6c,d shows the transmission map and enhanced image obtained by He et al. [1] using Figure 6b. Figure 6e,f shows the transmission map and enhanced image obtained by Santra et al. [20] method using Figure 6b. Figure 6g,h shows the transmission map and enhanced image obtained by the proposed algorithm using Figure 6b. The enhanced images obtained by He et al. [1] and Santra et al. [20] contain an artificial effect due to the transmission map. Meanwhile, the enhanced image obtained by the proposed algorithm has no artificial effect.

3. Experiment Result and Discussion

The color-casted sandstorm image is balanced by the proposed algorithm, and the balanced image has a hazy characteristic. In order to enhance the hazy image, this work applied the dehazing algorithm using CNN. This section shows the suitable performance of the proposed algorithm to enhance the sandstorm image. The categories of the assessment procedure are divided into two. The first is a subjective assessment, and the other is an objective assessment. Moreover, because the sandstorm image has a casted color, the subjective assessment is divided into two branches: color correction, and image enhancement, through comparison with state-of-the-art methods.

3.1. Subjective Assessment

The sandstorm image has a yellowish or reddish color cast. Therefore, to assess the enhanced sandstorm image subjectively, two procedures are required: color balancing and image enhancement. Therefore, this work was divided into two branches: color correction and enhanced image, and compared with state-of-the-art methods such as those of Al Ameen [10], Shi et al. [11], Shi et al. [15], Gao et al. [14], Ren et al. [17], He et al. [1], Meng et al. [2], Santra et al. [20], Zhao et al. [5], Hong et al. [9], and Yu et al. [21]. Moreover, to conduct comparisons in various environments, the detection in adverse weather nature (DAWN) dataset [30] was used, which has 323 natural sandstorm and dust storm images.

3.1.1. Color Correction

This section shows the image color is balanced compared with the state-of-the-art methods, such as those of Al Ameen [10], Shi et al. [11], Shi et al. [15], and Hong et al. [9], using the DAWN dataset [30].

Figure 7 and Figure 8 compare the color-balancing effect to that obtained with state-of-the-art methods. Shi et al.’s [11,15] methods contain a color-balancing procedure; however, the color-balanced image has a bluish artificial effect because these methods are used to balance the color channel mean shift of the color components. The color-balanced image obtained by Al Ameen [10] method has a yellowish or reddish casted color, which appears due to the use of a constant value; moreover, this is not an image-adaptive measure to enhance the image. Because Hong et al. [9] use an increase in saturation to enhance hazy images, if the image contains color-casting, then the balanced image will still contain a color-shifted effect, which may thicken due to the increase in saturation. However, the color-balanced image obtained by the proposed method has no color-casted effect.

3.1.2. Enhanced Image

The balanced image obtained using the proposed method is more natural than those obtained using the other methods. Therefore, this section assesses the enhancement of hazy images compared with state-of-the-art methods, such as those of Al Ameen [10], Shi et al. [11], Shi et al. [15], and Gao et al. [14]. Moreover, because the color-balanced image has hazy characteristics, existing dehazing algorithms are used, such as those of Ren et al. [17], He et al. [1], Meng et al. [2], Santra et al. [20], Zhao et al. [5], Hong et al. [9], and Yu et al. [21]. He et al. [1], Meng et al. [2], and Zhao et al.’s [5] methods enhance hazy images using DCP. Ren et al. [17], Santra et al. [20], and Yu et al.’s [21] methods enhance hazy images using CNN. Moreover, Hong et al. [9] enhance hazy images using gamma correction and increased saturation. Meanwhile, Gao et al. [14], Shi et al. [11], Shi et al. [15], and Al Ameen’s [10] methods enhance the sandstorm image, and these methods use the color-balancing procedure.

Figure 9 and Figure 10 show a performance comparison of the proposed method and state-of-the-art methods. He et al. [1] and Meng et al.’s [2] methods enhance hazy images; however, when used with color-casted images, the enhanced image has an artificial color because these methods have no color compensation procedure. Shi et al.’s [11,15] algorithms enhance sandstorm images, although these contain a color veil. However, due to the mean shift in color ingredients to balance the color channel, these methods sometimes have an artificial bluish color. Gao et al. [14] enhanced the sandstorm image. However, the enhanced image seems dim because of the transmission map. Al Ameen [10] enhanced a sandstorm image with a lightly casted color because this method uses a constant value to enhance the image and is not an image-adaptive measure. Ren et al. [17] and Santra et al.’s [20] methods enhance hazy images using CNN; however, these methods have no color-compensation procedure, and the enhanced image shows a color shift. Moreover, the image enhanced by Hong et al. [9] also has a casted color because this method does not have an image-adaptive color-balancing procedure but increases the image saturation of the image, meaning that the enhanced image has a thicker casted color veil. Yu et al. [21] enhanced hazy images; however, because this method has no color-compensation procedure, the enhanced image has color-shift components. Zhao et al. [5] improved a lightly color-casted sandstorm image; however, because this method has no suitable color-correction procedure, the enhanced reddish or orange color-casted image still contains color ingredients. Meanwhile, the image enhanced by the proposed algorithm has no shifted color and no artificial effect. Therefore, the proposed algorithm is suitable for application in the sandstorm image-enhancement sphere.

Figure 7. The performance comparison of color-balancing algorithms using state-of-the-art methods and the proposed method: (a) input; (b) Shi et al. [15]; (c) Shi et al. [11]; (d) Al Ameen [10]; (e) Hong et al. [9]; (f) Proposed method.

Figure 7. The performance comparison of color-balancing algorithms using state-of-the-art methods and the proposed method: (a) input; (b) Shi et al. [15]; (c) Shi et al. [11]; (d) Al Ameen [10]; (e) Hong et al. [9]; (f) Proposed method.

Figure 8. The performance comparison of color-balancing algorithms using state-of-the-art methods and the proposed method: (a) input; (b) Shi et al. [15]; (c) Shi et al. [11]; (d) Al Ameen [10]; (e) Hong et al. [9]; (f) Proposed method.

Figure 8. The performance comparison of color-balancing algorithms using state-of-the-art methods and the proposed method: (a) input; (b) Shi et al. [15]; (c) Shi et al. [11]; (d) Al Ameen [10]; (e) Hong et al. [9]; (f) Proposed method.

Figure 9. The performance of the enhanced image compared to state-of-the-art methods and the proposed method: (a) input; (b) He et al. [1]; (c) Meng et al. [2]; (d) Hong et al. [9]; (e) Shi et al. [15]; (f) Shi et al. [11]; (g) Gao et al. [14]; (h) Al Ameen [10]; (i) Ren et al. [17]; (j) Santra et al. [20]; (k) Zhao et al. [5]; (l) Yu et al. [21]; (m) Proposed method.

Figure 9. The performance of the enhanced image compared to state-of-the-art methods and the proposed method: (a) input; (b) He et al. [1]; (c) Meng et al. [2]; (d) Hong et al. [9]; (e) Shi et al. [15]; (f) Shi et al. [11]; (g) Gao et al. [14]; (h) Al Ameen [10]; (i) Ren et al. [17]; (j) Santra et al. [20]; (k) Zhao et al. [5]; (l) Yu et al. [21]; (m) Proposed method.

Figure 10. The performance of the enhanced image compared with state-of-the-art methods and the proposed method: (a) input; (b) He et al. [1]; (c) Meng et al. [2]; (d) Hong et al. [9]; (e) Shi et al. [15]; (f) Shi et al. [11]; (g) Gao et al. [14]; (h) Al Ameen [10]; (i) Ren et al. [17]; (j) Santra et al. [20]; (k) Zhao et al. [5]; (l) Yu et al. [21]; (m) Proposed method.

Figure 10. The performance of the enhanced image compared with state-of-the-art methods and the proposed method: (a) input; (b) He et al. [1]; (c) Meng et al. [2]; (d) Hong et al. [9]; (e) Shi et al. [15]; (f) Shi et al. [11]; (g) Gao et al. [14]; (h) Al Ameen [10]; (i) Ren et al. [17]; (j) Santra et al. [20]; (k) Zhao et al. [5]; (l) Yu et al. [21]; (m) Proposed method.

3.2. Objective Assessment

The color-casted sandstorm image is balanced by the proposed algorithm, and Figure 7 and Figure 8 show the performance of the proposed algorithm, which was shown to be suitably compared with state-of-the-art methods. Moreover, the dehazing algorithm used by the proposed method is superior to state-of-the art-methods in subjective terms. This section assesses the performance of the proposed method and how suitable it is for the enhancement of sandstorm images. To objectively assess the enhanced image, this work used two metrics: the natural image quality evaluator (NIQE) [35] and the underwater image quality measure (UIQM) [36]. The NIQE [35] metric indicates how natural an image is. The lower the NIQE [35] score, the more natural the enhanced image and the better its quality. Meanwhile, the UIQM [36] score shows how well an image is enhanced in terms of the image’s contrast, colorfulness, and sharpness. The higher the UIQM [36] score, the more enhanced the image. Moreover, to assess the generated transmission map, SSIM [28] and MSE metrics are used.

Table 1. The comparison of transmission map obtained through SSIM [28] and MSE metrics with state-of-the-art methods shown in Figure 4 and D hazy dataset [27] (PM is proposed method).

SSIM [28]	[1]	[20]	[17]	[2]	[5]	PM		MSE	[1]	[20]	[17]	[2]	[5]	PM
	0.872	0.789	0.912	0.758	0.621	0.950			0.029	0.071	0.061	0.065	0.050	0.003
	0.858	0.829	0.828	0.766	0.823	0.952			0.026	0.046	0.133	0.045	0.034	0.008
	0.794	0.895	0.820	0.559	0.712	0.927			0.036	0.014	0.128	0.092	0.061	0.005
	0.769	0.799	0.891	0.610	0.694	0.926			0.063	0.085	0.058	0.127	0.094	0.015
	0.771	0.784	0.874	0.609	0.672	0.941			0.054	0.078	0.088	0.100	0.069	0.004
AVG (5)	0.813	0.819	0.865	0.660	0.704	0.939		AVG(5)	0.042	0.059	0.094	0.086	0.062	0.007
AVG (1449)	0.789	0.778	0.878	0.636	0.680	0.920		AVG(1449)	0.052	0.083	0.070	0.107	0.076	0.010

shows how similar the transmission map is to the ground truth. The transmission map by Ren et al. [17] is more similar to the ground truth image, although the transmission map is dimmer than those obtained using He et al. [1], Santra et al. [20], Zhao et al. [5], and Meng et al.’s [2] methods according to their SSIM [28] score. Meanwhile, the transmission map used in He et al.’s [1] method is more similar to the ground truth image than those of Ren et al. [17], Santra et al. [20], Zhao et al. [5], and Meng et al. [2] according to MSE score. The transmission map obtained by the proposed method is more similar to the ground truth image than those of He et al. [1], Santra et al. [20], Ren et al. [17], Meng et al. [2], and Zhao et al. [5] according to both SSIM [28] and MSE scores.

Table 2. The comparison of enhanced image through the NIQE [35] metric with state of the art methods in Figure 9 (the lower score is the better-enhanced image, PM is proposed method).

	[1]	[9]	[14]	[10]	[2]	[15]	[17]	[11]	[20]	[21]	[5]	PM
	19.694	19.793	19.927	20.142	18.892	19.604	19.740	19.955	19.748	19.366	18.893	18.535
	20.280	20.164	20.343	20.579	20.351	20.275	20.336	20.129	20.264	20.061	20.148	19.829
	19.554	19.866	19.774	19.780	19.468	19.524	19.618	19.583	19.629	19.931	19.247	18.317
	19.689	19.671	19.741	19.692	19.512	19.588	19.637	19.635	19.595	19.629	19.278	19.094
	19.770	19.842	20.104	19.413	20.261	19.842	20.085	19.301	19.706	19.929	20.257	17.154
	19.934	20.371	20.404	20.588	19.635	19.737	20.130	20.151	20.105	20.327	19.409	19.040
	20.157	19.868	20.051	20.383	20.088	20.285	20.387	20.114	20.341	20.245	20.105	19.183
	19.883	19.794	19.750	18.936	19.962	19.599	20.032	19.275	19.942	20.153	19.864	16.758
	20.338	20.238	20.318	20.154	20.227	20.130	20.173	20.288	20.128	19.978	20.369	19.273
	20.105	19.723	20.121	20.253	20.135	19.719	20.193	19.695	20.057	19.639	19.790	19.263
AVG	19.940	19.933	20.053	19.992	19.853	19.830	20.033	19.813	19.952	19.926	19.736	18.645

and

Table 3. The comparison of enhanced image through the NIQE [35] metric with state-of-the-art methods in Figure 10 (the lower score is the better enhanced image, PM is proposed method).

	[1]	[9]	[14]	[10]	[2]	[15]	[17]	[11]	[20]	[21]	[5]	PM
	19.627	19.815	20.158	19.617	19.440	19.824	20.092	19.614	20.571	19.668	19.483	17.613
	20.985	21.170	21.436	21.353	20.302	21.090	21.307	20.918	21.176	21.492	20.356	19.672
	19.614	19.742	19.695	19.259	19.516	19.466	19.193	19.138	19.439	19.671	19.764	16.848
	20.204	20.420	20.347	20.309	20.209	20.048	20.194	20.221	20.132	21.216	19.744	19.530
	19.897	20.016	19.959	20.105	19.688	19.738	19.944	19.918	19.909	20.153	19.517	19.336
	20.781	20.363	20.300	19.806	20.706	20.232	20.881	20.222	20.740	20.448	21.109	17.506
	19.904	19.822	19.757	19.781	19.553	19.697	19.810	19.728	19.836	20.046	19.523	17.430
	19.495	19.278	18.424	19.116	19.691	19.040	19.617	18.561	19.700	19.175	18.437	16.002
	19.825	19.947	19.829	19.713	19.847	19.783	19.887	19.939	19.894	20.328	19.915	19.023
	19.826	19.595	19.774	19.888	19.900	19.633	19.871	19.600	19.786	19.771	19.652	19.181
AVG	20.016	20.017	19.968	19.895	19.885	19.855	20.080	19.786	20.118	20.197	19.750	18.214

show the NIQE [35] scores of Figure 9 and Figure 10. The lower the NIQE score, the better enhanced and more natural the image. The NIQE score obtained by He et al. [1] is higher than that obtained by Gao et al. [14] in some images because He et al.’s [1] method has no color-compensation procedure. Gao et al. [14] obtained a higher NIQE score than Al Ameen [10], though the image enhanced by Gao et al. [14] method has a smaller color shift than that obtained using Al Ameen’s method [10]. Meng et al.’s [2] method has a lower NIQE score than Gao et al.’s [14] method, although the enhanced image contains casted color. Shi et al. [15] obtained a lower NIQE score than Meng et al. [2] because Shi et al. [15] used a color-compensation procedure. Ren et al. [17] obtained a higher NIQE score than Shi et al. [15] because Ren et al. [17] did not contain a color-compensation procedure. Shi et al. [11] obtained a lower NIQE score than Ren et al. [17] in some images, because Ren et al.’s [17] method does not contain a color-compensation procedure. Santra et al. [20] obtained a higher NIQE score than that of Shi et al. [11] in some images because Santra et al.’s [20] method contains no color-compensation procedure. Hong et al. [9] obtained a higher NIQE score than Shi et al. [11] because Shi et al. [11] used a color-balancing procedure. Yu et al. [21] obtained a higher NIQE score than Zhao et al. [5] and Shi et al. [11] because Shi et al.’s [11] method contains a color-compensation procedure. Although the image has a shifted color, the NIQE score is lower than that of the non-color-shifted image. Therefore, the NIQE score is not an absolute but a referenceable measure. The proposed method has a lower NIQE score than other methods.

Table 4. The comparison of enhanced image through the UIQM [36] metric with state-of-the-art methods in Figure 9 (the higher score denotes a better enhanced image; PM is proposed method).

	[1]	[9]	[14]	[10]	[2]	[15]	[17]	[11]	[20]	[21]	[5]	PM
	0.865	0.768	0.652	1.158	1.364	0.968	1.037	0.959	1.098	0.984	1.581	1.543
	0.364	0.511	0.301	0.834	0.643	0.751	0.514	0.478	0.613	0.654	1.124	0.941
	0.906	0.689	0.564	0.870	1.226	0.860	0.970	0.857	0.954	0.719	1.310	1.410
	0.485	0.444	0.361	0.735	0.790	0.799	0.608	0.617	0.758	0.757	0.979	1.205
	0.934	0.718	0.765	0.946	0.937	0.899	0.896	1.005	0.905	0.674	0.966	1.476
	0.945	0.579	0.532	0.806	1.102	1.178	0.783	0.795	0.722	0.691	1.224	1.456
	0.420	0.498	0.387	0.905	0.780	0.684	0.770	0.544	0.672	0.448	1.007	1.009
	0.811	0.635	0.757	1.093	0.826	1.180	0.801	0.947	0.774	0.628	0.891	1.511
	0.429	0.444	0.527	0.867	0.479	0.722	0.599	0.573	0.470	0.443	0.634	1.237
	0.812	0.917	0.745	1.237	0.757	0.870	0.907	0.941	0.819	0.876	1.048	1.483
AVG	0.697	0.620	0.559	0.945	0.890	0.891	0.789	0.772	0.779	0.687	1.076	1.327

and

Table 5. The comparison of enhanced image through the UIQM [36] metric with state-of-the art-methods in Figure 10 (the higher score denotes a better-enhanced image; PM is proposed method).

	[1]	[9]	[14]	[10]	[2]	[15]	[17]	[11]	[20]	[21]	[5]	PM
	1.017	0.729	0.679	1.081	1.314	0.998	1.002	0.956	1.239	0.842	1.290	1.671
	0.577	0.602	0.463	1.010	0.927	0.956	0.790	0.661	0.735	0.509	1.031	1.163
	0.728	0.652	0.609	0.863	0.979	0.878	0.989	0.845	0.841	0.647	1.096	1.377
	0.492	0.478	0.398	0.967	0.806	0.918	0.680	0.580	0.600	0.459	1.041	1.096
	0.442	0.395	0.295	0.822	0.806	0.961	0.521	0.462	0.436	0.514	1.042	0.984
	0.573	0.638	0.835	1.099	0.563	1.024	0.736	0.810	0.580	0.534	0.627	1.310
	0.531	0.493	0.643	0.880	0.717	0.898	0.686	0.775	0.552	0.523	0.776	1.589
	1.021	1.003	1.149	1.286	1.058	1.142	1.176	1.168	1.100	0.926	1.121	1.462
	0.440	0.412	0.463	0.899	0.637	0.803	0.663	0.593	0.789	0.591	0.938	1.209
	0.629	0.671	0.573	1.087	0.689	1.020	0.652	0.772	0.670	0.682	0.943	1.382
AVG	0.645	0.607	0.611	0.999	0.850	0.960	0.790	0.762	0.754	0.623	0.991	1.324

compare the performance of the enhanced image with state-of-the-art methods and the proposed method through UIQM [36] score. A higher score denotes a better-enhanced image. He et al. [1] obtained a higher UIQM score than Gao et al. [14], although He et al.’s [1] method contains no color-compensation procedure. Gao et al.’s [14] method obtained a lower UIQM score than Al Ameen’s [10] method, although the image enhanced using Al Ameen’s [10] method has a casted color. Meng et al. [2] obtained a lower UIQM score than Al Ameen [10] because Meng et al.’s [2] method has no color-compensation procedure. Shi et al. [15] obtained a higher UIQM score than Meng et al. [2] because Shi et al. [15] used a color-compensation procedure. Ren et al. [17] obtained a lower UIQM score than Shi et al. [15] because Ren et al.’s [17] method has no color-compensation procedure. Shi et al. [11] obtained a lower UIQM score than Ren et al. [17], although Shi et al.’s [11] method contains a color-compensation procedure. Santra et al. [20] obtained a higher UIQM score than Shi et al. [11] in some images, although Santra et al.’s [20] method contains no color-compensation procedure, and the enhanced image has a casted color. Hong et al. [9] obtained a lower UIQM score than Shi et al. [11] because Shi et al.’s [11] method contains a color-compensation procedure. Zhao et al. [5] obtained a higher UIQM score than Yu et al. [21] and Gao et al. [14], although Gao et al.’s [14] method contains a color-compensation procedure. Through the UIQM score, though, if the image has a casted color because the UIQM score is higher than that of others, the UIQM is not an absolute but a referenceable measure. The image enhanced by the proposed method has a higher UIQM score than other methods.

Table 6. The comparison of enhanced image through averaged NIQE [35] metric with state-of-the-art methods in Figure 9 and Figure 10 and DAWN dataset [30] (a lower score denotes a better-enhanced image; PM is proposed method).

	[1]	[9]	[14]	[10]	[2]	[15]	[17]	[11]	[20]	[21]	[5]	PM
AVG (20)	19.978	19.975	20.011	19.943	19.869	19.843	20.056	19.799	20.035	20.061	19.743	18.429
AVG (323)	19.946	20.026	20.083	19.981	19.791	19.802	20.024	19.720	20.017	20.055	19.709	18.090

and

Table 7. The comparison of enhanced image through averaged UIQM [36] metric with state-of-the-art methods in Figure 9 and Figure 10 and DAWN dataset [30] (a higher score denotes a better-enhanced image; PM is proposed method).

	[1]	[9]	[14]	[10]	[2]	[15]	[17]	[11]	[20]	[21]	[5]	PM
AVG (20)	0.671	0.614	0.585	0.972	0.870	0.925	0.789	0.767	0.766	0.655	1.033	1.326
AVG (323)	0.812	0.720	0.695	0.990	0.991	0.926	0.895	0.856	0.874	0.730	1.120	1.410

show a comparison of the enhanced image with state-of-the-art methods and the proposed method through averaged NIQE [35] and UIQM [36] scores on Figure 9 and Figure 10 and the DAWN dataset [30]. Table 6 shows the averaged NIQE score of Figure 9 and Figure 10 and the DAWN dataset [30]. The existing dehazing method contains no color-compensation procedure; however, sometimes, the NIQE score is lower than that of the non-color-casted image on the color-casted image. Moreover, although the enhanced image contains a color shift, the UIQM score of the enhanced image with casted color is higher than that of the non-color-casted image. Therefore, the NIQE and UIQM metrics are not absolute but referenceable measures. The NIQE score of the proposed method was lower than those obtained for other methods, and the UIQM score of the proposed method was higher than those obtained for other methods.

4. Conclusions

The sandstorm image has an asymmetrically casted color, such as yellowish or reddish, due to the color-channel attenuation caused by sand particles. If the color-casted components are not considered when enhancing the sandstorm image, then the enhanced image has an artificial color. Therefore, this work balanced the image using a saturation-based color-correction algorithm on an asymmetrically casted color. The balanced image contains no color veil and seems hazy. Moreover, as the distribution of the haze ingredients is asymmetrical, a dehazing procedure was needed to enhance the hazy image; therefore, this work obtained a transmission map with hybrid theories, such as dark-channel prior and bright-channel prior, based on CNN. The enhanced image has no artificial effect or naturalism. The contribution of this work is that the proposed color-correction algorithm is based on saturation using the average value difference of color channels in sandstorm images with various color casts. Moreover, this method can easily and widely compensate images, even when the color channel is too rare due to great attenuation, and by using the hybrid transmission map, the proposed algorithm is naturally enhanced, although the image has regions that are too bright or dark. The next aim of this work is to enhance the image naturally, pursuing image-adaptive measures to balance the color and estimate the transmission map in low-light circumstances and a thick, hazy environment.