Abstract
Although many techniques are devised in rain removal research recently, using either the Convolutional Neural Network (CNN) [1] or the Generative Adversarial Network (GAN) [2], they always require both rain and no-rain training images exist in pairs to train their networks. Recently, the Cycle-Consistent Adversarial Networks (CycleGAN) [3] has demonstrated successful results in removing real rain distortion, without the no-rain training images [4]. Building on this success, we propose a new technique called the Wavelet-CycleGANs (W-CycleGANs) that has the same advantage. In addition, the wavelet properties can be used for the CycleGAN to remove rain from images at their frequency sub-bands, in the Hue, Saturation and Value (HSV) color space [5]. We train and compare the W-CycleGANs’ to the CycleGAN’s performance fairly, using the same set of rain images as the CycleGAN [4]. Their quantitative results are compared using the Natural Image Quality Evaluator (NIQE) [6] as real rain images in pairs simply do not exist in the real world. In addition, their qualitative results are compared using visual check at zoomed-in regions. Both results have demonstrated the W-CycleGANs’ superiority in removing real rain distortions.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (1998)
Goodfellow, I., Pouget-Abadie, J., Mirza, M. et al.: Generative adversarial nets. Adv. Neural Inf. Process. Syst., 27 (2014)
Zhu, J.-Y., Park, T., Isola, P., Efros, A. A.: Unpaired image-to-image translation using cycle-consistent adversarial networks. 2017 IEEE International Conference on Computer Vision (ICCV), pp. 2242–2251 (2017)
Tang, L.M., Lim, L.H., Siebert, P.: Removal of visual disruption caused by rain using cycle-consistent generative adversarial networks. In: Leal-Taixé, L., Roth, S. (eds.) Computer Vision—ECCV 2018 Workshops. ECCV 2018. Lecture Notes in Computer Science 11133 (2019)
Pascale, D.: A Review of RGB color spaces (2003)
Mittal, A., Soundararajan, R., Bovik, A.C.: Making a completely blind image quality analyzer. IEEE Signal Process. Lett. 22(3), 209–212 (2013)
Garg, K., Nayar, S.K.: Vision and rain. Int. J. Comput. Vision 75, 3–27 (2007)
Li, S., Ren, W., Zhang, J., Yu, J., Guo, X.: Single image rain removal via a deep decomposition-composition network. Comput. Vis. Image Underst. 186, 48–57 (2019)
Li, Y., Tan, R.T., Guo, X., Lu, J., Brown, M.S.: Rain streak removal using layer priors. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2736–2744 (2016)
Xu., Jing, Zhao, W., Liu, P., Tang, X.: An improved guidance image based method to remove rain and snow in a single image. Comput. Inf. Sci. 5, 49–55 (2012)
Zhang, H., Sindagi, V., Patel, V.M.: Image de-raining using a conditional generative adversarial network. IEEE Trans. Circuits Syst. Video Technol. 30(11), 3943–3956 (2020)
Wang, C., et al.: Perceptual adversarial networks for image-to-image transformation. IEEE Trans. Image Process. 27(8), 4066–4079 (2018)
Saravanan, G., Yamuna, G., Vivek, R.: A color image enhancement based on discrete wavelet transform. IJCA Proceedings on National Conference on Emerging Trends in Information and Communication Technology (2013)
Guo, T., Mousavi, H.S., Vu, T.H., Monga, V.: Deep wavelet prediction for image super-resolution. IEEE Conference on Computer Vision and Pattern Recognition Workshops (2017)
Fu, X., Huang, J., Ding, X., Liao, Y., Paisley, J.: Clearing the skies: a deep network architecture for single-image rain removal. IEEE Trans. Image Process. 26, 2944–2956 (2016)
Yang, W., Tan, R.T., Feng, J., Liu, J., Guo, Z., Yan, S.: Deep joint rain detection and removal from a single image. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1685–1694 (2017)
Ledig, C., Theis, L., Huszar, F. et al.: Photo-realistic single image super-resolution using a generative adversarial network. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 105–114 (2016)
Pathak, D., Krahenbuhl, P., Donahue, J., Darrell, T., Efros, A.A.: Context encoders: feature learning by inpainting. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2536–2544 (2016)
Radford, A., Metz, L., Chintala, S.: Unsupervised representation learning with deep convolutional generative adversarial networks (2015). CORR abs/1511.06434
Isola, P., Zhu, J.-Y., Zhou, T., Efros, A.A.: Image-to-image translation with conditional adversarial networks. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5967–5976 (2017)
Wei, Y., Zhang, Z., Wang, Y., Xu, M., Yang, Y., Yan, S., Wang, M.: DerainCycleGAN: rain attentive CycleGAN for single image deraining and rainmaking. IEEE Trans. Image Process. 30, 4788–4801 (2021)
Qi, Y., Zhang, H., Jin, Z., Liu, W.: Depth-guided asymmetric CycleGAN for rain synthesis and image deraining. Kluwer Academic Publishers 81(25), 35935–35952 (2022)
Gupta, V., Mahle, R., Shriwas, R.S.: Image denoising using wavelet transform method. 2013 Tenth International Conference on Wireless and Optical Communications Networks (WOCN), pp. 1–4 (2013)
Song, J., Jeong, J., Park, D., Kim, H., Seo, D., Ye, J.: Unsupervised Denoising for Satellite Imagery using Wavelet Subband CycleGAN (2020). ArXiv abs/2002.09847
Liu, P., Zhang, H., Zhang, K., Lin, L., Zuo, W.: Multi-level wavelet-cnn for image restoration. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 773–782 (2018)
Yang, W., Liu, J., Yang, S., Guo, Z.: Scale-free single image deraining via visibility enhanced recurrent wavelet learning. IEEE Trans. Image Process. 28(6), 2948–2961 (2019)
Zhao, J., Xie, J., Xiong, R., Ma, S., Huang, T., Gao, W.: Pyramid Convolutional Network for Single Image Deraining. CVPR Workshops (2019)
Vidakovic, B., Müller, P.: An Introduction to Wavelets. Lecture Notes in Statistics, Springer, New York 141, 1–18 (1999)
Paszke, A., Gross, S., Chintala, S., Chanan, G.: PyTorch. GitHub repository (2021)
Alessandro CAPRA, Alfio CASTORINA and Silvia CORCHS: Dynamic Range Optimization by Local Contrast Correction and Histogram Image Analysis. 2006 Digest of Technical Papers International Conference on Consumer Electronics, pp. 309–310 (2006)
Padma Priya, G., Venkateswarlu, T.: An optimal gamma correction based image contrast enhancement using DWT-SVD. Int. J. Res. Advent Technol. (IJRAT) 6(8), 2184–2190 (2018)
Bull, D.R., Zhang, F.: Intelligent Image and Video Compression (Second Edition). Elsevier Science Publishing Co Inc, pp. 107–142 (2021)
Farid, H.: Blind inverse gamma correction. IEEE Trans. Image Process. 10(10), 1428–1433 (2001)
Chiu, Y.-S., Cheng, F.-C., Huang, S.-C.: Efficient contrast enhancement using adaptive gamma correction and cumulative intensity distribution. 2011 IEEE International Conference on Systems, Man, and Cybernetics, Anchorage, pp. 2946–2950 (2011)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Tang, L.L., Lim, L.H.I., Siebert, P. (2023). Image Reconstruction for Rain Removal in Both Wavelet and Spatial Frequency Sub-bands Using W-CycleGANs. In: Carbone, G., Laribi, M.A., Jiang, Z. (eds) Advances in Automation, Mechanical and Design Engineering. SAMDE 2022. Mechanisms and Machine Science, vol 138. Springer, Cham. https://doi.org/10.1007/978-3-031-40070-4_14
Download citation
DOI: https://doi.org/10.1007/978-3-031-40070-4_14
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-40069-8
Online ISBN: 978-3-031-40070-4
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)