Skip to main content
SpringerLink
Log in

Zero-reference single underwater image enhancement

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Underwater images play an essential role in acquiring and analyzing underwater information. Autonomous Underwater Vehicles (AUVs) highly rely on the quality of the captured underwater images, in order to carry out several activities. Due to the poor lighting conditions and the limited capacity of the optical imaging device, captured underwater images usually contain severe color distortions and contrast reduction. To this end, most existing deep learning-based underwater image enhancement methods synthesize the pseudo ground-truth, or employ the in-air clear images as references to train the models. However, the synthesized or selected reference images are generally unsatisfying due to the lack of diversity and applicability. This paper presents a novel underwater image enhancement approach based on training an end-to-end underwater image enhancement network, without using any reference image. A novel encoder-decoder network structure and a set of non-reference loss functions are designed to measure the enhancement quality. The subjective and objective evaluations show that the proposed algorithm outperforms the state-of-the-art approaches.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

Data Availability

The data that support the findings of this study are available from the corresponding author upon reasonable request.

References

  1. Barbosa WV, Amaral HGB, Rocha TL, Nascimento ER (2018) Visual-quality-driven learning for underwater vision enhancement. 2018 25th IEEE International Conference on Image Processing (ICIP), Athens, Greece, pp 3933–3937. https://doi.org/10.1109/ICIP.2018.8451356

  2. Buchsbaum G (1980) A spatial processor model for object colour perception. J Franklin Inst 10(1):1–26

    Article  Google Scholar 

  3. Chen X et al (2019) Towards real-time advancement of underwater visual quality with GAN. IEEE Trans Ind Electron 66(12):9350–9359

    Article  Google Scholar 

  4. Drews PLJ et al (2016) Underwater depth estimation and image restoration based on single images. IEEE Comput Graph Appl 36(2):24–35

    Article  Google Scholar 

  5. Dudhane A et al (2020) Deep underwater image restoration and beyond. IEEE Signal Process Lett 27:675–679

    Article  Google Scholar 

  6. Ebner M (2007) Color constancy, Hoboken, NJ, USA:Wiley

  7. Fabbri C, Islam MJ, Sattar J (2018) Enhancing underwater imagery using generative adversarial networks. 2018 IEEE International Conference on Robotics and Automation (ICRA), Brisbane, QLD, Australia, pp 7159–7165. https://doi.org/10.1109/ICRA.2018.8460552

  8. Fan Q, Chen D, Yuan L, Hua G, Yu N, Chen B (2018) Decouple learning for parameterized image operators. In: Ferrari V, Hebert M, Sminchisescu C, Weiss Y (eds) Computer Vision – ECCV vol 11217. Springer, Cham. https://doi.org/10.1007/978-3-030-01261-8_27

  9. Fu X, Zhuang P, Huang Y, Liao Y, Zhang X, Ding X (2014) A retinex-based enhancing approach for single underwater image. IEEE Int Conf Image Process (ICIP) 2014:4572–4576. https://doi.org/10.1109/ICIP.2014.7025927

    Article  Google Scholar 

  10. Galdran A et al (2015) Automatic red-channel underwater image restoration. J Vis Commun Image Represent 26:132–145

    Article  Google Scholar 

  11. Gulrajani I, Ahmed F, Arjovsky M, Dumoulin V, Courville AC (2017) Improved training of Wasserstein GANs. Proc. NIPS pp 5767–5777

  12. Hamaguchi R, Fujita A, Nemoto K, Imaizumi T, Hikosaka S (2018) Effective use of dilated convolutions for segmenting small object instances in remote sensing imagery. IEEE Winter Conf Appl Comput Vis (WACV) 2018:1442–1450. https://doi.org/10.1109/WACV.2018.00162

    Article  Google Scholar 

  13. Hamza AB, Krim H (2001) A variational approach to maximum a posteriori estimation for image denoising[C]// Energy Minimization Methods in Computer Vision and Pattern Recognition, Third International Workshop, EMMCVPR 2001, Sophia Antipolis, France, September 3–5, 2001, Proceedings. Springer-Verlag

  14. Hashisho Y et al (2019) Underwater color restoration using u-net denoising autoencoder. 2019 11th International Symposium on Image and Signal Processing and Analysis (ISPA). IEEE

  15. He K, Sun J, Tang X (2010) Single image haze removal using dark channel prior. IEEE Trans Pattern Anal Mach Intell 33(12):2341–2353

    Google Scholar 

  16. Hummel R (1977) Image enhancement by histogram transformation. Comput Graph Image Process 6(2):184–195

  17. Islam Md J, Luo P, Sattar J (2020) Simultaneous enhancement and super-resolution of underwater imagery for improved visual perception."arXiv e-prints: arXiv-2002

  18. Islam MdJ, Xia Y, Sattar J (2020) Fast underwater image enhancement for improved visual perception. IEEE Robot Autom Lett 5(2):3227–3234

    Article  Google Scholar 

  19. Johnson J, Alahi A, Fei-Fei L (2016) Perceptual losses for real-time style transfer and super-resolution. In Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, Proceedings, Part II 14 (pp 694–711). Springer International Publishing

  20. Li X, Wu J, Lin Z, Liu H, Zha H (2018) Recurrent squeeze-and-excitation context aggregation net for single image deraining. In Proceedings of the European conference on computer vision (ECCV), pp 254–269

  21. Li J et al (2017) WaterGAN: Unsupervised generative network to enable real-time color correction of monocular underwater images. IEEE Robot Autom Lett 3(1):387–394

    MathSciNet  Google Scholar 

  22. Li C et al (2019) An underwater image enhancement benchmark dataset and beyond. IEEE Trans Image Process 29:4376–4389

    Article  MATH  Google Scholar 

  23. Li C, Guo J, Guo C (2018) Emerging from water: Underwater image color correction based on weakly supervised color transfer. IEEE Signal Process Lett 25(3):323–327

    Article  Google Scholar 

  24. Liu Y-C, Chan W-H, Chen Y-Q (1995) Automatic white balance for digital still camera. IEEE Trans Consum Electron 41(3):460–466

    Article  Google Scholar 

  25. Ludvigsen M et al (2007) Applications of geo-referenced underwater photo mosaics in marine biology and archaeology. Oceanography 20(4):140–149

    Article  Google Scholar 

  26. Nascimento E, Campos M, Barros W (2009) Stereo based structure recovery of underwater scenes from automatically restored images. 2009 XXII Brazilian Symposium on Computer Graphics and Image Processing. IEEE

  27. Panetta K, Gao C, Agaian S (2015) Human-visual-system-inspired underwater image quality measures. IEEE J Oceanic Eng 41(3):541–551

    Article  Google Scholar 

  28. Peng Y-T, Cosman PC (2017) Underwater image restoration based on image blurriness and light absorption. IEEE Trans Image Process 26(4):1579–1594

    Article  MathSciNet  MATH  Google Scholar 

  29. Reza AM (2004) Realization of the contrast limited adaptive histogram equalization (CLAHE) for real-time image enhancement. J VLSI Signal Process Syst Signal Image Video Technol 38(1):35–44

    Article  Google Scholar 

  30. Strachan NJC (1993) Recognition of fish species by colour and shape. Image Vis Comput 11(1):2–10

    Article  MathSciNet  Google Scholar 

  31. Wang Y, Zhang J, Cao Y, Wang Z (2017) A deep CNN method for underwater image enhancement. In 2017 IEEE international conference on image processing (ICIP), pp 1382–1386. IEEE

  32. Wang N et al (2019) UWGAN: underwater GAN for real-world underwater color restoration and dehazing. arXiv preprint arXiv:1912.10269

  33. Wang P, Chen P, Yuan Y, Liu D, Huang Z, Hou X, Cottrell G (2018) Understanding convolution for semantic segmentation. In 2018 IEEE winter conference on applications of computer vision (WACV), pp 1451–1460. IEEE

  34. Yang M, Sowmya A (2015) An underwater color image quality evaluation metric. IEEE Trans Image Process 24(12):6062–6071

    Article  MathSciNet  MATH  Google Scholar 

  35. Yu X, Qu Y, Hong M (2018) Underwater-GAN: Underwater image restoration via conditional generative adversarial network. International Conference on Pattern Recognition. Springer, Cham

  36. Zhang S, Zhang J, Fang S, Cao Y (2014) Underwater stereo image enhancement using a new physical model. In 2014 IEEE International Conference on Image Processing (ICIP), pp 5422–5426. IEEE

  37. Zhang S et al (2017) Underwater image enhancement via extended multi-scale Retinex. Neurocomputing 245:1–9

    Article  Google Scholar 

  38. Zhao H et al (2015) Loss functions for neural networks for image processing." arXiv preprint arXiv:1511.08861

  39. Zhu JY, Park T, Isola P, Efros AA (2017) Unpaired image-to-image translation using cycle-consistent adversarial networks. In Proceedings of the IEEE international conference on computer vision, pp 2223–2232

Download references

Acknowledgements

This work was supported by the National Key Research and Development Program of China (Grant No.2022ZD0160400) and the National Natural Science Foundation of China (Grant Nos. 62071323 and 62176178). We gratefully acknowledge the support from Shanghai Artificial Intelligence Laboratory.

Funding

National Key Research and Development Program of China; Award No. 2022ZD0160400. National Natural Science Foundation of China; Award Nos. 62071323, 62176178.

Author information

Authors and Affiliations

  1. School of Electrical and Information Engineering, Tianjin University, Tianjin, China

    Aiping Yang, Chaochen Wang, Jinbin Wang, Qian Wang & Tengfei Zhang

  2. Shanghai Artificial Intelligence Laboratory, Shanghai, China

    Aiping Yang

Authors
  1. Aiping Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Chaochen Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jinbin Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Qian Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Tengfei Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Aiping Yang.

Ethics declarations

Conflicts of interest

The authors declare no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yang, A., Wang, C., Wang, J. et al. Zero-reference single underwater image enhancement. Multimed Tools Appl 82, 46423–46438 (2023). https://doi.org/10.1007/s11042-023-15695-1

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-023-15695-1

Keywords

Search

Navigation