Skip to main content
SpringerLink
Log in

Flow Learning Based Dual Networks for Low-Light Image Enhancement

  • Published:
Neural Processing Letters Aims and scope Submit manuscript

Abstract

The deep learning-based low-light image enhancement task aims to learn a mapping that converts low-light images to normally exposed images by training with paired or unpaired datasets. Most of these existing methods are based on convolutional neural networks, which largely limits the network’s ability to learn global information. Meanwhile, the reconstruction loss or adversarial loss they adopt often cannot accurately measure the visual distance between the prediction and the target, resulting in blurred regions in the enhancement results. In this paper, we propose a novel flow learning based dual networks (FDN), which consists of a dual network and a flow learning based model. The dual network is mainly composed of a reidual-based Unet encoder and a residual-based Swin Transformer encoder, which can make up for the lack of global information processing and has more advantages in processing deep and shallow information. Moreover, we use a single loss function named negative log-likelihood to train the entire network, which enables the flow models to adequately learn the complex conditional distribution of normally exposed images and avoid blurry outputs. Experimental results on two benchmark datasets show that the proposed FDN method can achieve the sate-of-the-art performances on low-light image enhancement task.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Li C, Guo C, Han L-H, Jiang J, Cheng M-M, Gu J, Loy CC (2021) Low-light image and video enhancement using deep learning: a survey. IEEE Trans Pattern Anal Mach Intell 44:9396–9416

    Article  Google Scholar 

  2. Liu J, Xu D, Yang W, Fan M, Huang H (2021) Benchmarking low-light image enhancement and beyond. Int J Comput Vis 129(4):1153–1184

    Article  Google Scholar 

  3. Wang H, Peng J, Chen D, Jiang G, Zhao T, Fu X (2020) Attribute-guided feature learning network for vehicle reidentification. IEEE Multimedia 27(4):112–121

    Article  Google Scholar 

  4. Ibrahim H, Kong NSP (2007) Brightness preserving dynamic histogram equalization for image contrast enhancement. IEEE Trans Consum Electron 53(4):1752–1758

    Article  Google Scholar 

  5. Abdullah-Al-Wadud M, Kabir MH, Dewan MAA, Chae O (2007) A dynamic histogram equalization for image contrast enhancement. In: 2007 digest of technical papers international conference on consumer electronics, pp 1–2. https://doi.org/10.1109/ICCE.2007.341567

  6. Wang S, Zheng J, Hu H-M, Li B (2013) Naturalness preserved enhancement algorithm for non-uniform illumination images. IEEE Trans Image Process 22(9):3538–3548

    Article  Google Scholar 

  7. Guo X, Li Y, Ling H (2016) Lime: low-light image enhancement via illumination map estimation. IEEE Trans Image Process 26(2):982–993

    Article  MathSciNet  MATH  Google Scholar 

  8. Lore KG, Akintayo A, Sarkar S (2017) LLNet: a deep autoencoder approach to natural low-light image enhancement. Pattern Recognit 61:650–662

    Article  Google Scholar 

  9. Wei C, Wang W, Yang W, Liu J (2018) Deep Retinex decomposition for low-light enhancement. arXiv preprint arXiv:1808.04560

  10. Zhang Y, Zhang J, Guo X (2019) Kindling the darkness: a practical low-light image enhancer. In: Proceedings of the 27th ACM international conference on multimedia, pp 1632–1640

  11. Triantafyllidou D, Moran S, McDonagh S, Parisot S, Slabaugh G (2020) Low light video enhancement using synthetic data produced with an intermediate domain mapping. In: European conference on computer vision. Springer, pp 103–119

  12. Yang W, Wang S, Fang Y, Wang Y, Liu J (2020) From fidelity to perceptual quality: A semi-supervised approach for low-light image enhancement. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3063–3072

  13. Jiang Y, Gong X, Liu D, Cheng Y, Fang C, Shen X, Yang J, Zhou P, Wang Z (2021) EnlightenGAN: deep light enhancement without paired supervision. IEEE Trans Image Process 30:2340–2349

    Article  Google Scholar 

  14. Guo C, Li C, Guo J, Loy CC, Hou J, Kwong S, Cong R (2020) Zero-reference deep curve estimation for low-light image enhancement. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 1780–1789

  15. Jaini P, Kobyzev I, Yu Y, Brubaker M (2020) Tails of Lipschitz triangular flows. In: International conference on machine learning. PMLR, pp 4673–4681

  16. Dinh L, Krueger D, Bengio Y (2014) Nice: non-linear independent components estimation. arXiv preprint arXiv:1410.8516

  17. Dinh L, Sohl-Dickstein J, Bengio S (2016) Density estimation using real NVP. arXiv preprint arXiv:1605.08803

  18. Kingma DP, Dhariwal P (2018) Glow: generative flow with invertible 1x1 convolutions. Adv Neural Inf Process Syst 31

  19. Lugmayr A, Danelljan M, Gool LV, Timofte R (2020) Srflow: learning the super-resolution space with normalizing flow. In: European conference on computer vision. Springer, pp 715–732

  20. Ronneberger O, Fischer P, Brox T (2015) U-net: convolutional networks for biomedical image segmentation. In: International conference on medical image computing and computer-assisted intervention. Springer, pp 234–241

  21. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778

  22. Wang H, Yao M, Jiang G, Mi Z, Fu X (2023) Graph-collaborated auto-encoder hashing for multi-view binary clustering. arXiv preprint arXiv:2301.02484

  23. Wang H, Jiang G, Peng J, Deng R, Fu X (2022) Towards adaptive consensus graph: multi-view clustering via graph collaboration. IEEE Trans Multimedia

  24. Wang H, Wang Y, Zhang Z, Fu X, Zhuo L, Xu M, Wang M (2020) Kernelized multiview subspace analysis by self-weighted learning. IEEE Trans Multimedia 23:3828–3840

    Article  Google Scholar 

  25. Liu Z, Lin Y, Cao Y, Hu H, Wei Y, Zhang Z, Lin S, Guo B (2021) Swin transformer: hierarchical vision transformer using shifted windows. In: Proceedings of the IEEE/CVF international conference on computer vision, pp 10012–10022

  26. Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser Ł, Polosukhin I (2017) Attention is all you need. Adv Neural Inf Process Syst 30

  27. Wang Y, Wan R, Yang W, Li H, Chau L-P, Kot A (2022) Low-light image enhancement with normalizing flow. In: Proceedings of the AAAI conference on artificial intelligence, vol 36, pp 2604–2612

  28. Zamir SW, Arora A, Khan S, Hayat M, Khan FS, Yang M-H, Shao L (2020) Learning enriched features for real image restoration and enhancement. In: European conference on computer vision. Springer, pp 492–511

  29. Hu J, Shen L, Sun G (2018) Squeeze-and-excitation networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 7132–7141

  30. Woo S, Park J, Lee J-Y, Kweon, IS (2018) CBAM: convolutional block attention module. In: Proceedings of the European conference on computer vision (ECCV), pp 3–19

  31. Ying Z, Li G, Ren Y, Wang R, Wang W (2017) A new low-light image enhancement algorithm using camera response model. In: The IEEE international conference on computer vision (ICCV)

  32. Hu C-H, Yu J, Wu F, Zhang Y, Jing X-Y, Lu X-B, Liu P (2021) Face illumination recovery for the deep learning feature under severe illumination variations. Pattern Recognit 111:107724

    Article  Google Scholar 

  33. Risheng L, Long M, Jiaao Z, Xin F, Zhongxuan L (2021) Retinex-inspired unrolling with cooperative prior architecture search for low-light image enhancement. In: Proceedings of the IEEE conference on computer vision and pattern recognition

  34. Zhang Y, Guo X, Ma J, Liu W, Zhang J (2021) Beyond brightening low-light images. Int J Comput Vis 129(4):1013–1037

    Article  Google Scholar 

  35. Xu X, Wang R, Fu C-W, Jia J (2022) SNR-aware low-light image enhancement. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 17714–17724

  36. Yang W, Wang W, Huang H, Wang S, Liu J (2021) Sparse gradient regularized deep Retinex network for robust low-light image enhancement. IEEE Trans Image Process 30:2072–2086

    Article  Google Scholar 

  37. Bychkovsky V, Paris S, Chan E, Durand F (2011) Learning photographic global tonal adjustment with a database of input/output image pairs. In: CVPR 2011. IEEE, pp 97–104

  38. Park J, Lee J-Y, Yoo D, Kweon IS (2018) Distort-and-recover: color enhancement using deep reinforcement learning. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 5928–5936

  39. Gharbi M, Chen J, Barron JT, Hasinoff SW, Durand F (2017) Deep bilateral learning for real-time image enhancement. ACM Trans Graph TOG 36(4):1–12

    Article  Google Scholar 

  40. Chen Y-S, Wang Y-C, Kao M-H, Chuang Y-Y (2018) Deep photo enhancer: unpaired learning for image enhancement from photographs with GANs. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 6306–6314

  41. Wang R, Zhang Q, Fu C-W, Shen X, Zheng W-S, Jia J (2019) Underexposed photo enhancement using deep illumination estimation. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 6849–6857

  42. Wang Z, Bovik AC, Sheikh HR, Simoncelli EP (2004) Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process 13(4):600–612

    Article  Google Scholar 

  43. Loh YP, Chan CS (2019) Getting to know low-light images with the exclusively dark dataset. Comput Vis Image Underst 178:30–42

    Article  Google Scholar 

Download references

Funding

This work was supported by the National Natural Science Foundation of China under Grant 62272240 and Grant 61802203, and in part by the Nanjing University of Posts and Telecommunications Science Foundation under Grant NY221081.

Author information

Authors and Affiliations

  1. College of Automation and College of Artificial Intelligence, Nanjing University of Posts and Telecommunications, No. 9 Wenyuan Road, Nanjing, 210023, Jiangsu, People’s Republic of China

    Siyu Wang, Changhui Hu, Weilin Yi, Ziyun Cai & Mingliang Zhai

  2. School of Automation, Southeast University, No. 2 Sipailou, Nanjing, 210018, Jiangsu, People’s Republic of China

    Wankou Yang

Authors
  1. Siyu Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Changhui Hu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Weilin Yi
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ziyun Cai
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Mingliang Zhai
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Wankou Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Changhui Hu.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, S., Hu, C., Yi, W. et al. Flow Learning Based Dual Networks for Low-Light Image Enhancement. Neural Process Lett 55, 8115–8130 (2023). https://doi.org/10.1007/s11063-023-11303-3

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11063-023-11303-3

Keywords

Search

Navigation