Skip to main content
SpringerLink
Log in

Infrared-visible Image Fusion Using Accelerated Convergent Convolutional Dictionary Learning

  • Research Article-Computer Engineering and Computer Science
  • Published:
Arabian Journal for Science and Engineering Aims and scope Submit manuscript

Abstract

Techniques for the fusion of infrared and visible images have gradually become a popular research topic in the field of computer vision. In our paper, accelerated convergent convolutional dictionary learning (CDL) is first introduced for infrared-visible image fusion. The proposed method combines the advantages of CDL and convolutional sparse representation (CSR) while also compensating for model mismatches between the training and fusion stages. Each image is decomposed into a base layer and a detail layer, for which different fusion strategies are used. Unlike previous CSR/CDL-based fusion methods, we introduce a practical and convergent Fast Block Proximal Gradient Using a Diagonal Majorizer (FBPG-M) method with two-block and multiblock schemes into the detail layer. Influenced by various imaging mechanisms, an ‘averaging’ fusion strategy is used for the base layer. Our method is evaluated and compared qualitatively and quantitatively with five typical fusion methods on 10 public datasets. The model is both subjectively and objectively evaluated, and the results show that the proposed method achieves notable success in terms of preserving details and focusing on targets.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Notes

  1. https://figshare.com/articles/TN_Image_Fusion_Dataset/1008029.

  2. https://www.researchgate.net/publication/304246314.

References

  1. Wang, Zhishe; Xu, Jiawei; Jiang,: Xiaolin and Yan, Xiaomei.: Infrared and visible image fusion via hybrid decomposition of NSCT and morphological sequential toggle operator. Optik International Journal for Light Electron Optics 201,(2019)

  2. Duan, Chaowei; Wang, Zhisheng; Xing, Changda; Lu, Shanshan: Infrared and visible image fusion using multi-scale edge-preserving decomposition and multiple saliency features. Optik 228,(2021)

  3. Chen, Jun; Li, Xuejiao; Luo, Linbo; Mei, Xiaoguang; Ma, Jiayi: Infrared and visible image fusion based on target-enhanced multiscale transform decomposition. Information Sciences 508, 64–78 (2020)

    Article  Google Scholar 

  4. Ma, Jiayi; Chen, Chen; Li, Chang; Huang, Jun: Infrared and visible image fusion via gradient transfer and total variation minimization. Information Fusion 31, 100–109 (2016)

    Article  Google Scholar 

  5. Dai, Liyang; Liu, Gang; Huang, Lei; Xiao, Gang; Xu, Zhao; Ruan, Junjin: Feature transfer method for infrared and visible image fusion via fuzzy lifting scheme. Infrared Physics & Technology (2021)

  6. Qilei Li,Wei Wu,Lu Lu,Zuoyong Li,Awais Ahmad ,Gwanggil Jeon.: Infrared and visible images fusion by using sparse representation and guided filter, Journal of Intelligent Transportation Systems, 254-263, (2020)

  7. Yin, Haitao; Li, Shutao: Multimodal image fusion with joint sparsity model. Optical Engineering 50,(2011)

  8. Visual attention guided image fusion with sparse representation: Yang, Bin and Li, Shutao. Optik - International Journal for Light and Electron Optics 125, 4881–4888 (2014)

  9. Gao, Zhisheng; Zhang, Chengfang: Texture clear multi-modal image fusion with joint sparsity model. Optik - International Journal for Light and Electron Optics 121305, S0030402616310725 (2016)

    Google Scholar 

  10. Zhang, C.F.; Yi, L.Z.: Multimodal image fusion with adaptive joint sparsity model. J. Electron. Imaging 28, 013043 (2019)

    Google Scholar 

  11. Chengfang Zhang, Ziliang Feng, Zhisheng Gao, Xin Jin, Dan Yan, Liangzhong Yi.: Salient feature multimodal image fusion with a joint sparse model and multiscale dictionary learning, Opt. Eng, vol. 59, pp.051402,(2020)

  12. Liu, Y.; Wang, Z.: Simultaneous image fusion and denoising with adaptive sparse representation. IET Image Processing 9(5), 347–357 (2015)

    Article  Google Scholar 

  13. Wohlberg, B.: Efficient algorithms for convolutional sparse representations. IEEE Trans. Image Process 25, 301–315 (2016)

    Article  MathSciNet  Google Scholar 

  14. Garcia-Cardona, Cristina; Wohlberg, Brendt: Convolutional Dictionary Learning: A Comparative Review and New Algorithms, IEEE Transactions on Computational. Imaging 4(3), 366–381 (2018)

    MathSciNet  Google Scholar 

  15. Il and Yong, Jeffrey A and Fessler.: Convolutional Dictionary Learning: Acceleration and Convergence. IEEE Transactions on Image Processing 27(4), 1697–1712 (2018)

  16. Chun, Il Yong; Fessler, Jeffrey A.: Convergent convolutional dictionary learning using Adaptive Contrast Enhancement (CDL-ACE): Application of CDL to image denoising,2017 International Conference on Sampling Theory and Applications (SampTA), (2017)

  17. Liu, Y.; Chen, X.; Ward, R.K.: Image fusion with convolutional sparse representation. IEEE Signal Process. Lett.vol. 23, 1882–1886 (2016)

    Article  Google Scholar 

  18. Wu, Minghui; Ma, Yong; Fan, Fan; Mei, Xiaoguang; Huang, Jun: Infrared and visible image fusion via joint convolutional sparse representation. Journal of the Optical Society of America A.vol. 37(7), 1105–1115 (2020)

    Article  Google Scholar 

  19. Chengrui, Gao; Liu, Feiqiang; Hua, Yan: Infrared and visible image fusion using dual-tree complex wavelet transform and convolutional sparse representation. Journal of Intelligent & Fuzzy Systems 39(3), 4617–4629 (2020)

    Article  Google Scholar 

  20. Shao, Luling; Wu, Jin; Wu, Minghui: Infrared and Visible Image Fusion Based on Spatial Convolution Sparse representation. Journal of Physics: Conference Series 1634,(2020)

  21. Jian, W.; Chunxia, Q.; Xiufei, Z.; Ke, Y.; Ping, R.: A multi-source image fusion algorithm based on gradient regularized convolution sparse representation. Journal of Systems Engineering and Electronics 31(3), 447–459 (2020)

    Article  Google Scholar 

  22. Li, by Linguo., Tan, Ling., Li, Shujing., Ye, Qing.: Image fusion based on convolution sparse representation and pulse coupled neural network in non-subsampled contourlet domain. International Journal of Embedded Systems 12(1), 447–459 (2020)

  23. Zhang, Chengfang.: Convolutional dictionary learning using global matching tracking (CDL-GMT) Application to visible-infrared image fusion,4th International Conference on Mechatronics and Intelligent Robotics,(2021)

  24. Li, H.; Wu, X.J.; Kittler, J.: Infrared and Visible Image Fusion using a Deep Learning Framework,24rd International Conference on. IEEE,pp.2705–2710,(2018)

  25. Liu, Yu and Chen, Xun and Cheng, Juan and Peng, Hu and Wang, Zengfu.: Infrared and visible image fusion with convolutional neural networks,International Journal of Wavelets, Multiresolution and Information Processing,(2018)

  26. Ma, Jiayi., Yu, Wei., Liang, Pengwei., Li, Chang., and Jiang, Junjun.: FusionGAN: A generative adversarial network for infrared and visible image fusion,Information Fusion,48,11-26, (2019)

  27. Xydeas, C.S.; P. V.V.: Objective image fusion performance measure. Military Technical. Courier 56(4), 181–193 (2000)

  28. Piella, Gemma; Heijmans, H.: A new quality metric for image fusion. International Conference on Image Processing IEEE, (2003)

  29. Zhao, Jiying., Laganiere, R., and Liu, Z.: Performance assessment of combinative pixel-level image fusion based on an absolute feature measurement. International Journal of Innovative Computing Information Control Ijicic 3.6(2006)

  30. Chen, Y.; Blum, R.S.: A New Automated Quality Assessment Algorithm for Image Fusion. Image and Vision Computing 27, 1421–1432 (2009)

    Article  Google Scholar 

  31. Liu, Z.; et al.: Objective assessment of multiresolution image fusion algorithms for context enhancement in night vision: a comparative study. IEEE Trans. Pattern Anal. Machine Intell. 34(1), 94–109 (2011)

    Article  Google Scholar 

  32. Heide, F.; Heidrich, W.; Wetzstein, G.: Fast and flexible convolutional sparse coding, in Proc, pp. 5135–5143. IEEE CVPR, Boston, MA (2015)

    Google Scholar 

  33. Zeiler, M.D.; Krishnan, D.; Taylor, G.W.; Fergus, R.: Deconvolutional networks, in Proc, pp. 2528–2535. IEEE CVPR, San Francisco, CA (2010)

    Google Scholar 

  34. http://www.escience.cn/people/jiayima/paper.dhome

  35. https://github.com/hli1221

  36. https://sites.google.com/site/yuliu316316

  37. Li, S.; Kang, X.; Hu, J.: Image Fusion with Guided Filtering. IEEE Transactions on Image Processing 22(7), 2864–2875 (2013)

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported by Sichuan Science and Technology Program (Grants 2020YFS0351).

Funding

This study was funded by the Sichuan Science and Technology Program (Grant 2020YFS0351).

Author information

Authors and Affiliations

  1. National Key Laboratory of Fundamental Science on Synthetic Vision, College of Computer Science, Sichuan University, Chengdu, 610039, China

    Chengfang Zhang

  2. The Center of Laboratory and Equipment, Sichuan Police College, Luzhou, 646000, China

    Chengfang Zhang & Ziliang Feng

Authors
  1. Chengfang Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ziliang Feng
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Chengfang Zhang.

Ethics declarations

Conflicts of interest

The authors declare that they have no conflict of interest.

Ethical approval

This article does not contain any study performed on humans or animals by the authors.

Informed consent

Informed consent was obtained from all individual participants included in the study.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, C., Feng, Z. Infrared-visible Image Fusion Using Accelerated Convergent Convolutional Dictionary Learning. Arab J Sci Eng 47, 10295–10306 (2022). https://doi.org/10.1007/s13369-021-06380-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13369-021-06380-2

Keywords

Search

Navigation