Skip to main content
SpringerLink
Log in

Remote Sensing Image Fusion Based on Multi-morphological Convolutional Neural Network

  • Conference paper
  • First Online:
Machine Learning for Cyber Security (ML4CS 2020)

Part of the book series: Lecture Notes in Computer Science ((LNSC,volume 12488))

Included in the following conference series:

  • 859 Accesses

Abstract

In this paper, we propose a method for remote sensing image fusion based on multi-morphological convolutional neural network (MCNN). First, MCNN combines a local discrete cosine transform (LDCT) dictionary and a curvelet transform (CT) dictionary to form a decomposition dictionary. By adjusting the size of the threshold, the morphological component analysis (MCA) method is used to sparsely decompose the remote sensing image, the texture component and the cartoon component (segmented smooth component) are extracted from the remote sensing image, respectively. Secondly, combined with the convolutional neural network (CNN) that has achieved good results in the field of image processing. By inputting different forms of source images into the network, it outputs a fused image with end-to-end characteristics. The MCNN not only solves the problems that most existing models based on sparse algorithms are relatively complex and have high computational complexity, but also solves the problem that in the traditional deep learning fusion method, the original information in the source image is often lost when acquiring the image information. Compared with other commonly used remote sensing image fusion methods, experimental results show that MCNN has achieved good results in terms of objective evaluation indicators and subjective visual performance.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Zhao, W., Jiao, L.C., et al.: Superpixel-based multiple local CNN for panchromatic and multispectral image classification. IEEE Trans. Geosci. Remote Sens. 55(7), 4141–4156 (2017)

    Article  Google Scholar 

  2. Han, J.W., Zhang, D.W., Cheng, G., Guo, L., Ren, J.C.: Object detection in optical remote sensing images based on weakly supervised learning and high-level feature learning. IEEE Trans. Geosci. Remote Sens. 53(6), 3325–3337 (2015)

    Article  Google Scholar 

  3. Vivone, G., Alparone, L., et al.: A critical comparison among pansharpening algorithms. IEEE Trans. Geosci. Remote Sens. 53(5), 2565–2586 (2015)

    Article  Google Scholar 

  4. Garzelli, A.: A review of image fusion algorithms based on the super resolution paradigm. IEEE Trans. Geosci. Remote Sens. 8(10), 797 (2016)

    Google Scholar 

  5. Ghassemian, H.: A review of remote sensing image fusion methods. Inf. Fusion 32(PA), 75–89 (2016)

    Google Scholar 

  6. Carper, W.J., Lillesand, T.M., Kiefer, P.W.: The use of intensity-hue-saturation transformations for merging SPOT panchromatic and multispectral image data. Photogram. Eng. Remote Sens. 56(4), 459–467 (1990)

    Google Scholar 

  7. Yang, Y., Wan, W.G., et al.: Remote sensing image fusion based on adaptive IHS and multi-scale guided filter. IEEE Access 4(1), 4573–4582 (2016)

    Article  Google Scholar 

  8. Pohl, C., Van Genderen, J.L.: Review article multisensor image fusion in remote sensing: concepts, methods and applications. Int. J. Remote Sens. 19(5), 823–854 (1998)

    Article  Google Scholar 

  9. Shahdoosti, H.R., Ghassemian, H.: Combining the spectral PCA and spatial PCA fusion methods by an optimal fifilter. Inf. Fusion 27(C), 150–160 (2016)

    Google Scholar 

  10. Zhou, H.Z., Wu, S., Mao, D.F., et al.: Improved Brovey method for multi-sensor image fusion. J. Remote Sens. 16(2), 343–360 (2012)

    Google Scholar 

  11. Thomas, C., Ranchin, T., Wald, L., Chanussot, J.: Synthesis of multispectral images to high spatial resolution: a critical review of fusion methods based on remote sensing physics. IEEE Trans. Geosci. Remote Sens. 46(5), 1301–1312 (2008)

    Article  Google Scholar 

  12. Pajares, G., Cruz, J.M.D.L.: A wavelet-based image fusion tutorial. Pattern Recogn. 37(9), 1855–1872 (2004)

    Article  Google Scholar 

  13. Pradhan, P.S., King, R.L., Younan, N.H., Holcomb, D.W.: Estimation of the number of decomposition levels for a wavelet-based multiresolution multisensor image fusion. IEEE Trans. Geosci. Remote Sens. 44(12), 3674–3686 (2006)

    Article  Google Scholar 

  14. Cunha, A.L.D., Zhou, J., Do, M.N.: The nonsubsampled contourlet transform: theory, design, and applications. IEEE Trans. Image Process. 15(10), 3089–3101 (2006)

    Article  Google Scholar 

  15. Yang, Y.T., Zhu, M., He, B.G., et al.: Fusion algorithm based on improved projected gradient NMF and NSCT. Optics Precis. Eng. 19(5), 1143–1150 (2011)

    Article  Google Scholar 

  16. Chavez, P.S., Sides, S.C., Anderson, J.A.: Comparison of three different methods to merge multiresolution and multispectral data: Landsat TM and SPOT panchromatic. Photogramm. Eng. Remote Sens. 57(3), 265–303 (1991)

    Google Scholar 

  17. Shao, Z.F., Liu, J., Cheng, Q.M.: Fusion of infrared and visible images based on focus measure operators in the curvelet domain. Appl. Opt. 51(12), 1910–1921 (2012)

    Article  Google Scholar 

  18. Lin, H., Tian, Y.F., Pu, R.L., Liang, L.: Remotely sensing image fusion based on wavelet transform and human vision system. Signal Process. Image Process. Pattern Recog. 8, 291–298 (2015)

    Google Scholar 

  19. Donoho, D.L.: Compressed sensing. IEEE Trans. Inf. Theory 52(4), 1289–1306 (2006)

    Article  MathSciNet  Google Scholar 

  20. Li, S.T., Yin, H.T., Fang, L.Y.: Remote sensing image fusion via sparse representations over learned dictionaries. IEEE Trans. Geosci. Remote Sens. 51(9), 4779–4789 (2013)

    Article  Google Scholar 

  21. Olshausen, B.A., Field, D.J.: Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature 381, 607–609 (1996)

    Article  Google Scholar 

  22. Li, S., Yang, B.: A new pan-sharpening method using a compressed sensing technique. IEEE Trans. Geosci. Remote Sens. 49(2), 738–746 (2011)

    Article  Google Scholar 

  23. Zhu, X.X., Bamler, R.: A sparse image fusion algorithm with application to pan-sharpening. IEEE Trans. Geosci. Remote Sens. 51(5), 2827–2836 (2013)

    Article  Google Scholar 

  24. Yang, X.M., Jian, L.H., Yan, B.Y., et al.: A sparse representation based pansharpening method. Future Gener. Comput. Syst. 88, 385–399 (2018)

    Article  Google Scholar 

  25. Starck, J.L., Elad, M., Donoho, D.L.: Redundant multiscale transforms and their application for morphological component analysis. Adv. Imaging Electron. Phys. 132, 287–348 (2004)

    Article  Google Scholar 

  26. Starck, J.L., Elad, M., Donoho, D.L.: Image decomposition via the combination of sparse representation and a variational approach. IEEE Trans. Image Process. 14(10), 1570–1582 (2005)

    Article  MathSciNet  Google Scholar 

  27. Elad, M., Starck, J.L., Querre, P., Donoho, D.L.: Simultaneous cartoon and texture image inpainting using morphological component analysis (MCA). Appl. Comput. Harmonic Anal. 19(3), 340–358 (2005)

    Article  MathSciNet  Google Scholar 

  28. Yong, X.Y., Ward, R.K., Birch, G.E.: Artifact removal in EEG using morphological component analysis. In: IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2009, pp. 345–348 (2009)

    Google Scholar 

  29. Fadili, M.J., Starck, J.L.: Em algorithm for sparse representation-based image inpainting. IEEE International Conference on Image Processing. ICIP 2005, pp. 61–64. Genoa, Italia (2005)

    Google Scholar 

  30. Redmon, J., Divvala, S., Girshick, R., Farhadi, A.: You only look once: unified, real-time object detection. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016, pp. 779–788 (2016)

    Google Scholar 

  31. Liu, Y., Chen, X., Peng, H., Wang, Z.F.: Multi-focus image fusion with a deep convolutional neural network. Inf. Fusion 36, 191–207 (2017)

    Article  Google Scholar 

  32. Romero, A., Gatta, C., Camps-Valls, G.: Unsupervised deep feature extraction for remote sensing image classification. IEEE Trans. Geosci. Remote Sens. 54(3), 1349–1362 (2016)

    Article  Google Scholar 

  33. Liang, H.M., Li, Q.: Hyperspectral imagery classification using sparse representations of convolutional neural network features. Remote Sens. 8(2), 99 (2016)

    Article  Google Scholar 

  34. Masi, G., Cozzolino, D., Verdoliva, L., et al.: Pansharpening by convolutional neural networks. Remote Sens. 8(7), 594 (2016)

    Article  Google Scholar 

  35. Rao, Y.Z., He, L., Zhu, J.W.: A residual convolutional neural network for pan-shaprening. In: International Workshop on Remote Sensing with Intelligent Processing., RSIP 2017, pp. 1–4 (2017)

    Google Scholar 

  36. Wei, Y.C., Yuan, Q.Q., Shen, H.F., Zhang, L.P.: Boosting the accuracy of multispectral image pansharpening by learning a deep residual network. IEEE Geosci. Remote Sens. Lett. 14(10), 1795–1799 (2017)

    Article  Google Scholar 

  37. Yuan, Q.Q., Wei, Y.C., Meng, X.C., et al.: A multiscale and multidepth convolutional neural network for remote sensing imagery pan-sharpening. IEEE J. Sel. Top. Appl. Earth Observ. Remote Sens. 11(3), 978–989 (2018)

    Google Scholar 

  38. Zhong, J.Y., Yang, B., et al.: Remote sensing image fusion with convolutional neural network. Sens. Imag. 17(1), (2016)

    Google Scholar 

  39. Dong, C., Loy, C.C., He, K., Tang, X.: Image super-resolution using deep convolutional networks. IEEE Trans. Pattern Anal. Mach. Intell. 38(2), 295–307 (2016)

    Article  Google Scholar 

  40. Ye, F., Li, X., Zhang, X.: FusionCNN: a remote sensing image fusion algorithm based on deep convolutional neural networks. Multimedia Tools and Applications 78(11), 14683–14703 (2018). https://doi.org/10.1007/s11042-018-6850-3

    Article  Google Scholar 

Download references

Acknowledgment

This research is funded by the Natural Science Foundation of Shandong (ZR2019MF060, ZR2017MF008), A Project of Shandong Province Higher Educational Science and Technology Key Program (J18KZ016), and the Yantai Science and Technology Plan (2018YT06000271).

Author information

Authors and Affiliations

  1. School of Computer and Control Engineering, Yantai University, Yantai, 264000, China

    Meng Zhu, Jindong Xu & Zhaowei Liu

Authors
  1. Meng Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jindong Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zhaowei Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jindong Xu .

Editor information

Editors and Affiliations

  1. Xidian University, Xi'an, China

    Xiaofeng Chen

  2. Guangzhou University, Guangzhou, China

    Hongyang Yan

  3. Michigan State University, East Lansing, MI, USA

    Qiben Yan

  4. Division of Computer, Electrical and Mathematical Sciences and Engineering, King Abdullah University of Science, Thuwal, Saudi Arabia

    Xiangliang Zhang

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Zhu, M., Xu, J., Liu, Z. (2020). Remote Sensing Image Fusion Based on Multi-morphological Convolutional Neural Network. In: Chen, X., Yan, H., Yan, Q., Zhang, X. (eds) Machine Learning for Cyber Security. ML4CS 2020. Lecture Notes in Computer Science(), vol 12488. Springer, Cham. https://doi.org/10.1007/978-3-030-62463-7_45

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-62463-7_45

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-62462-0

  • Online ISBN: 978-3-030-62463-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation