Skip to main content
SpringerLink
Log in

3D residual spatial–spectral convolution network for hyperspectral remote sensing image classification

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Hyperspectral remote sensing images (HRSI) are 3D image cubes that contain hundreds of spectral bands and have two spatial dimensions and one spectral dimension. HRSI analysis are commonly used in a wide variety of applications such as object detection, precision agriculture and mining. HRSI classification purposes to assign each pixel in HRSI to a unique class. Deep learning is seen as an effective method to improve HRSI classification. In particular, convolutional neural networks (CNNs) are increasingly used in remote sensing field. In this study, a hybrid 3D residual spatial–spectral convolution network (3D-RSSCN) is proposed to extract deep spatiospectral features using 3D CNN and ResNet18 architecture. Simultaneously spatiospectral features extraction is provided using 3D CNN. In deeper CNNs, ResNet architecture is used to achieve higher classification performance as the number of layers increases. In addition, thanks to the ResNet architecture, problems such as degradation and vanishing gradient that may occur in deep networks are overcome. The high dimensionality of the HRSIs increases the computational complexity. Thus, most of studies apply dimension reduction as preprocessing. In the proposed study, principal component analysis (PCA) is used as the preprocessing step for optimum spectral band extraction. The proposed 3D-RSSCN method is tested with Indian pines, Pavia University and Salinas datasets and compared against various deep learning-based methods (SAE, RPNet, 2D CNN, 3D CNN, M3D CNN, HybridSN, FC3D CNN, SSRN, FuSENet, S3EResBoF). As a result of the applications, the best classification accuracy among these methods compared in all datasets is obtained with the proposed 3D-RSSCN. The proposed 3D-RSSCN method has the best accuracy and time performance in classifying.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Data availability

The datasets analyzed during the current study are available in the https://www.ehu.eus/ccwintco/index.php/Hyperspectral_Remote_Sensing_Scenes.

References

  1. Chen C, Jiang F, Yang C et al (2018) Hyperspectral classification based on spectral–spatial convolutional neural networks. Eng Appl Artif Intell 68:165–171. https://doi.org/10.1016/j.engappai.2017.10.015

    Article  Google Scholar 

  2. Jia J, Wang Y, Chen J et al (2020) Status and application of advanced airborne hyperspectral imaging technology: a review. Infrared Phys Technol 104:103115. https://doi.org/10.1016/j.infrared.2019.103115

    Article  Google Scholar 

  3. Sun H, Ren J, Zhao H et al (2019) Superpixel based feature specific sparse representation for spectral-spatial classification of hyperspectral images. Remote Sens. https://doi.org/10.3390/rs11050536

    Article  Google Scholar 

  4. Firat H, Uçan M, Hanbay D (2021) Hyperspectral image classification using MiniVGGNet. J Comput Sci IDAP:295–303

    Google Scholar 

  5. Fırat H, Hanbay D (2021) 4CF-Net: hiperspektral uzaktan algılama görüntülerinin spektral uzamsal sınıflandırılması için yeni 3B evrişimli sinir ağı. Gazi Üniversitesi Mühendislik Mimarlık Fakültesi Derg 1:439–453. https://doi.org/10.17341/gazimmfd.901291

    Article  Google Scholar 

  6. Mohan A, MeenakshiSundaram V (2020) V3O2: hybrid deep learning model for hyperspectral image classification using vanilla-3D and octave-2D convolution. J Real Time Imag Process. https://doi.org/10.1007/s11554-020-00966-z

    Article  Google Scholar 

  7. Gowtham B, Kumar IA, Reddy TS et al (2021) Hyperspectral image analysis using principal component analysis and siamese network. Turkish J Comput Math Educ 12:1191–1198

    Google Scholar 

  8. Firat H, Asker ME, Hanbay D (2022) Classification of hyperspectral remote sensing images using different dimension reduction methods with 3D/2D CNN. Remote Sens Appl Soc Environ. https://doi.org/10.1016/j.rsase.2022.100694

    Article  Google Scholar 

  9. Mohan A, Venkatesan M (2020) HybridCNN based hyperspectral image classification using multiscale spatiospectral features. Infrared Phys Technol. https://doi.org/10.1016/j.infrared.2020.103326

    Article  Google Scholar 

  10. Ahmad M, Khan A, Khan AM et al (2019) Spatial prior fuzziness pool-based interactive classification of hyperspectral images. Remote Sens 11:1–19. https://doi.org/10.3390/rs11091136

    Article  Google Scholar 

  11. Li J, Bioucas-Dias JM, Plaza A (2010) Semisupervised hyperspectral image segmentation using multinomial logistic regression with active learning. IEEE Trans Geosci Remote Sens 48:4085–4098. https://doi.org/10.1109/TGRS.2010.2060550

    Article  Google Scholar 

  12. Wang Y, Yu W, Fang Z (2020) Multiple Kernel-based SVM classification of hyperspectral images by combining spectral, spatial, and semantic information. Remote Sens. https://doi.org/10.3390/RS12010120

    Article  Google Scholar 

  13. Ham JS, Chen Y, Crawford MM, Ghosh J (2005) Investigation of the random forest framework for classification of hyperspectral data. IEEE Trans Geosci Remote Sens 43:492–501. https://doi.org/10.1109/TGRS.2004.842481

    Article  Google Scholar 

  14. Li Y, Zhang H, Shen Q (2017) Spectral-spatial classification of hyperspectral imagery with 3D convolutional neural network. Remote Sens. https://doi.org/10.3390/rs9010067

    Article  Google Scholar 

  15. Zhao C, Wan X, Zhao G et al (2017) Spectral-spatial classification of hyperspectral imagery based on stacked sparse autoencoder and random forest. Eur J Remote Sens 50:47–63. https://doi.org/10.1080/22797254.2017.1274566

    Article  Google Scholar 

  16. Data H, Chen Y, Lin Z et al (2014) Deep learning-based classification of hyperspectral data. IEEE J Sel Top Appl Earth Obs Remote Sens 7:2094–2107. https://doi.org/10.1109/JSTARS.2014.2329330

    Article  Google Scholar 

  17. Mughees A, Tao L (2017) Efficient deep auto-encoder learning for the classification of hyperspectral images. In: 2016 international conference on virtual reality and visualization (ICVRV), 44–51. https://doi.org/10.1109/ICVRV.2016.16

  18. Zhong P, Gong Z, Li S, Schonlieb CB (2017) Learning to diversify deep belief networks for hyperspectral image classification. IEEE Trans Geosci Remote Sens 55:3516–3530. https://doi.org/10.1109/TGRS.2017.2675902

    Article  Google Scholar 

  19. Chen Y, Zhao X, Jia X (2015) Spectral-spatial classification of hyperspectral data based on deep belief network. IEEE J Sel Top Appl Earth Obs Remote Sens 8:2381–2392. https://doi.org/10.1109/JSTARS.2015.2388577

    Article  Google Scholar 

  20. Li J, Xi B, Li Y et al (2018) Hyperspectral classification based on texture feature enhancement and deep belief networks. Remote Sens. https://doi.org/10.3390/rs10030396

    Article  Google Scholar 

  21. Nogay HS, Akinci TC, Yilmaz M (2021) Detection of invisible cracks in ceramic materials using by pre-trained deep convolutional neural network. Neural Comput Appl. https://doi.org/10.1007/s00521-021-06652-w

    Article  Google Scholar 

  22. Zhang C, Sargent I, Pan X et al (2019) Joint deep learning for land cover and land use classification. Remote Sens Environ 221:173–187. https://doi.org/10.1016/j.rse.2018.11.014

    Article  Google Scholar 

  23. Firat H, Uçan M, Hanbay D (2021) Classification of hyperspectral remote sensing images using hybrid 3D–2D CNN architecture. J Comput Sci IDAP:132–140

    Google Scholar 

  24. Üzen H, Turkoglu M, Aslan M, Hanbay D (2022) Depth-wise squeeze and excitation block-based Efficient-Unet model for surface defect detection. Vis Comput. https://doi.org/10.1007/s00371-022-02442-0

    Article  Google Scholar 

  25. Mu C, Guo Z, Liu Y (2020) A multi-scale and multi-level spectral-spatial feature fusion network for hyperspectral image classification. Remote Sens. https://doi.org/10.3390/RS12010125

    Article  Google Scholar 

  26. Meng Z, Li L, Tang X et al (2019) Multipath residual network for spectral-spatial hyperspectral image classification. Remote Sens 11:1–19. https://doi.org/10.3390/rs11161896

    Article  Google Scholar 

  27. Song W, Li S, Fang L (2018) Hyperspectral Image classification with deep feature fusion network. IEEE Trans Geosci Remote Sens 99:3173–3184. https://doi.org/10.1109/IGARSS.2019.8898520

    Article  Google Scholar 

  28. Zhong Z, Li J, Luo Z, Chapman M (2018) Spectral-spatial residual network for hyperspectral image classification: a 3-D deep learning framework. IEEE Trans Geosci Remote Sens 56:847–858. https://doi.org/10.1109/TGRS.2017.2755542

    Article  Google Scholar 

  29. Roy SK, Krishna G, Dubey SR, Chaudhuri BB (2019) HybridSN: exploring 3D-2D CNN feature hierarchy for hyperspectral image classification. arXiv 17:277–281

  30. Ahmad M, Khan AM, Mazzara M et al (2020) A fast and compact 3-D CNN for hyperspectral image classification. IEEE Geosci Remote Sens Lett. https://doi.org/10.1109/LGRS.2020.3043710

    Article  Google Scholar 

  31. Ge Z, Cao G, Li X, Fu P (2020) Hyperspectral image classification method based on 2D–3D CNN and multibranch feature fusion. IEEE J Sel Top Appl Earth Obs Remote Sens 13:5776–5788. https://doi.org/10.1109/JSTARS.2020.3024841

    Article  Google Scholar 

  32. He M, Bo Li HC (2017) Multi-scale 3D deep convolutional neural network for hyperspectral image classification. IEEE Int Conf Image Process 2017:3904–3908

    Google Scholar 

  33. Firat H, Hanbay D (2021) 3B ESA Tabanlı ResNet50 Kullanılarak Hiperspektral Görüntülerin Sınıflandırılması classification of hyperspectral images using 3D CNN based ResNet50. In: 2021 29th signal processing and communications applications conference, p 6–9 https://doi.org/10.1109/SIU53274.2021.9477899

  34. Luo F, Zhang L, Zhou X et al (2020) Sparse-adaptive hypergraph discriminant analysis for hyperspectral image classification. IEEE Geosci Remote Sens Lett 17:1082–1086. https://doi.org/10.1109/LGRS.2019.2936652

    Article  Google Scholar 

  35. Liu S, Shi Q, Zhang L (2021) Few-shot hyperspectral image classification with unknown classes using multitask deep learning. IEEE Trans Geosci Remote Sens 59:5085–5102. https://doi.org/10.1109/TGRS.2020.3018879

    Article  Google Scholar 

  36. Luo F, Zhang L, Du B, Zhang L (2020) Dimensionality reduction with enhanced hybrid-graph discriminant learning for hyperspectral image classification. IEEE Trans Geosci Remote Sens 58:5336–5353. https://doi.org/10.1109/TGRS.2020.2963848

    Article  Google Scholar 

  37. Shi Q, Tang X, Yang T et al (2021) Hyperspectral image denoising using a 3-D attention denoising network. IEEE Trans Geosci Remote Sens. https://doi.org/10.1109/TGRS.2020.3045273

    Article  Google Scholar 

  38. Shi Q, Liu M, Li S et al (2021) A deeply supervised attention metric-based network and an open aerial image dataset for remote sensing change detection. IEEE Trans Geosci Remote Sens. https://doi.org/10.1109/TGRS.2021.3085870

    Article  Google Scholar 

  39. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE computer society conference on computer vision and pattern recognition. IEEE computer society, pp 770–778

  40. Sun W, Du Q (2018) Graph-regularized fast and robust principal component analysis for hyperspectral band selection. IEEE Trans Geosci Remote Sens 56:3185–3195. https://doi.org/10.1109/TGRS.2018.2794443

    Article  Google Scholar 

  41. Rodarmel C, Shan J (2002) Principal component analysis for hyperspectral image classification. Surv L Inf Sci 62:115–122

    Google Scholar 

  42. Zhang Y, Jiang X, Wang X, Cai Z (2019) Spectral-spatial hyperspectral image classification with superpixel pattern and extreme learning machine. Remote Sens. https://doi.org/10.3390/rs11171983

    Article  Google Scholar 

  43. Appice A, Malerba D (2019) Segmentation-aided classification of hyperspectral data using spatial dependency of spectral bands. ISPRS J Photogramm Remote Sens 147:215–231. https://doi.org/10.1016/j.isprsjprs.2018.11.023

    Article  Google Scholar 

  44. Uddin MP, Al MM, Hossain MA (2020) PCA-based feature reduction for hyperspectral remote sensing image classification. IETE Tech Rev Inst Electron Telecommun Eng India. https://doi.org/10.1080/02564602.2020.1740615

    Article  Google Scholar 

  45. Uddin MP, Mamum MA (2017) Feature extraction for hyperspectral image classification. 2021IEEE Reg 10 Humanit Technol Conf 41:6248–6287. https://doi.org/10.1080/01431161.2020.1736732

    Article  Google Scholar 

  46. Jia X, Kuo BC, Crawford MM (2013) Feature mining for hyperspectral image classification. Proc IEEE 101:676–697. https://doi.org/10.1109/JPROC.2012.2229082

    Article  Google Scholar 

  47. Kang X, Xiang X, Li S, Benediktsson JA (2017) PCA-based edge-preserving features for hyperspectral image classification. IEEE Trans Geosci Remote Sens 55:7140–7151. https://doi.org/10.1109/TGRS.2017.2743102

    Article  Google Scholar 

  48. Melgani F, Bruzzone L (2004) Classification of hyperspectral remote sensing images with support vector machines. IEEE Trans Geosci Remote Sens 42:1778–1790. https://doi.org/10.1109/TGRS.2004.831865

    Article  Google Scholar 

  49. Chen Y, Lin Z, Zhao X et al (2014) Deep learning-based classification of hyperspectral data. IEEE J Sel Top Appl Earth Obs Remote Sens 7:2094–2107. https://doi.org/10.1109/JSTARS.2014.2329330

    Article  Google Scholar 

  50. Xu Y, Du B, Zhang F, Zhang L (2018) Hyperspectral image classification via a random patches network. ISPRS J Photogramm Remote Sens 142:344–357. https://doi.org/10.1016/j.isprsjprs.2018.05.014

    Article  Google Scholar 

  51. Makantasis K, Karantzalos K, Doulamis A, Doulamis N (2015) Deep supervised learning for hyperspectral data classification through convolutional neural networks. In: 2015 IEEE international geoscience and remote sensing symposium (IGARSS), pp 4959–4962. https://doi.org/10.1109/IGARSS.2015.7326945

  52. Ben Hamida A, Benoit A, Lambert P, Ben Amar C (2018) 3-D deep learning approach for remote sensing image classification. IEEE Trans Geosci Remote Sens 56:4420–4434. https://doi.org/10.1109/TGRS.2018.2818945

    Article  Google Scholar 

  53. Roy SK, Dubey SR, Chatterjee S, Chaudhuri BB (2020) FuSENet: Fused squeeze-and-excitation network for spectral-spatial hyperspectral image classification. IET Image Process 14:1653–1661. https://doi.org/10.1049/iet-ipr.2019.1462

    Article  Google Scholar 

  54. Roy SK, Chatterjee S, Bhattacharyya S et al (2020) Lightweight spectral-spatial squeeze-and-excitation residual bag-of-features learning for hyperspectral classification. IEEE Trans Geosci Remote Sens 58:5277–5290. https://doi.org/10.1109/TGRS.2019.2961681

    Article  Google Scholar 

  55. AbdElaziz M, Dahou A, Abualigah L et al (2021) Advanced metaheuristic optimization techniques in applications of deep neural networks: a review. Neural Comput Appl 33:14079–14099. https://doi.org/10.1007/s00521-021-05960-5

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Technology, Vocational School of Technical Sciences, Dicle University, Diyarbakır, Turkey

    Hüseyin Firat

  2. Department of Electricity and Energy, Vocational School of Technical Sciences, Dicle University, Diyarbakır, Turkey

    Mehmet Emin Asker

  3. Department of Electronics and Automation, Vocational School of Technical Sciences, Fırat University, Elazığ, Turkey

    Mehmet İlyas Bayindir

  4. Department of Computer Engineering, Engineering Faculty, İnonu University, Malatya, Turkey

    Davut Hanbay

Authors
  1. Hüseyin Firat
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mehmet Emin Asker
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mehmet İlyas Bayindir
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Davut Hanbay
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

All the authors contributed equally to this study.

Corresponding author

Correspondence to Hüseyin Firat.

Ethics declarations

Conflict of interest

The authors declared that they have no conflicts of interest in this study. We declare that we do not have any commercial or associative interest that represents a conflict of interest in connection with the work submitted.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Firat, H., Asker, M.E., Bayindir, M.İ. et al. 3D residual spatial–spectral convolution network for hyperspectral remote sensing image classification. Neural Comput & Applic 35, 4479–4497 (2023). https://doi.org/10.1007/s00521-022-07933-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-022-07933-8

Keywords

Search

Navigation