Skip to main content
SpringerLink
Log in

Semi-Supervised Land Cover Classification of Remote Sensing Imagery Using CycleGAN and EfficientNet

  • Surveying and Geo-Spatial Engineering
  • Published:
KSCE Journal of Civil Engineering Aims and scope Submit manuscript

Abstract

Image classification of very high resolution (VHR) images is a fundamental task in the remote sensing domain for various applications, such as land cover mapping, vegetation mapping, and urban planning. Recently, deep learning-based semantic segmentation networks demonstrated the promising performance for pixel-level image classification. However, deep learning-based approaches are generally limited by the requirement of a sufficient amount of labeled data to obtain stable accuracy, and acquiring reference labels of remotely-sensed VHR images is very labor-extensive and expensive. Hence, this paper applied a semi-supervised learning-based CycleGAN and EfficientNet for VHR remote sensing image classification to overcome this problem. The proposed method achieved the highest accuracy than the other benchmarks. The largest increase in accuracy was observed in a test site containing complex objects due to the regularization effect of the semi-supervised method using unlabeled data. Moreover, results indicated that a relatively sufficient amount of unlabeled data compared with labeled data are required to increase the classification accuracy by controlling the amount of labeled and unlabeled data. Finally, we verified that the semi-supervised method returned significantly improved results irrespective of the three classification network structures, displaying the applicability of the method for semi-supervised image classification on remotely-sensed VHR images.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Audebert N, Le Saux B, Lefèvre S (2016) Semantic segmentation of earth observation data using multimodal and multi-scale deep networks. Proceedings of Asian conference on computer vision, November 21–23, Taipei, Taiwan

  • Bai H, Cheng J, Huang X, Liu S, Deng C (2021) HCANet: A hierarchical context aggregation network for semantic segmentation of high-resolution remote sensing images. IEEE Geoscience and Remote Sensing Letters 19:1–5, DOI: https://doi.org/10.1109/LGRS.2021.3063799

    Google Scholar 

  • Bartholome E, Belward AS (2005) GLC2000: A new approach to global land cover mapping from Earth observation data. International Journal of Remote Sensing 26(9):1959–1977, DOI: https://doi.org/10.1080/01431160412331291297

    Article  Google Scholar 

  • Berthelot D, Carlini N, Goodfellow I, Papernot N, Oliver A, Raffel C (2019) Mixmatch: A holistic approach to semi-supervised learning. Proceeding of Advances Neural Information Processing System Conference, December 8–14, Vancouver, Canada

  • Bischke B, Helber P, Folz J, Borth D, Dengel A (2019) Multi-task learning for segmentation of building footprints with deep neural networks. Proceedings of 2019 IEEE International Conference on Image Processing, September 22–25, Taipei, Taiwan

  • Chen LC, Papandreou G, Kokkinos I, Murphy K, Yuille AL (2017) Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs. IEEE Transactions on Pattern Analysis and Machine Intelligence 40(4):834–848, DOI: https://doi.org/10.1109/TPAMI.2017.2699184

    Article  Google Scholar 

  • Daudt RC, Le SB, Boulch A (2018) Fully convolutional siamese networks for change detection. Proceedings of 2018 25th IEEE International Conference on Image Processing, October 7–10, Athens, Greece

  • Diakogiannis FI, Waldner F, Caccetta P, Wu C (2020) Resunet-a: A deep learning framework for semantic segmentation of remotely sensed data. ISPRS Journal of Photogrammetry and Remote Sensing 162:94–114, DOI: https://doi.org/10.1016/j.isprsjprs.2020.01.013

    Article  Google Scholar 

  • Dong R, Pan X, Li F (2019a) DenseU-net-based semantic segmentation of small objects in urban remote sensing images. IEEE Access 7:65347–65356, DOI: https://doi.org/10.1109/ACCESS.2019.2917952

    Article  Google Scholar 

  • Dong S, Zhuang Y, Yang Z, Pang L, Chen H, Long T (2019b) Land cover classification from VHR optical remote sensing images by feature ensemble deep learning network. IEEE Geoscience and Remote Sensing Letters 17(8):1396–1400, DOI: https://doi.org/10.1109/LGRS.2019.2947022

    Article  Google Scholar 

  • Foody GM (2002) Status of land cover classification accuracy assessment. Remote Sensing of Environment 80(1):185–201, DOI: https://doi.org/10.1016/S0034-4257(01)00295-4

    Article  Google Scholar 

  • Goodfellow IJ, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Bengio Y (2014) Generative adversarial nets. Proceeding of Advances Neural Information Processing System Conference, December 8–13, Montreal, Canada

  • He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for Image recognition. Proceedings of 2016 IEEE conference on computer vision and pattern recognition, June 27–30, Las Vegas, NV, USA

  • Hung C, Tsai H, Liou T, Lin Y, Yang H (2018) Adversarial learning for semi-supervised semantic segmentation. Proceedings of 2018 IEEE conference on computer vision and pattern recognition, June 18–23, Salt Lake City, UT, USA

  • Iglovikov V, Mushinskiy S, Osin V (2017) Satellite imagery feature detection using deep convolutional neural network: A kaggle competition. Proceedings of 2017 IEEE conference on computer vision and pattern recognition, June 21–26, Honolulu, HI, USA

  • Jin H, Stehman SV, Mountrakis G (2014) Assessing the impact of training sample selection on accuracy of an urban classification: A case study in Denver, Colorado. International Journal of Remote Sensing 35(6):2067–2081, DOI: https://doi.org/10.1080/01431161.2014.885152

    Article  Google Scholar 

  • Jung H, Choi H, Kang M (2021) Boundary enhancement semantic segmentation for building extraction from remote sensed image. IEEE Transactions on Geoscience and Remote Sensing 60:1–12, DOI: https://doi.org/10.1109/TGRS.2021.3108781

    Google Scholar 

  • Kampffmeyer M, Salberg AB, Jenssen R (2016) Semantic segmentation of small objects and modeling of uncertainty in urban remote sensing images using deep convolutional neural networks. Proceedings of 2016 IEEE conference on computer vision and pattern recognition, June 27–30, Las Vegas, NV, USA

  • Kampffmeyer M, Salberg AB, Jenssen R (2018) Urban land cover classification with missing data modalities using deep convolutional neural networks. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing 11(6):1758–1768, DOI: https://doi.org/10.1109/JSTARS.2018.2834961

    Article  Google Scholar 

  • Kwak T (2021) Semi-supervised learning framework for very high resolution image classification using CycleGAN. MSc Thesis, Seoul National University, Seoul, Korea

    Google Scholar 

  • Lin TY, Dollár P, Girshick R, He K, Hariharan B, Belongie S (2017) Feature pyramid networks for object detection. Proceedings of 2017 IEEE conference on computer vision and pattern recognition, June 21–26, Honolulu, HI, USA

  • Liu Q, Kampffmeyer M, Jenssen R, Salberg AB (2020) Dense dilated convolutions’ merging network for land cover classification. IEEE Transactions on Geoscience and Remote Sensing 58(9):6309–6320, DOI: https://doi.org/10.1109/TGRS.2020.2976658

    Article  Google Scholar 

  • Long J, Shelhamer E, Darrell T (2015) Fully convolutional networks for semantic segmentation. Proceedings of 2015 IEEE conference on computer vision and pattern recognition, June 7–12, Boston, MA, USA

  • Mondal AK, Agarwal A, Dolz J, Desrosiers C (2019) Revisiting CycleGAN for semi-supervised segmentation. Proceedings of 2019 IEEE conference on computer vision and pattern recognition, June 15–20, Long Beach, CA, USA

  • Peng D, Bruzzone L, Zhang Y, Guan H, Ding H, Huang X (2020) SemiCDNet: A semisupervised convolutional neural network for change detection in high resolution remote-sensing images. IEEE Transactions on Geoscience and Remote Sensing 59(7):5891–5906, DOI: https://doi.org/10.1109/TGRS.2020.3011913

    Article  Google Scholar 

  • Peng D, Zhang Y, Guan H (2019) End-to-end change detection for high resolution satellite images using improved UNet++. Remote Sensing 11(11):1382, DOI: https://doi.org/10.3390/rs11111382

    Article  Google Scholar 

  • Protopapadakis E, Doulamis A, Doulamis N, Maltezos E (2020) Semi-supervised fine-tuning for deep learning models in remote sensing applications. Proceedings of international symposium on visual computing, November 5–7, San Diego, California, USA

  • Reddy P, Viswanath P, Reddy, BE (2018) Semi-supervised learning: A brief review. International Journal of Engineering and Technology 7(1.8):81, DOI: https://doi.org/10.14419/ijet.v7i1.8.9977

    Article  Google Scholar 

  • Ronneberger O, Fischer P, Brox T (2015) U-net: Convolutional networks for biomedical image segmentation. Proceedings of international conference on medical image computing and computer-assisted intervention, October 5–9, Munich, Germany

  • Sherrah J (2016) Fully convolutional networks for dense semantic labelling of high-resolution aerial imagery. Proceedings of 2016 IEEE conference on computer vision and pattern recognition, June 27–30, Las Vegas, NV, USA

  • Shi W, Gong Y, Ding C, Tao ZM, Zheng N (2018a) Transductive semi-supervised deep learning using min-max features. In Proceedings of the European Conference on Computer Vision, September 8–14, Munich, Germany

  • Shi Y, Li Q, Zhu X (2018b) Building footprint generation using improved generative adversarial networks. IEEE Geoscience and Remote Sensing Letters 16(4):603–607, DOI: https://doi.org/10.1109/LGRS.2018.2878486

    Article  Google Scholar 

  • Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. Proceedings of 2014 IEEE conference on computer vision and pattern recognition, June 24–27, Columbus, Ohio, USA

  • Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, Rabinovich A (2015) Going deeper with convolutions. Proceedings of 2015 IEEE conference on computer vision and pattern recognition, June 7–12, Boston, MA, USA

  • Tan M, Le Q (2019) Efficientnet: Rethinking model scaling for convolutional neural networks. Proceedings of international conference on machine learning, Jun 9–15, Long Beach, CA, USA

  • Wambugu N, Chen Y, Xiao Z, Wei M, Bello A, Junior JM, Li J (2021) A hybrid deep convolutional neural network for accurate land cover classification. International Journal of Applied Earth Observation and Geoinformation 103:102515, DOI: https://doi.org/10.1016/j.jag.2021.102515

    Article  Google Scholar 

  • Wang J, Ding C, Chen S, He C, Luo B (2020) Semi-supervised remote sensing image semantic segmentation via consistency regularization and average update of pseudo-label. Remote Sensing 12(21):3603, DOI: https://doi.org/10.3390/rs12213603

    Article  Google Scholar 

  • Wang X, Chen B, Ding H, Tang J, Luo B (2022) Semi-supervised semantic segmentation of remote sensing images with iterative contrastive network. IEEE Geoscience and Remote Sensing Letters 19:1–5, DOI: https://doi.org/10.1109/LGRS.2022.3157032

    Google Scholar 

  • Wu H, Prasad S (2017) Semi-supervised deep learning using pseudo labels for hyperspectral image classification. IEEE Transactions on Image Processing 27(3):1259–1270, DOI: https://doi.org/10.1109/TIP.2017.2772836

    Article  MathSciNet  MATH  Google Scholar 

  • Yan LC, Yoshua B, Geoffrey H (2015) Deep learning. Nature 521(7553): 436–444, DOI: https://doi.org/10.1038/nature14539

    Article  Google Scholar 

  • Zhang Z, Liu Q, Wang Y (2018) Road extraction by deep residual u-net. IEEE Geoscience and Remote Sensing Letters 15(5):749–753, DOI: https://doi.org/10.1109/LGRS.2018.2802944

    Article  Google Scholar 

  • Zhao H, Shi J, Qi X, Wang X, Jia J (2017) Pyramid scene parsing network. Proceedings of 2017 IEEE conference on computer vision and pattern recognition, June 21–26, Honolulu, HI, USA

  • Zhou L, Zhang C, Wu M (2018) D-linknet: Linknet with pretrained encoder and dilated convolution for high resolution satellite imagery road extraction. Proceedings of 2018 IEEE conference on computer vision and pattern recognition, June 18–23, Salt Lake City, UT, USA

  • Zhu JY, Park T, Isola P, Efros AA (2017) Unpaired image-to-image translation using cycle-consistent adversarial networks. Proceedings of 2017 IEEE conference on computer vision and pattern recognition, June 21–26, Honolulu, HI, USA

Download references

Acknowledgments

This work is financially supported by the Korea Ministry of Land, Infrastructure and Transport (MOLIT) as 「Innovative Talent Education Program for Smart City」, the Institute of Engineering Research at Seoul National University, and a grant (20009742) of Ministry-Cooperation R&D program of Disaster-Safety, funded by Ministry of Interior and Safety (MOIS, Korea). This paper is partly based on the author’s M.S. thesis.

Author information

Authors and Affiliations

  1. Dept. of Civil and Environmental Engineering, Seoul National University, Seoul, 08826, Korea

    Taehong Kwak & Yongil Kim

Authors
  1. Taehong Kwak
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yongil Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yongil Kim.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kwak, T., Kim, Y. Semi-Supervised Land Cover Classification of Remote Sensing Imagery Using CycleGAN and EfficientNet. KSCE J Civ Eng 27, 1760–1773 (2023). https://doi.org/10.1007/s12205-023-2285-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12205-023-2285-0

Keywords

Search

Navigation