Abstract
Semantic segmentation has been successfully adopted for scenarios such as indoor, outdoor, urban scenes, and synthetic scenes, but applications with scarce labeled data such as search-and-rescue (SAR), have not been addressed. In this work, we propose a transfer learning approach where the U-Net convolutional neural network incorporates ResNet-50 as an encoder for the segmentation of objects in SAR situations. First, the proposed model is trained and validated with 19 classes of the CityScapes dataset. Then we test the proposed approach by i) training the model with a set of 14 Cityscapes classes with relevant similarities to SAR classes, and ii) using transfer learning with the self-developed dataset in SAR scenarios, which has 349 semantic labeled SAR images. The results indicate good recognition in classes with significant presence on the training images.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Adla, D., Reddy, G.V.R., Nayak, P., Karuna, G.: A full-resolution convolutional network with a dynamic graph cut algorithm for skin cancer classification and detection. Healthcare Analytics 3, 100,154 (2023). https://doi.org/10.1016/j.health.2023.100154
Badrinarayanan, V., Kendall, A., Cipolla, R.: SegNet: a deep convolutional encoder-decoder architecture for image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 39(12), 2481–2495 (2017). https://doi.org/10.1109/TPAMI.2016.2644615
Chen, L.-C., Zhu, Y., Papandreou, G., Schroff, F., Adam, H.: Encoder-decoder with atrous separable convolution for semantic image segmentation. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11211, pp. 833–851. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01234-2_49
Cordts, M., et al.: Cityscapes dataset. https://www.cityscapes-dataset.com/benchmarks/#scene-labeling-task
Cordts, M., et al.: The cityscapes dataset for semantic urban scene understanding. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3213–3223 (2016). https://doi.org/10.1109/CVPR.2016.350
Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: Imagenet: a large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). https://doi.org/10.1109/CVPR.2009.5206848
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016)
Huang, J., et al.: Quantitative pharmacokinetic parameter ktrans map assists in regional segmentation of nasopharyngeal carcinoma in dynamic contrast-enhanced magnetic resonance imaging (dce-mri). Biome. Signal Process. Control 87, 105,433 (2024). https://doi.org/10.1016/j.bspc.2023.105433
Huang, L., Yuan, Y., Guo, J., Zhang, C., Chen, X., Wang, J.: Interlaced sparse self-attention for semantic segmentation (2019)
Jeon, H.G., et al.: A large-scale virtual dataset and egocentric localization for disaster responses. IEEE Trans. Pattern Anal. Mach. Intell. (2021). https://doi.org/10.1109/TPAMI.2021.3094531
Jurado-Rodríguez, D., Jurado, J.M., Pádua, L., Neto, A., Muñoz-Salinas, R., Sousa, J.J.: Semantic segmentation of 3d car parts using uav-based images. Compu. Graph. 107, 93–103 (2022). https://doi.org/10.1016/j.cag.2022.07.008
Kim, K., Choi, J.Y.: Application of closed-circuit television image segmentation for irrigation channel water level measurement. Water 15(18) (2023). https://doi.org/10.3390/w15183308
Liu, J., Zhou, W., Cui, Y., Yu, L., Luo, T.: Gcnet: grid-like context-aware network for rgb-thermal semantic segmentation. Neurocomputing 506, 60–67 (2022). https://doi.org/10.1016/j.neucom.2022.07.041
Long, M., Zhu, H., Wang, J., Jordan, M.I.: Deep transfer learning with joint adaptation networks. In: Proceedings of the 34th International Conference on Machine Learning, vol. 70, pp. 2208–2217 (2017)
Lu, A., Honarvar Shakibaei Asli, B.: Seismic image identification and detection based on tchebichef moment invariant. Electronics 12(17) (2023). https://doi.org/10.3390/electronics12173692
Michieli, U., Biasetton, M., Agresti, G., Zanuttigh, P.: Adversarial learning and self-teaching techniques for domain adaptation in semantic segmentation. IEEE Trans. Intell. Veh. 5(3), 508–518 (2020). https://doi.org/10.1109/TIV.2020.2980671
Morales, J., Vázquez-Martín, R., Mandow, A., Morilla-Cabello, D., García-Cerezo, A.: The UMA-SAR dataset: multimodal data collection from a ground vehicle during outdoor disaster response training exercises. Inter. J. Rob. Res. 40(6–7), 835–847 (2021). https://doi.org/10.1177/02783649211004959
Mottaghi, R., et al.: The role of context for object detection and semantic segmentation in the wild. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 891–898 (2014). https://doi.org/10.1109/CVPR.2014.119
Palacios, F., Diago, M.P., Melo-Pinto, P., Tardaguila, J.: Early yield prediction in different grapevine varieties using computer vision and machine learning. Precision Agriculture 24, 407–435 (2023). https://doi.org/10.1007/s11119-022-09950-y
Shelhamer, E., Long, J., Darrell, T.: Fully convolutional networks for semantic segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 39(4), 640–651 (2017). https://doi.org/10.1109/TPAMI.2016.2572683
Sugirtha, T., Sridevi, M.: Semantic segmentation using modified u-net for autonomous driving. In: 2022 IEEE International IOT, Electronics and Mechatronics Conference (IEMTRONICS), pp. 1–7 (2022). https://doi.org/10.1109/IEMTRONICS55184.2022.9795710
Vianna, P., Farias, R., de Albuquerque Pereira, W.C.: U-net and segnet performances on lesion segmentation of breast ultrasonography images. Res. Biomed. Eng. 37, 171–179 (2021). https://doi.org/10.1007/s42600-021-00137-4
Wang, H., Xu, S., Bin Fang, K., Dai, Z.S., Wei, G.Z., Chen, L.F.: Contrast-enhanced magnetic resonance image segmentation based on improved u-net and inception-resnet in the diagnosis of spinal metastases. J. Bone Oncol. 42, 100,498 (2023) . https://doi.org/10.1016/j.jbo.2023.100498
Yousri, R., Elattar, M.A., Darweesh, M.S.: A deep learning-based benchmarking framework for lane segmentation in the complex and dynamic road scenes. IEEE Access 9, 117,565–117,580 (2021) https://doi.org/10.1109/ACCESS.2021.3106377
Zhao, H., Shi, J., Qi, X., Wang, X., Jia, J.: Pyramid scene parsing network. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 6230–6239 (2017). https://doi.org/10.1109/CVPR.2017.660
Zhou, B., Zhao, H., Puig, X., Fidler, S., Barriuso, A., Torralba, A.: Scene parsing through ade20k dataset. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5122–5130 (2017). https://doi.org/10.1109/CVPR.2017.544
Zhou, W., Lv, Y., Lei, J., Yu, L.: Embedded control gate fusion and attention residual learning for rgb-thermal urban scene parsing. IEEE Trans. Intell. Transp. Syst. 24(5), 4794–4803 (2023). https://doi.org/10.1109/TITS.2023.3242651
Acknowledgments
This work has been partially funded by the Ministerio de Ciencia, Innovación y Universidades, Gobierno de España, project PID2021-122944OB-I00. The first author received a grant from Asociación Universitaria Iberoamericana de Postgrado (AUIP), Universidad de Málaga, and Universidad Técnica de Manabí.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Salas-Espinales, A., Vélez-Chávez, E., Vázquez-Martín, R., García-Cerezo, A., Mandow, A. (2024). U-Net/ResNet-50 Network with Transfer Learning for Semantic Segmentation in Search and Rescue. In: Marques, L., Santos, C., Lima, J.L., Tardioli, D., Ferre, M. (eds) Robot 2023: Sixth Iberian Robotics Conference. ROBOT 2023. Lecture Notes in Networks and Systems, vol 978. Springer, Cham. https://doi.org/10.1007/978-3-031-59167-9_21
Download citation
DOI: https://doi.org/10.1007/978-3-031-59167-9_21
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-59166-2
Online ISBN: 978-3-031-59167-9
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)