Abstract
This paper analyzed the quality of the xBD image-training dataset for identifying building damage across a variety of natural hazards using deep learning convolutional neural networks. Specifically, we evaluated the pros and cons of combining training datasets across multiple natural hazards and provided recommendations on using the provided training dataset to optimize classification accuracy for building damage detection. The xBD dataset was rebalanced, using random over-sampling and under-sampling methods. Random over-sampling randomly duplicates the minority class, while random under-sampling randomly cuts-off the majority class. With the balanced dataset, we used the xBD baseline architecture as a starting point in the classification and find that it overfit to the no damage class; therefore, we improved the base classification algorithm by modifying the top layers of ResNet50. We found that not all classes (destroyed, major damage, minor damage, and no damage) were uniformly identifiable across natural hazards; therefore, we retrained the weights from ImageNet, adding five new convolution, batch normalization, and max pooling layers on top of ResNet50. One dropout layer, with a rate of 0.5 was also added in-between the fully connected layers to reduce overfitting and improve performance. We also evaluate the identifiability of the four damage classes in the xbd dataset. Because classification performance was significantly higher for the “no damage” class as compared to “minor”, “major”, and “destroyed” classes, we evaluated merging classes. We kept the “no damage” class and created a second merged class (“damaged”) representing “minor damage,” “major damage,” and “destroyed.” We used the same architecture for the multiclass classification and the binary classification but without the ImageNet weights. Based on this work, we recommend that users be aware of performance differences across natural hazards and across damage classes. Earthquake building damage is extremely limited in the training data and, as a result, application of the trained algorithm on earthquake data cannot be evaluated given the xBD dataset. Building damage due to volcano and tsunami are also poorly represented in the training data, and do not have sufficient data for model validation (especially within all damage classes). Wind hazards are well-represented and therefore application of the algorithm trained using either the wind-only data or the multi-hazard dataset is reliable. The multi-class algorithm trained with wind hazard specific data slightly outperforms a multihazard trained multiclass model (F1 score 0.70 vs. 0.67). Both models have similar performance across all four classes (F1 > 0.5). For flood, fire, and tsunami hazards, we recommend using the binary damage classes as identifiability is low for at least two of the classes in each hazard. For flood building damage, binary classification performance resulted in a significantly higher F1 score when trained with the flood specific dataset versus the multihazard data (0.72 vs. 0.54). On the other hand, for fire building damage, classification performance is slightly higher when the model is trained on multi-hazard data, rather than trained using a fire specific dataset (F1 score 0.46 vs. 0.42).
Similar content being viewed by others
References
Al-Stouhi S (2016) Transfer learning for class imbalance problems with inadequate data. Knowl Inf Syst 48(1):201–228
Amit SNKB, Aoki Y (2017) Disaster detection from aerial imagery with convolution neural network. In: 2017 International electronics symposium on knowledge creation and intelligent computing (IES-KCIC). https://doi.org/10.1109/KCIC.2017.8228593
Anirudh R, Jungkyo Jung J, Vitor Silva S, Giuseppe M (2022) Earthquake building damage detection based on synthetic aperture radar imagery and machine learning. Nat Hazard Earth Syst Sci
Chen B, Chen Z, Deng L, Duan Y, Jie Z (2016) Building change detection with RGB-D map generated from UAV images. Neurocomputing 208:350–364
Cooner A, Shao Y, Campbell J (2016) Detection of urban damage using remote sensing and machine learning algorithms: revisiting the 2010 Haiti Earthquake. Remote Sens 8(10):868
Copernicus: Sentinel-2 - Satellite Missions - eoPortal Directory (2022) Retrieved from COPERNICUS: SENTINEL-2 - SATELLITE MISSIONS - EOPORTAL DIRECTORY: https://earth.esa.int/web/eoportal/satellite-missions/c-missions/copernicus-sentinel-2
Dell'Oro L (2017) Overview of UNITAR-UNOSAT and GDACS Satellite Mapping Coordination System. Retrieved from Appliedsciences.nasa.gov: https://appliedsciences.nasa.gov/sites/default/files/2020-11/gdacspart1bv2.pdf
Federal Emergency Management Agency (FEMA) (2018) Hazus hurricane model user guidance. Technical report, Federal Emergency Management Agency, pp 2
Gao BC (1996) A normalized difference water index for remote sensing of vegetation liquid water from space. Remote Sens Environ 58:257–266
Gupta RH (2019) xBD: A dataset for assessing building damage from satellite imagery. In: Computer vision and pattern recognition
Johnson JK (2019) Survey on deep learning with class imbalance. J Big Data 7(1):1–19
Kaur N et al (2022) “Dahitra: Damage assessment using a novel hierarchical transformer architecture.” Texas A&M University
Koshimura S, Moya L, Mas E, Bai Y (2020) Tsunami damage detection with remote sensing. Geosciennces 10:177
Kubat MMS (2000) Addressing the curse of imbalanced training sets: one-sided selection. In: Fourteenth international conference on machine learning
Laigen Dong JS (2013) A comprehensive review of earthquake-induced building damage detection with remote sensing techniques. ISPRS J Photogram Remote Sens 84:85–99
LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521(7553):436–444
Leevy JL, Khoshgoftaar TM, Bauder RA (2018) A survey on addressing high-class imbalance in big data. J Big Data 5:42
Li Yundong WH (2019) Building damage detection from post-event aerial. Appl Sci 9(6):1128
Malmgren J, and Karlberg T (2023) Deep learning for building damage assessment of the 2023 Turkey earthquakes. Retrieved from KTH Royal Institute of Technology: https://kth.diva-portal.org/smash/get/diva2:1795168/FULLTEXT01.pdf
Mason DC (2021) Floodwater detection in urban areas using Sentinel-1 and WorldDEM data. J Appl Remote Sens 5(03):032003–032003
Mateusz Buda AM (2018) A systematic study of the class imbalance problem in convolutional neural networks. Comput vis Pattern Recogn 106:249–259
Naito S, Tomozawa H, Mori Y, Nakamura H, Fujiwara H (2018) Damage detection method for buildings with machine-learning techniques utilizing images of automobile running surveys aftermath of the 2016 Kumamoto Earthquake. J Disaster Res 13:928–942
Qiuhong Ke JL (2018) Chapter 5 - Computer vision for human–machine interaction. In: Computer vision and pattern recognition. Academic Press, pp 127–145
Rashidian V (2021) Detecting demolished buildings after a natural hazard using high resolution RGB satellite imagery and modified U-Net convolutional neural networks. Remote Sens 13(11):2176
Scikit-learn (2020) Scikit-learn F1-Score. Retrieved from Scikit-learn: https://scikit-learn.org/stable/modules/generated/sklearn.metrics.f1_score.html
Sghaier et al (2017) River extraction from high-resolution SAR images combining a structural feature set and mathematical morphology. IEEE J Sel Topics Appl Earth Obs Remote Sens 10:1025–1038
Shunichi Koshimura LM (2020) Tsunami damage detection with remote sensing: a review. Geosciences 10:2–28
Stephenson LO, Kohne T, Zhan E, Cahill EB, Yun S-H, Ross Z, and Simons M (2021). Deep learning-based damage mapping with {InSAR} coherence time series. In: (IEEE) Transactions on geoscience and remote sensing, pp 1–17
Tay C, Yun S-H, Chin S, Bhardwaj A, Jung J, Hill E (2020) Rapid flood and damage mapping using synthetic aperture radar in response to typhoon hagibis Japan. Sci Data 7(1):100–108
TensorFlow (2021) Data augmentation : TensorFlow Core. Retrieved from Data augmentation : TensorFlow Core: https://www.tensorflow.org/tutorials/images/data_augmentation
Valentijn T (2020) Multi-hazard and spatial transferability of a CNN for automated building damage assessment. Remote Sens
Xu Lu (2019) High-resolution remote sensing image change detection combined with pixel-level and object-level. IEEE 7:78909–78918
Xu SD (2022) Seismic multi-hazard and impact estimation via causal inference from satellite imagery. Nat Commun 13:7793
Yuan X (2021) Automated building segmentation and damage assessment from satellite images for disaster relief. Int Arch Photogram, Remote Sens Spatial Inf Sci 43:741–748
Zhang JMI (2003) KNN approach to unbalanced data distributions: a case study involving information extraction. In: Proceedings of the ICML’2003 workshop on learning from imbalanced datasets
Zhang JX, Zhao Z, Huang GM, Lu Z (2012) CASMSAR: an integrated airborne SAR mapping system. Photogramm Eng Remote Sens 78(11):1110–1114
Funding
This research was funded by the National Geospatial Intelligence Agency, grant number # HM04762010006.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors have not disclosed any competing interests.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Sodeinde, O.R., Koch, M., Moaveni, B. et al. One versus all: identifiability with a multi-hazard and multiclass building damage imagery dataset and a deep learning neural network. Nat Hazards (2024). https://doi.org/10.1007/s11069-024-06500-9
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s11069-024-06500-9