Qian Gao
ABSTRACT
Extracting 3D information from single aerial image plays an important role in computer vision and remote sensing. However, due to the structural complexity of ground objects and noise introduced during the generation stage of ground truth labels, it is challenging to automatically recover the regularized height map from only one orthogonal photography. In this paper, we propose a novel deep network for estimating accurate and regularized height map from a single aerial image. The network mainly contains two sub-networks, namely the height map derivation sub-network and the boundary guidance sub-network. They are sequentially connected together, so that the corresponding boundary map can be directly calculated after the height map is obtained. We also propose a loss function suitable for semantic boundary guidance, which is similar to SSIM loss function at the edges of the ground targets. Apart from pursuing accuracy of height regression, boundary regularity constraints derived from semantic labels are also employed to form a joint metric criterion. We perform a qualitative and quantitative evaluations on ISPRS remote sensing dataset, and the result indicate that our framework improve both accuracy and regularity of estimated depth map.
- Amirkolaee, H.A., Arefi, H.: Height estimation from single aerial images using a deep convolutional encoder-decoder network. ISPRS journal of photogrammetry and remote sensing 149, 50-66 (2019)Google Scholar
- Badrinarayanan, V., Handa, A., Cipolla, R.: Segnet: A deep convolutional encoderdecoder architecture for robust semantic pixel-wise labelling. arXiv preprint arXiv:1505.07293 (2015)Google Scholar
- Badrinarayanan, V., Kendall, A., Cipolla, R.: Segnet: A deep convolutional encoder-decoder architecture for image segmentation. IEEE transactions on pattern analysis and machine intelligence 39(12), 2481-2495 (2017)Google Scholar
- Carvalho, M., Le Saux, B., Trouv´e-Peloux, P., Champagnat, F., Almansa, A.: Multitask learning of height and semantics from aerial images. IEEE Geoscience and Remote Sensing Letters (2019)Google Scholar
- Chen, L.C., Papandreou, G., Kokkinos, I., Murphy, K., Yuille, A.L.: Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs. IEEE transactions on pattern analysis and machine intelligence 40(4), 834-848 (2017)Google Scholar
- Chen, Y., Li, J., Xiao, H., Jin, X., Yan, S., Feng, J.: Dual path networks. In:Advances in Neural Information Processing Systems. pp. 4467-4475 (2017)Google Scholar
- Dai, J., Li, Y., He, K., Sun, J.: R-fcn: Object detection via region-based fully convolutional networks. In: Advances in neural information processing systems. pp. 379-387 (2016)Google Scholar
- Dubost, F., Bortsova, G., Adams, H., Ikram, A., Niessen, W.J., Vernooij, M., De Bruijne, M.: Gp-unet: Lesion detection from weak labels with a 3d regression network. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. pp. 214-221. Springer (2017)Google ScholarDigital Library
- Eigen, D., Fergus, R.: Predicting depth, surface normals and semantic labels with a common multi-scale convolutional architecture. In: Proceedings of the IEEE international conference on computer vision. pp. 2650-2658 (2015)Google ScholarDigital Library
- Gamal, M., Siam, M., Abdel-Razek, M.: Shuffleseg: Real-time semantic segmentation network. arXiv preprint arXiv:1803.03816 (2018)Google Scholar
- Haarbrink, R., Eisenbeiss, H., : Accurate dsm production from unmanned helicopter systems. Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci 37, 1259-1264 (2008)Google Scholar
- He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 770-778 (2016)Google ScholarCross Ref
- Hore, A., Ziou, D.: Image quality metrics: Psnr vs. ssim. In: 2010 20th International Conference on Pattern Recognition. pp. 2366-2369. IEEE (2010)Google ScholarDigital Library
- Kendall, A., Badrinarayanan, V., Cipolla, R.: Bayesian segnet: Model uncertainty in deep convolutional encoder-decoder architectures for scene understanding. arXiv preprint arXiv:1511.02680 (2015)Google Scholar
- Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Advances in neural information processing systems. pp. 1097-1105 (2012)12Google ScholarDigital Library
- Li, X., Chen, H., Qi, X., Dou, Q., Fu, C.W., Heng, P.A.: H-denseunet: hybrid densely connected unet for liver and tumor segmentation from ct volumes. IEEE transactions on medical imaging 37(12), 2663-2674 (2018)Google Scholar
- Liu, C., Chen, L.C., Schroff, F., Adam, H., Hua, W., Yuille, A.L., Fei-Fei, L.: Auto-deeplab: Hierarchical neural architecture search for semantic image segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp. 82-92 (2019)Google ScholarCross Ref
- Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 3431-3440 (2015)Google ScholarCross Ref
- Noh, H., Hong, S., Han, B.: Learning deconvolution network for semantic segmentation. In: Proceedings of the IEEE international conference on computer vision. pp. 1520-1528 (2015)Google ScholarDigital Library
- Noronha, S., Nevatia, R.: Detection and modeling of buildings from multiple aerial images. IEEE Transactions on pattern analysis and machine intelligence 23(5), 501-518 (2001)Google ScholarDigital Library
- Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical image segmentation. In: International Conference on Medical image computing and computer-assisted intervention. pp. 234{241. Springer (2015)Google Scholar
- Ros, G., Sellart, L., Materzynska, J., Vazquez, D., Lopez, A.M.: The synthia dataset: A large collection of synthetic images for semantic segmentation of urban scenes. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 3234-3243 (2016)Google ScholarCross Ref
- Shin, H.C., Roth, H.R., Gao, M., Lu, L., Xu, Z., Nogues, I., Yao, J., Mollura, D., Summers, R.M.: Deep convolutional neural networks for computer-aided detection: Cnn architectures, dataset characteristics and transfer learning. IEEE transactions on medical imaging 35(5), 1285-1298 (2016)Google Scholar
- Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014)Google Scholar
- Verma, V., Kumar, R., Hsu, S.: 3d building detection and modeling from aerial lidar data. In: 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’06). vol. 2, pp. 2213-2220. IEEE (2006)Google ScholarDigital Library
- Vincent, O.R., Folorunso, O., : A descriptive algorithm for sobel image edge detection. In: Proceedings of Informing Science & IT Education Conference (InSITE). vol. 40, pp. 97-107. Informing Science Institute California (2009)Google ScholarCross Ref
- Wu, H., Cai, Z., Wang, Y.: Vison-based auxiliary navigation method using augmented reality for unmanned aerial vehicles. In: IEEE 10th International Conference on Industrial Informatics. pp. 520-525. IEEE (2012)Google ScholarCross Ref
- Xie, S., Girshick, R., Doll´ar, P., Tu, Z., He, K.: Aggregated residual transformations for deep neural networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 1492-1500 (2017)Google ScholarCross Ref
- Zhan, H., Garg, R., Saroj Weerasekera, C., Li, K., Agarwal, H., Reid, I.: Unsupervised learning of monocular depth estimation and visual odometry with deep feature reconstruction. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp. 340-349 (2018)Google ScholarCross Ref
- Zhang, X., Ye, Z., Zhu, J.H., LI, S.: Unmanned aerial vehicle flight simulation and training system based on virtual reality [j]. Acta Simulata Systematica Sinica 8(2002)Google Scholar
Index Terms
- Height Estimation from Single Aerial Imagery with a Deep Boundary-Guided Network
Recommendations
Accurate and Robust Patient Height and Weight Estimation in Clinical Imaging Using a Depth Camera
Medical Image Computing and Computer Assisted Intervention – MICCAI 2023AbstractAccurate and robust estimation of the patient’s height and weight is essential for many clinical imaging workflows. Patient’s safety, as well as a number of scan optimizations, rely on this information. In this paper we present a deep-learning ...
Unmanned aerial vehicles UAVs attitude, height, motion estimation and control using visual systems
This paper presents an implementation of an aircraft pose and motion estimator using visual systems as the principal sensor for controlling an Unmanned Aerial Vehicle (UAV) or as a redundant system for an Inertial Measure Unit (IMU) and gyros sensors. ...
Aerial Target Threat Estimation in Unmanned Aerial Vehicle Reconnaissance Based on Neural Network
ICNSER '22: Proceedings of the 3rd International Conference on Industrial Control Network and System Engineering ResearchThe application of unmanned aerial vehicle (UAV) is the inevitable trend of air combat. In the process of air combat, UAVs have many advantages in target reconnaissance and detection. In the course of UAV reconnaissance and detection, a new threat ...
Comments