Abstract
Simultaneous localization and mapping (SLAM) is the task to estimate agent’s ego-motion in the map and reconstruct the 3D geometric of an unknown environment in parallel. Although many SLAM algorithms have been proposed in the past decades, few efforts have been devoted to conducting accurate real-time dense SLAM on resource- and computation-constrained platforms. In this paper, we leverage a shared binary neural network (BNN) architecture to learn robust feature descriptors for depth estimation and pose estimation modules simultaneously, which not only improves the system’s accuracy, but also reduces the computation cost. Also, we propose several optimization strategies targeting feature extraction, feature aggregation as well as feature matching, and to accelerate them on embedded platform. Experimental results demonstrate that our design maintains accurate real-time pose estimation while yielding high-quality dense 3D maps. Our demo video is available at https://github.com/CICAIsubmission/CICAI2023.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Chen, G., et al.: Stereoengine: an FPGA-based accelerator for real-time high-quality stereo estimation with binary neural network. IEEE Trans. Comput.-Aided Des. Integr. Circ. Syst. 39, 4179–4190 (2020)
Geiger, A., et al.: Vision meets robotics: the kitti dataset. Int. J. Rob. Res. 32, 1231–1237 (2013)
Huang, Q., et al.: EDS-SLAM: an energy-efficient accelerator for real-time dense stereo slam with learned feature matching. In: Proceedings of IEEE/ACM International Conference On Computer Aided Design (ICCAD) (2023)
Ling, Y., et al.: Real-time dense mapping for online processing and navigation. J. Field Rob. 35, 1004–1036 (2019)
Liu, R., et al.: ESLAM: an energy-efficient accelerator for real-time orb-slam on FPGA platform. In: Proceedings of the 56th Annual Design Automation Conference 2019, pp. 1–6 (2019)
Liu, Y., et al.: MobileSP: an FPGA-based real-time keypoint extraction hardware accelerator for mobile VSLAM. IEEE Trans. Circuits Syst. I Regul. Pap. 69(12), 4919–4929 (2022)
Vemulapati, V., et al.: FSLAM: an efficient and accurate slam accelerator on SoC FPGAs. In: Proceedings of International Conference on Field-Programmable Technology (ICFPT) (2022)
Wang, et al.: ac\(^2\) slam: FPGA accelerated high-accuracy slam with heapsort and parallel keypoint extractor. In: Proceedings of International Conference on Field-Programmable Technology (ICFPT) (2021)
Xu, Z., et al.: CNN-based feature-point extraction for real-time visual slam on embedded FPGA. In: Proceedings of IEEE 28th Annual International Symposium on Field-Programmable Custom Computing Machines (FCCM) (2020)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Huang, Q., Zhang, Y., Zheng, J., Shang, G., Chen, G. (2024). A CNN-Based Real-Time Dense Stereo SLAM System on Embedded FPGA. In: Fang, L., Pei, J., Zhai, G., Wang, R. (eds) Artificial Intelligence. CICAI 2023. Lecture Notes in Computer Science(), vol 14474. Springer, Singapore. https://doi.org/10.1007/978-981-99-9119-8_53
Download citation
DOI: https://doi.org/10.1007/978-981-99-9119-8_53
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-99-9118-1
Online ISBN: 978-981-99-9119-8
eBook Packages: Computer ScienceComputer Science (R0)