Abstract
Real-time 3D reconstruction is vital for various applications, such as human-robot interaction, virtual reality, and environment perception. The prevalence of low power devices and the rapid advancement of human-robot interaction techniques have resulted in the widespread use of RGB-D sensors for 3D reconstruction. However, high computational complexity and high fidelity make it challenging to perform dense reconstruction in real-time on low-power devices. In this paper, we propose a 3D reconstruction system that runs in real-time without GPU. Our system has three key novelties. The first one is the Single Instruction Multiple Data (SIMD) to speed up feature extraction. The second one is a depth image completion strategy to fill holes in the depth image. The last one is a sparse Robin-Hood voxel hashing algorithm to generate a consistent 3D model from key frames. Real world benchmark shows that our system can run on mobile devices at speeds of up to 30 fps in certain situations. TUM-RGBD dataset is conducted for depth image completion and feature extraction acceleration. On average, compared to ORBSLAM2 and ORBSLAM3, the feature extraction module is 1–2 times faster. We also evaluate our algorithm on the ICL-NUIM dataset which provides the ground truth of surface reconstruction and outperform FlashFusion’s performance while delivering competitive results against the state-of-the-art BundleFusion.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Besl, P., McKay, N.D.: A method for registration of 3-d shapes. IEEE Trans. Pattern Anal. Mach. Intell. 14(2), 239–256 (1992). https://doi.org/10.1109/34.121791
Campos, C., Elvira, R., Rodríguez, J.J.G., Montiel, J.M.M., Tardós, J.D.: Orb-slam3: an accurate open-source library for visual, visual-inertial, and multimap slam. IEEE Trans. Robot. 37(6), 1874–1890 (2021). https://doi.org/10.1109/TRO.2021.3075644
Dai, A., Nießner, M., Zollöfer, M., Izadi, S., Theobalt, C.: Bundlefusion: real-time globally consistent 3d reconstruction using on-the-fly surface re-integration. In: ACM Transactions on Graphics 2017 (TOG) (2017)
Endres, F., Hess, J., Sturm, J., Cremers, D., Burgard, W.: 3-d mapping with an rgb-d camera. IEEE Trans. Rob. 30(1), 177–187 (2014). https://doi.org/10.1109/TRO.2013.2279412
Han, L., Fang, L.: Flashfusion: real-time globally consistent dense 3d reconstruction using CPU computing. In: Robotics: Science and Systems XIV (2018)
Handa, A., Whelan, T., McDonald, J., Davison, A.J.: A benchmark for rgb-d visual odometry, 3d reconstruction and slam. In: 2014 IEEE International Conference on Robotics and Automation (ICRA), pp. 1524–1531 (2014). https://doi.org/10.1109/ICRA.2014.6907054
Kähler, O., Prisacariu, V.A., Ren, C.Y., Sun, X., Torr, P.H.S., Murray, D.W.: Very high frame rate volumetric integration of depth images on mobile device. In: Proceedings International Symposium on Mixed and Augmented Reality 2015, IEEE Transactions on Visualization and Computer Graphics, vol. 22, no. 11 (2015)
Kerl, C., Sturm, J., Cremers, D.: Dense visual slam for rgb-d cameras. In: 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 2100–2106 (2013). https://doi.org/10.1109/IROS.2013.6696650
Klingensmith, M., Dryanovski, I., Srinivasa, S.S., Xiao, J.: Chisel: real time large scale 3d reconstruction onboard a mobile device using spatially hashed signed distance fields. In: Robotics: Science and Systems (2015)
Ku, J., Harakeh, A., Waslander, S.L.: In defense of classical image processing: fast depth completion on the CPU. In: 2018 15th Conference on Computer and Robot Vision (CRV), pp. 16–22. IEEE (2018)
Mur-Artal, R., Tardós, J.D.: ORB-SLAM2: an open-source SLAM system for monocular, stereo and RGB-D cameras. IEEE Trans. Rob. 33(5), 1255–1262 (2017). https://doi.org/10.1109/TRO.2017.2705103
Newcombe, R.A., et al.: Kinectfusion: real-time dense surface mapping and tracking. In: 2011 10th IEEE International Symposium on Mixed and Augmented Reality, pp. 127–136 (2011). https://doi.org/10.1109/ISMAR.2011.6092378
Nießner, M., Zollhöfer, M., Izadi, S., Stamminger, M.: Real-time 3d reconstruction at scale using voxel hashing. Int. Conf. Comput. Graph. Interact. Techniq. (2013)
Palazzolo, E., Behley, J., Lottes, P., Giguère, P., Stachniss, C.: Refusion: 3d reconstruction in dynamic environments for rgb-d cameras exploiting residuals. In: 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 7855–7862 (2019). https://doi.org/10.1109/IROS40897.2019.8967590
Rublee, E., Rabaud, V., Konolige, K., Bradski, G.: Orb: an efficient alternative to sift or surf. In: 2011 International Conference on Computer Vision, pp. 2564–2571 (2011). https://doi.org/10.1109/ICCV.2011.6126544
Sturm, J., Engelhard, N., Endres, F., Burgard, W., Cremers, D.: A benchmark for the evaluation of rgb-d slam systems. In: Proceedings of the International Conference on Intelligent Robot Systems (IROS) (2012)
Whelan, T., Johannsson, H., Kaess, M., Leonard, J.J., McDonald, J.: Robust real-time visual odometry for dense rgb-d mapping. In: 2013 IEEE International Conference on Robotics and Automation, pp. 5724–5731 (2013). https://doi.org/10.1109/ICRA.2013.6631400
Whelan, T., Leutenegger, S., Salas-Moreno, R.F., Glocker, B., Davison, A.J.: Elasticfusion: dense slam without a pose graph. In: Robotics: Science and Systems (2015)
Xiang, X., et al.: Mobile3dscanner: an online 3d scanner for high-quality object reconstruction with a mobile device. IEEE Trans. Visual Comput. Graph. 27(11), 4245–4255 (2021). https://doi.org/10.1109/TVCG.2021.3106491
Zeng, M., Zhao, F., Zheng, J., Liu, X.: Octree-based fusion for realtime 3d reconstruction. Graph. Model. 75, 126–136 (2013). https://doi.org/10.1016/j.gmod.2012.09.002
Zhang, J., Zhu, C., Zheng, L., Xu, K.: Rosefusion: random optimization for online dense reconstruction under fast camera motion. ACM Trans. Graph. (SIGGRAPH 2021) 40(4) (2021)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Chen, Y., Wang, X., Wang, J., Wang, D., Zhou, H., Zhang, J. (2023). A Real-Time and Globally Consistent Meshing Reconstruction Without GPU. In: Yang, H., et al. Intelligent Robotics and Applications. ICIRA 2023. Lecture Notes in Computer Science(), vol 14268. Springer, Singapore. https://doi.org/10.1007/978-981-99-6486-4_4
Download citation
DOI: https://doi.org/10.1007/978-981-99-6486-4_4
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-99-6485-7
Online ISBN: 978-981-99-6486-4
eBook Packages: Computer ScienceComputer Science (R0)