Abstract
Accurate and efficient object detection is crucial for safe and efficient operation of earth-moving equipment in mining. Traditional 2D image-based methods face limitations in dynamic and complex mine environments. To overcome these challenges, 3D object detection using point cloud data has emerged as a comprehensive approach. However, training models for mining scenarios is challenging due to sensor height variations, viewpoint changes, and the need for diverse annotated datasets.
This paper presents novel contributions to address these challenges. We introduce a synthetic dataset SimMining-3D [1] specifically designed for 3D object detection in mining environments. The dataset captures objects and sensors positioned at various heights within mine benches, accurately reflecting authentic mining scenarios. An automatic annotation pipeline through ROS interface reduces manual labor and accelerates dataset creation.
We propose evaluation metrics accounting for sensor-to-object height variations and point cloud density, enabling accurate model assessment in mining scenarios. Real data tests validate our model’s effectiveness in object prediction. Our ablation study emphasizes the importance of altitude and height variation augmentations in improving accuracy and reliability.
The publicly accessible synthetic dataset [1] serves as a benchmark for supervised learning and advances object detection techniques in mining with complimentary pointwise annotations for each scene. In conclusion, our work bridges the gap between synthetic and real data, addressing the domain shift challenge in 3D object detection for mining. We envision robust object detection systems enhancing safety and efficiency in mining and related domains.
Supported by Rio Tinto Centre for Mine Automation, Australian Centre for Robotics.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
SimMining-3D. https://github.com/MehalaBala/SimMining_3D
Nikolenko, S.I.: Synthetic simulated environments. In: Nikolenko, S.I. (ed.) Synthetic Data for Deep Learning. SOIA, vol. 174, pp. 195–215. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-75178-4_7
Yue, X., Wu, B., Seshia, S.A., Keutzer, K., Sangiovanni-Vincentelli, A.L.: A LiDAR point cloud generator: from a virtual world to autonomous driving. In: Proceedings of the ACM International Conference on Multimedia Retrieval, Yokohama, Japan, 11–14 June 2018, pp. 458–464 (2018)
Smith, A., et al.: A deep learning framework for semantic segmentation of underwater environments. In: OCEANS 2022, Hampton Roads, pp. 1–7 (2022)
Saputra, R.P., Rakicevic, N., Kormushev, P.: Sim-to-real learning for casualty detection from ground projected point cloud data. In: 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Macau, China, pp. 3918–3925 (2019). https://doi.org/10.1109/IROS40897.2019.8967642
Koenig, K., Howard, A.: Design and use paradigms for Gazebo an open-source multi-robot simulator. In: Proceedings of the IEEE-RSJ International Conference on Intelligent Robots and Systems, pp. 2149–2154 (2004)
Dworak, D., Ciepiela, F., Derbisz, J., Izzat, I., Komorkiewicz, M., Wójcik, M.: Performance of LiDAR object detection deep learning architectures based on artificially generated point cloud data from CARLA simulator. In: 2019 24th International Conference on Methods and Models in Automation and Robotics (MMAR), Miedzyzdroje, Poland, pp. 600–605 (2019). https://doi.org/10.1109/MMAR.2019.8864642
Rong, G., et al.: LGSVL simulator: a high fidelity simulator for autonomous driving. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), Rhodes, Greece, pp. 1–6 (2020). https://doi.org/10.1109/ITSC45102.2020.9294422
Udacity Dataset (2018). https://github.com/udacity/self-driving-car/tree/master/datasets
Sánchez, M., Morales, J., Martínez, J.L., Fernández-Lozano, J.J., García-Cerezo, A.: Automatically annotated dataset of a ground mobile robot in natural environments via gazebo simulations. Sensors 22, 5599 (2022). https://doi.org/10.3390/s22155599
Tallavajhula, A., Meriçli, Ç., Kelly, A.: Off-road lidar simulation with data-driven terrain primitives. In: 2018 IEEE International Conference on Robotics and Automation (ICRA), Brisbane, QLD, Australia, pp. 7470–7477 (2018). https://doi.org/10.1109/ICRA.2018.8461198
Balamurali, M., et al.: A framework to address the challenges of surface mining through appropriate sensing and perception. In: 17th International Conference on Control, Automation, Robotics and Vision (ICARCV), pp. 261–267 (2022). https://doi.org/10.1109/ICARCV57592.2022.10004309
OpenPCDet Development Team. OpenPCDet: An opensource toolbox for 3D object detection from point clouds (2020). https://github.com/open-mmlab/OpenPCDet
Lang, A.H., Vora, S., Caesar, H., Zhou, L., Yang, J., Beijbom, O.: PointPillars: fast encoders for object detection from point clouds. CoRR, abs/1812.05784 (2018)
Acknowledgements
This work has been supported by the Australian Centre for Robotics and the Rio Tinto Centre for Mine Automation, the University of Sydney.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Balamurali, M., Mihankhah, E. (2024). SimMining-3D: Altitude-Aware 3D Object Detection in Complex Mining Environments: A Novel Dataset and ROS-Based Automatic Annotation Pipeline. In: Liu, T., Webb, G., Yue, L., Wang, D. (eds) AI 2023: Advances in Artificial Intelligence. AI 2023. Lecture Notes in Computer Science(), vol 14471. Springer, Singapore. https://doi.org/10.1007/978-981-99-8388-9_5
Download citation
DOI: https://doi.org/10.1007/978-981-99-8388-9_5
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-99-8387-2
Online ISBN: 978-981-99-8388-9
eBook Packages: Computer ScienceComputer Science (R0)