Abstract
In this paper, a novel method for distance estimation from multiple cameras to the object viewed with these cameras is presented. The core element of the method is multilayer neural network named Multi-DisNet, which is used to learn the relationship between the sizes of the object bounding boxes in the cameras images and the distance between the object and the cameras. The Multi-DisNet was trained using a supervised learning technique where the input features were manually calculated parameters of the objects bounding boxes in the cameras images and outputs were ground-truth distances between the objects and the cameras. The presented distance estimation system can be of benefit for all applications where object (obstacle) distance estimation is essential for the safety such as autonomous driving applications in automotive or railway. The presented object distance estimation system was evaluated on the images of real-world railway scenes. As a proof-of-concept, the results on the fusion of two sensors, an RGB and thermal camera mounted on a moving train, in the Multi-DisNet distance estimation system are shown. Shown results demonstrate both the good performance of Multi-DisNet system to estimate the mid (up to 200 m) and long-range (up to 1000 m) object distance and benefit of sensor fusion to overcome the problem of not reliable object detection.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Jiménez, F., Naranjo, J.E., Anaya, J.J., García, F., Ponz, A., Armingol, J.M.: Advanced driver assistance system for road environments to improve safety and efficiency. Transp. Res. Procedia 14, 2245–2254 (2016)
Weichselbaum, J., Zinner, C., Gebauer, O., Pree, W.: Accurate 3D-vision-based obstacle detection for an autonomous train. Comput. Ind. 64(9), 1209–1220 (2013)
Bouain, M., Ali, K.M.A., Berdjag, D., Fakhfakh, N., Atitallah, R.B.: An embedded multi-sensor data fusion design for vehicle perception tasks. J. Commun. 13(1), 8–14 (2018)
Kim, S., Kim, H., Yoo, W., Huh, K.: Sensor fusion algorithm design in detecting vehicles using laser scanner and stereo vision. IEEE Trans. Intell. Transp. Syst. 17(4), 1072–1084 (2016)
Leu, A., Aiteanu, D., Gräser, A.: High speed stereo vision based automotive collision warning system. In: Precup, R.E., Kovács, S., Preitl, S., Petriu, E. (eds.) Applied Computational Intelligence in Engineering and Information Technology, vol. 1, pp. 187–199. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-28305-5_15
Bernini, N., Bertozzi, M., Castangia, L., Patander, M., Sabbatelli, M.: Real-time obstacle detection using stereo vision for autonomous ground vehicles: a survey. In: 2014 IEEE 17th International Conference on Intelligent Transportation Systems (ITSC), China, pp. 873–878 (2014)
Ristić-Durrant, D., et al.: SMART concept of an integrated multi-sensory on-board system for obstacle recognition. In: 7th Transport Research Arena TRA 2018, Austria, 16–19 April 2018
Saxena, A., Sung, H., Ng, A.Y.: 3-D depth reconstruction from a single still image. Int. J. Comput. Vis. 76(1), 53–69 (2007)
Project SMART. http://www.smartrail-automation-project.net
Haseeb, M.A., Guan, J., Ristić-Durrant, D., Gräser, A.: DisNet: a novel method for distance estimation from monocular camera. In: 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems - IROS, Spain (2018)
Chen, S.Y.: Kalman filter for robot vision: a survey. IEEE Trans. Industr. Electron. 59(11), 4409–4420 (2012)
Caltagironea, L., Bellonea, M., Svenssonb, L., Wahdea, M.: LIDAR-camera fusion for road detection using fully convolutional neural networks. Robot. Auton. Syst. 111, 125–131 (2019). https://doi.org/10.1016/j.robot.2018.11.002
Asvadi, A., Garrote, L., Premebida, C., Peixoto, P., Nunes, U.J.: Multimodal vehicle detection: fusing 3D-LIDAR and color camera data. Pattern Recogn. Lett. 115, 20–29 (2017). https://doi.org/10.1016/j.patrec.2017.09.038
Geiger, A., Lenz, P., Stiller, C., Urtasun, R.: Vision meets robotics: the KITTI dataset. Int. J. Robot. Res. 32(11), 1231–1237 (2013). https://doi.org/10.1177/0278364913491297
Berkeley Deep Drive BDD 100 K dataset. https://bdd-data.berkeley.edu/. Accessed 15 Feb 2019
Ye, T., Wang, B., Song, P., Li, J.: Automatic railway traffic object detection system using feature fusion refine neural network under shunting mode. Sensors 18(6), 1916 (2018). https://doi.org/10.3390/s18061916
The imaging source, GigE color zoom camera. https://www.theimagingsource.com/. Accessed 15 Feb 2019
FLIR thermal imaging, Tau2. https://www.flir.com/products/tau-2/. Accessed 15 Feb 2019
Dutta, A., Gupta, A., Zissermann, A.: VGG image annotator (VIA). http://www.robots.ox.ac.uk/~vgg/software/via. Accessed 15 Feb 2019
COCO dataset. https://arxiv.org/pdf/1405.0312.pdf. Accessed 15 Feb 2019
Redmon, J., Farhadi, A.: YOLOv3: an incremental improvement. arXiv (2018)
Ristić-Durrant, D., et al.: SMART: a novel on-board integrated multi-sensor long-range obstacle detection system for railways. In: RAILCON, Nis, November 2018
Duvieubourg, L., Cabestaing, F., Ambellouis, S., Bonnet, P.: Long distance vision sensor for driver assistance. IFAC Proc. Vol. 40(15), 330–336 (2007)
Pinggera, P., Franke, U., Mester, R.: High-performance long range obstacle detection using stereo vision. In: 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems-IROS, pp. 1308–1313 (2015)
Shift2Rail Joint Undertaking, Multi-annual Action Plan, Brussels, November 2015
Pedregosa, F., et al.: Scikit-learn: machine learning in Python. J. Mach. Learn. Res. 12, 2825–2830 (2011)
Acknowledgements
This research has received funding from the Shift2Rail Joint Undertaking under the European Union’s Horizon 2020 research and innovation programme under grant agreement No. 730836.
Special thanks to Serbian Railways Infrastructure, and Serbia Cargo for support in realization of the SMART OD Field tests.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Abdul, H.M., Danijela, RD., Axel, G., Milan, B., Dušan, S. (2019). Multi-DisNet: Machine Learning-Based Object Distance Estimation from Multiple Cameras. In: Tzovaras, D., Giakoumis, D., Vincze, M., Argyros, A. (eds) Computer Vision Systems. ICVS 2019. Lecture Notes in Computer Science(), vol 11754. Springer, Cham. https://doi.org/10.1007/978-3-030-34995-0_41
Download citation
DOI: https://doi.org/10.1007/978-3-030-34995-0_41
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-34994-3
Online ISBN: 978-3-030-34995-0
eBook Packages: Computer ScienceComputer Science (R0)