Abstract
Existing vision based supervised approaches to lateral vehicle control are capable of directly mapping RGB images to the appropriate steering commands. However, they are prone to suffering from inadequate robustness in real world scenarios due to a lack of failure cases in the training data. In this paper, a framework for training a more robust and scalable model for lateral vehicle control is proposed. The framework only requires an unlabeled sequence of RGB images. The trained model takes a point cloud as input and predicts the lateral offset to a subsequent frame from which the steering angle is inferred. The frame poses are in turn obtained from visual odometry. The point cloud is conceived by projecting dense depth maps into 3D. An arbitrary number of additional trajectories from this point cloud can be generated during training. This is to increase the robustness of the model. Online experiments conducted on a driving simulator show that the performance of our model is superior to that of a supervised model trained on the same initial data set and comparable to the same model but trained on data collected with noise injection.
This work was supported by the Munich Center for Machine Learning.
F. Müller and Q. Khan—Contributed equally.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Bojarski, M., et al.: End to end learning for self-driving cars. arXiv e-prints arXiv:1604.07316, April 2016
Canny, J.: A computational approach to edge detection. IEEE Trans. PAMI (1986)
Charles, R., et al.: Pointnet: deep learning on point sets for 3D classification and segmentation. In: CVPR 2017, pp. 77–85 (2017)
Chen, C., et al.: Deepdriving: learning affordance for direct perception in autonomous driving. In: ICCV 2015, pp. 2722–2730 (2015)
Chen, Z., Huang, X.: End-to-end learning for lane keeping of self-driving cars. In: 2017 IEEE Intelligent Vehicles Symposium (IV), pp. 1856–1860 (2017)
Codevilla, F., et al.: End-to-end driving via conditional imitation learning. In: ICRA (2018)
Codevilla, F., et al.: On offline evaluation of vision-based driving models. In: ECCV (2018)
Cordts, M., et al.: The cityscapes dataset for semantic urban scene understanding. In: CVPR (2016)
Dosovitskiy, A., et al.: CARLA: an open urban driving simulator. In: CoRL (2017)
Geiger, A., et al.: Are we ready for autonomous driving? The Kitti vision benchmark suite. In: CVPR (2012)
Godard, C., Aodha, O.M., Firman, M., Brostow, G.: Digging into self-supervised monocular depth estimation. In: ICCV (2019)
Hubschneider, C., et al.: Adding navigation to the equation: turning decisions for end-to-end vehicle control. In: ITSC (2017)
Kendall, A., et al.: Learning to drive in a day. In: ICRA (2019)
Khan, Q., et al.: Towards generalizing sensorimotor control across weather conditions. In: IROS (2019)
Khan, Q., et al.: Self-supervised steering angle prediction for vehicle control using visual odometry. In: AISTATS (2021)
Li, C., et al.: A model based path planning algorithm for self-driving cars in dynamic environment. In: Chinese Automation Congress (CAC) (2015)
Li, D., Zhao, D., Zhang, Q., Chen, Y.: Reinforcement learning and deep learning based lateral control for autonomous driving. IEEE Comput. Intell. Mag. 14, 83 (2019)
Mur-Artal, R., et al.: ORB-SLAM: a versatile and accurate monocular SLAM system. IEEE Trans. Robot. 31, 1147–1163 (2015)
Oliveira, R., Lima, P.F., Collares Pereira, G., Mårtensson, J., Wahlberg, B.: Path planning for autonomous bus driving in highly constrained environments. In: ITSC (2019)
Pomerleau, D.: Alvinn: an autonomous land vehicle in a neural network. In: NIPS (1988)
Rajamani, R.: Vehicle Dynamics and Control. 2nd edn., Springer, Cham (2012). https://doi.org/10.1007/978-1-4614-1433-9
Schönberger, J.L., Frahm, J.M.: Structure-from-motion revisited. In: Conference on Computer Vision and Pattern Recognition (CVPR) (2016)
Schönberger, J.L., Zheng, E., Frahm, J.-M., Pollefeys, M.: Pixelwise view selection for unstructured multi-view stereo. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9907, pp. 501–518. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46487-9_31
Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction. MIT Press, Cambridge (2018)
Toromanoff, M., Wirbel, E., Wilhelm, F., Vejarano, C., Perrotton, X., Moutarde, F.: End to end vehicle lateral control using a single fisheye camera. In: IROS (2018)
Wang, D., Qi, F.: Trajectory planning for a four-wheel-steering vehicle. In: IEEE International Conference on Robotics and Automation (ICRA) (2001)
Wang, T., Chang, D.E.: Improved reinforcement learning through imitation learning pretraining towards image-based autonomous driving. In: ICCAS (2019)
Wenzel, P., et al.: Vision-based mobile robotics obstacle avoidance with deep reinforcement learning. In: ICRA (2021)
Wenzel, P., et al.: 4Seasons: a cross-season dataset for multi-weather SLAM in autonomous driving. In: GCPR (2020)
Zhang, Q., et al.: Model-free reinforcement learning based lateral control for lane keeping. In: 2019 International Joint Conference on Neural Networks (IJCNN) (2019)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 Springer Nature Switzerland AG
About this paper
Cite this paper
Müller, F., Khan, Q., Cremers, D. (2022). Lateral Ego-Vehicle Control Without Supervision Using Point Clouds. In: El Yacoubi, M., Granger, E., Yuen, P.C., Pal, U., Vincent, N. (eds) Pattern Recognition and Artificial Intelligence. ICPRAI 2022. Lecture Notes in Computer Science, vol 13363. Springer, Cham. https://doi.org/10.1007/978-3-031-09037-0_39
Download citation
DOI: https://doi.org/10.1007/978-3-031-09037-0_39
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-09036-3
Online ISBN: 978-3-031-09037-0
eBook Packages: Computer ScienceComputer Science (R0)