Skip to main content
SpringerLink
Log in

FOV Constraint Region Analysis and Path Planning for Mobile Robot with Observability to Multiple Feature Points

  • Regular Papers
  • Robot and Applications
  • Published:
International Journal of Control, Automation and Systems Aims and scope Submit manuscript

Abstract

In visual servoing tasks, it is an important problem to maintain the observability to feature points on objects, which are usually used to calculate the pose between objects and robots. In particular, when the robot’s vision has a limited field of view (FOV) and the points on objects are distributed separately, the problem is more serious. In this paper, based on FOV constraint region analysis and path planning, we propose a novel method for a mobile robot equipped with a pan-tilt camera to keep all points on objects in its view. According to the Horizontal-FOV and Vertical-FOV angular aperture of camera, bounding boxes assisting to calculate the regions with FOV constraint are acquired firstly. Then the region where the robot inside it cannot keep all points in its view can be obtained. Finally the mobile robot plans a shortest path from the current position to the destination, which can avoid the region with FOV constraint. The results of simulations and experiments prove that our method can make mobile robot keep all feature points in its view when it is moving.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. R. Luo, S. Chou, X. Yang, and N. Peng, “Hybrid eye-to-hand and eye-in-hand visual servo system for parallel robot conveyor object tracking and fetching,” Proc. of IECON Annual Conference of the IEEE Industrial Electronics Society, 2014.

  2. B. Thuilot, P. Martinet, L. Cordesses, and J. Gallice, “Position based visual servoing: keeping the object in the field of vision,” Proc. of IEEE International Conference on Robotics and Automation, 2002.

  3. G. Chesi, K. Hashimoto, D. Prattichizzo, and A. Vicino, “Keeping features in the field of view in eye-in-hand visual servoing: A switching approach,” IEEE Transactions on Robotics, vol. 20, no. 5, pp. 908–914, 2004.

    Article  Google Scholar 

  4. Y. Fang, X. Liu, and X. Zhang, “Adaptive active visual servoing of nonholonomic mobile robots,” IEEE Transactions on Industrial Electronics, vol. 59, no. 1, pp. 486–497, 2012.

    Article  Google Scholar 

  5. D. Park, J. Kwon, and I. Ha, “Novel position-based visual servoing approach to robust global stability under field-of-view constraint,” IEEE Transactions on Industrial Electronics, vol. 59, no. 12, pp. 4735–4752, 2012.

    Article  Google Scholar 

  6. Z. Kang, W. Zou, H. Ma, and Z. Zhu, “Adaptive trajectory tracking of wheeled mobile robots based on a fish-eye camera,” International Journal of Control, Automation and Systems, vol. 17, no. 9, pp. 2297–2309, 2019.

    Article  Google Scholar 

  7. Z. Zhu and W. Zou, “Scalable and occlusion-aware multi-cues correlation filter for robust stereo visual tracking,” International Journal of Robotics and Automation, vol. 34, no. 5, pp. 488–499, 2019.

    Article  Google Scholar 

  8. G. Lopez Nicolas, M. Aranda, and Y. Mezouar, “Formation of differential-drive vehicles with field-of-view constraints for enclosing a moving target,” Proc. of IEEE International Conference on Robotics and Automation, 2017.

  9. V. Le and T. Pham, “Optimal tracking a moving target for integrated mobile robot-pan tilt-stereo camera,” Proc. of IEEE International Conference on Advanced Intelligent Mechatronics, 2016.

  10. N. Tinh, P. Cat, P. Tuan, and T. Q. Bui, “Visual control of integrated mobile robot-pan tilt-camera system for tracking a moving target,” Proc. of IEEE International Conference on Robotics and Biomimetics, 2014, pp. 1566–1571.

  11. S. Heshmati-Alamdari, C. P. Bechlioulis, M. V. Liarokapis, and K. J. Kyriakopoulos, “Prescribed performance image based visual servoing under field of view constraints,” Proc. of IEEE/RSJ International Conference on Intelligent Robots and Systems, 2014.

  12. L. Burlion and H. De Plinval, “Keeping a ground point in the camera field of view of a landing UAV,” Proc. of IEEE International Conference on Robotics and Automation, 2013.

  13. H. Lang, Y. Wang, and W. de Silva Clarence, “Visual servoing with lqr control for mobile robots,” Proc. of IEEE International Conference on Control and Automation, 2010.

  14. K. Nagarajan, W. Yu, and N. Gans, “Keeping multiple moving objects in field of view of a mobile robot,” Proc. of IEEE International Workshop on Robotic and Sensors Environments, 2010.

  15. N. R. Gans, G. Hu, K. Nagarajan, and W. E. Dixon, “Keeping multiple moving targets in the field of view of a mobile camera,” IEEE Transactions on Robotics, vol. 27, no. 4, pp. 822–828, 2011.

    Article  Google Scholar 

  16. B. Thuilot, P. Martinet, L. Cordesses, and J. Gallice, “Position based visual servoing: keeping the object in the field of vision,” Proc. of IEEE International Conference on Robotics and Automation, 2002.

  17. J. C. Kim, D. S. Pae, and M. T. Lim, “Obstacle avoidance path planning based on output constrained model predictive control,” International Journal of Control, Automation and Systems, vol. 17, no. 11, pp. 2850–2861, 2019.

    Article  Google Scholar 

  18. J. G. Kang, S. Y. An, W. S. Choi, and S. Y. Oh, “Recognition and path planning strategy for autonomous navigation in the elevator environment,” International Journal of Control, Automation and Systems, vol. 8, no. 4, pp. 808–821, 2010.

    Article  Google Scholar 

  19. Q. Zou, M. Cong, D. Liu, and Y. Du, “Robotic path planning based on episodic-cognitive map,” International Journal of Control, Automation and Systems, vol. 17, no. 5, pp. 1304–1313, 2019.

    Article  Google Scholar 

  20. S. Bhattacharya, R. Murrieta-Cid, and S. Hutchinson, “Path planning for a differential drive robot: minimal length paths — a geometric approach,” Proc. of IEEE/RSJ International Conference on Intelligent Robots and Systems, 2005.

  21. S. Bhattacharya and S. Hutchinson, “Controllability and properties of optimal paths for a differential drive robot with field-of-view constraints,” Proc. of IEEE International Conference on Robotics and Automation, 2006.

  22. S. Bhattacharya, R. Murrieta-Cid, and S. Hutchinson, “Optimal paths for landmark-based navigation by differentialdrive vehicles with field-of-view constraints,” IEEE Transactions on Robotics, vol. 23, no. 1, pp. 47–59, 2007.

    Article  Google Scholar 

  23. P. Salaris, D. Fontanelli, L. Pallottino, and A. Bicchi, “Shortest paths for a robot with nonholonomic and field-of-view constraints,” IEEE Transactions on Robotics, vol. 26, no. 2, pp. 269–281, 2010.

    Article  Google Scholar 

  24. P. Salaris, L. Pallottino, S. Hutchinson, and A. Bicchi, “From optimal planning to visual servoing with limited fov,” Proc. of IEEE/RSJ International Conference on Intelligent Robots and Systems, 2011.

  25. P. Salaris, A. Cristofaro, L. Pallottino, and A. Bicchi, “Epsilon-optimal synthesis for vehicles with vertically bounded field-of-view,” IEEE Transactions on Automatic Control, vol. 60, no. 5, pp. 1204–1218, 2014.

    Article  MathSciNet  Google Scholar 

  26. P. Salaris, A. Cristofaro, and L. Pallottino, “Epsilonoptimal synthesis for unicycle-like vehicles with limited field-of-view sensors,” IEEE Transactions on Robotics, vol. 31, no. 6, pp. 1404–1418, 2015.

    Article  Google Scholar 

  27. P. Salaris, L. Pallottino, and A. Bicchi, “Optimal synthesis for nonholonomic vehicles with constrained side sensors,” arXiv preprint arXiv:1102.1820, 2011.

  28. P. Salaris, D. Fontanelli, L. Pallottino, and A. Bicchi, “Shortest paths for non-holonomic vehicles with limited field of view camera,” Proc. of IEEE Conference on Decision and Control, 2009.

  29. J. B. Hayet, H. Carlos, C. Esteves, and R. Murrieta Cid, “Motion planning for maintaining landmarks visibility with a differential drive robot,” Robotics and Autonomous Systems, vol. 62, no. 4, pp. 456–473, 2014.

    Article  Google Scholar 

  30. V. Lepetit, F. Moreno Noguer, and P. Fua, “Epnp: Efficient perspective-n-point camera pose estimation,” International Journal of Computer Vision, vol. 81, no. 2, pp. 155–166, 2009.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Institute of Automation, Chinese Academy of Sciences; School of Artificial Intelligence, the University of Chinese Academy of Science, Beijing, China

    Hongxuan Ma, Wei Zou, Siyang Sun, Zheng Zhu & Zhaobing Kang

Authors
  1. Hongxuan Ma
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Wei Zou
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Siyang Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Zheng Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Zhaobing Kang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Wei Zou.

Additional information

Recommended by Associate Editor Tae-Koo Kang under the direction of Editor-in-Chief Keum-Shik Hong.

This work is supported by the National Key Research and Development Program of China (Project 2018YFB1306303), The National Natural Science Foundation of China (Grant No. 61773374), and in part by the Major Basic Research Projects of Natural Science Foundation of Shandong Province under Grant ZR2019ZD07.

Hongxuan Ma received his B.Sc. degree from Central South University, China, in 2016. He is currently a Ph.D. degree candidate in Institute of Automation at Chinese Academy of Sciences, China. He is also with University of Chinese Academy of Sciences, Beijing. His research interests include computer vision and robotics.

Wei Zou received his B.Eng. degree from Baotou University of Iron and Steel Technology, China, in 1997, an M.Eng. degree from Shandong University of Technology, China, in 2000, and a Ph.D. degree from IACAS, China, in 2003. Currently, he is a professor at the Research Center of Precision Sensing and Control, IACAS. His research interests include visual servoing and robot navigation.

Siyang Sun received her B.Sc. degree from University of Science and Technology Beijing, in 2015 and a Ph.D. degree in Institute of Automation, Chinese Academy of Sciences, Beijing, China. Her current research interests include object detection, computer vision and deep learning.

Zheng Zhu received his B.Sc. degree from Zhen Zhou University, China, in 2014 and his Ph.D. degree in Institute of Automation at Chinese Academy of Sciences, China. His research interests include computer vision, deep learning and robotics.

Zhangbing Kang received his B.Eng. degree from Dezhou University, China, in 2008, an M.Eng. degree from Harbin Institute of Technology, China, and a Ph.D degree in Institute of Automation at Chinese Academy of Science, China. His research interests include visual servoing and robot location and navigation.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ma, H., Zou, W., Sun, S. et al. FOV Constraint Region Analysis and Path Planning for Mobile Robot with Observability to Multiple Feature Points. Int. J. Control Autom. Syst. 19, 3785–3800 (2021). https://doi.org/10.1007/s12555-020-0165-7

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12555-020-0165-7

Keywords

Search

Navigation