Skip to main content
SpringerLink
Log in

System for deployment of groups of unmanned micro aerial vehicles in GPS-denied environments using onboard visual relative localization

  • Published:
Autonomous Robots Aims and scope Submit manuscript

Abstract

A complex system for control of swarms of micro aerial vehicles (MAV), in literature also called as unmanned aerial vehicles (UAV) or unmanned aerial systems (UAS), stabilized via an onboard visual relative localization is described in this paper. The main purpose of this work is to verify the possibility of self-stabilization of multi-MAV groups without an external global positioning system. This approach enables the deployment of MAV swarms outside laboratory conditions, and it may be considered an enabling technique for utilizing fleets of MAVs in real-world scenarios. The proposed visual-based stabilization approach has been designed for numerous different multi-UAV robotic applications (leader-follower UAV formation stabilization, UAV swarm stabilization and deployment in surveillance scenarios, cooperative UAV sensory measurement) in this paper. Deployment of the system in real-world scenarios truthfully verifies its operational constraints, given by limited onboard sensing suites and processing capabilities. The performance of the presented approach (MAV control, motion planning, MAV stabilization, and trajectory planning) in multi-MAV applications has been validated by experimental results in indoor as well as in challenging outdoor environments (e.g., in windy conditions and in a former pit mine).

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22
Fig. 23
Fig. 24
Fig. 25
Fig. 26
Fig. 27
Fig. 28
Fig. 29
Fig. 30

Similar content being viewed by others

Notes

  1. http://www.vicon.com/.

  2. https://pixhawk.org/modules/px4flow.

  3. http://www.mikrokopter.de/.

  4. For the \(320 \times 240\) resolution, the frame rate is limited by the camera, which can provide images at 60 Hz.

References

  • Balch, T., & Hybinette, M. (2000). Social potentials for scalable multi-robot formations. In Proceedings of IEEE conference on robotics and automation.

  • Ballard, D. H. (1981). Generalizing the Hough transform to detect arbitrary shapes. Pattern Recognition, 13(2), 111–122.

    Article  MATH  Google Scholar 

  • Barfoot, T. D., & Clark, C. M. (2004). Motion planning for formations of mobile robots. Robotics and Autonomous Systems, 46, 65–78.

    Article  Google Scholar 

  • Barnes, L., Garcia, R., Fields, M., & Valavanis, K. (2008). Swarm formation control utilizing ground and aerial unmanned systems. In IEEE/RSJ international conference on intelligent robots and systems.

  • Bennet, D. J., & McInnes, C. R. (2009). Verifiable control of a swarm of unmanned aerial vehicles. Journal of Aerospace Engineering, 223(7), 939–953.

    Google Scholar 

  • Berman, S., Halasz, A., Hsieh, M., & Kumar, V. (2009). Optimized stochastic policies for task allocation in swarms of robots. IEEE Transactions on Robotics, 25(4), 927–937.

    Article  Google Scholar 

  • Bošnak, M., Matko, D., & Blažič, S. (2012). Quadrocopter control using an on-board video system with off-board processing. Robotics and Autonomous Systems, 60(4), 657–667.

    Article  Google Scholar 

  • Buerkle, A., & Leuchter, S. (2009). Development of micro UAV swarms. Autonome mobile systeme 2009 (pp. 217–224)., Informatik aktuell series Berlin: Springer.

    Chapter  Google Scholar 

  • Cai, W., Yu, Q., & Wang, H. (2004). A fast contour-based approach to circle and ellipse detection. In 5th world congress on intelligent control and automation (WCICA).

  • Cai, N., Xi, J.-X., & Zhong, Y.-S. (2011). Swarm stability of high-order linear time-invariant swarm systems. Control Theory Applications IET, 5(2), 402–408.

    Article  MathSciNet  Google Scholar 

  • Carreras, M., Ridao, P., García, R., & Nicosevici, T. (2003). Vision-based localization of an underwater robot in a structured environment. In ICRA.

  • Chao, Z., Zhou, S.-L., Ming, L., & Zhang, W.-G. (2012). UAV formation flight based on nonlinear model predictive control. Mathematical Problems in Engineering, 2012(1), 1–16.

    Article  MATH  Google Scholar 

  • Cheah, C. C., Hou, S. P., & Slotine, J. J. E. (2009). Region-based shape control for a swarm of robots. Automatica, 45(10), 2406–2411.

    Article  MathSciNet  MATH  Google Scholar 

  • Christensen, A., O’Grady, R., & Dorigo, M. (2009). From fireflies to fault-tolerant swarms of robots. IEEE Transactions on Evolutionary Computation, 13(4), 754–766.

    Article  Google Scholar 

  • Doitsidis, L., Weiss, S., Renzaglia, A., Kosmatopoulos, E., Siegwart, R., Scaramuzza, D., et al. (2012). Optimal surveillance coverage for teams of micro aerial vehicles in GPS-denied environments using onboard vision. Autonomous Robots, 33(1–2), 173–188.

    Article  Google Scholar 

  • Faigl, J., Krajník, T., Chudoba, J., Preucil, L., Saska, M. (2013). Low-cost embedded system for relative localization in robotic swarms. In Proceedings of IEEE international conference on robotics and automation.

  • Faigl, J., Krajník, T., Vonásek, V., & Přeučil, L. (2012). On Localization Uncertainty in an Autonomous Inspection. In IEEE international conference on robotics and automation (ICRA).

  • Fazli, P., Davoodi, A., & Mackworth, A. (2013). Multi-robot repeated area coverage. Autonomous Robots, 34(4), 251–276.

    Article  Google Scholar 

  • Filho, C., Lima Neto, F., Lins, A., Nascimento, A., & Lima, M. (2009). fish school search. Nature-inspired algorithms for optimisation, studies in computational intelligence (pp. 261–277). Berlin: Springer.

    Chapter  Google Scholar 

  • Garca Carrillo, L., Sanchez, A., Dzul, A., & Lozano, R. (2011). Stabilization and trajectory tracking of a quad-rotor using vision. Journal of Intelligent & Robotic Systems, 61, 103–118.

    Article  Google Scholar 

  • Hamann, H., & Worn, H. (2008). A framework of spacetime continuous models for algorithm design in swarm robotics. Swarm Intelligence, 2, 209–239.

    Article  Google Scholar 

  • Holland, O., Woods, J., Nardi, R., & Clark, A. (2005). Beyond swarm intelligence: The UltraSwarm. In IEEE swarm intelligence symposium.

  • Jia, L.-Q., Liu, H.-M., Wang, Z.-H., & Chen, H. (2011). An effective non-HT circle detection for centers and radii. In ICMLC.

  • Kennedy, J., & Eberhart, R. (1995). Particle swarm optimization. In Proceedings of the IEEE international conference on neural networks (Vol. 4).

  • Kloetzer, M., & Belta, C. (2007). Temporal logic planning and control of robotic swarms by hierarchical abstractions. IEEE Transactions on Robotics, 23(2), 320–330.

    Article  Google Scholar 

  • Krajník, T., Nitsche, M., Faigl, J., Vanek, P., Saska, M., Přeučil, L., Duckett, T., & Mejail, M. (2014). A practical multirobot localization system. Journal of Intelligent & Robotic Systems, Online first, 2014.http://dx.doi.org/10.1007/s10846-014-0041-x.

  • Kumar, M., Garg, D., & Kumar, V. (2010). Segregation of heterogeneous units in a swarm of robotic agents. IEEE Transactions on Automatic Control, 55(3), 743–748.

    Article  MathSciNet  Google Scholar 

  • Lange, S., Sunderhauf, N., & Protzel, P. (2009). A vision based onboard approach for landing and position control of an autonomous multirotor uav in GPS-denied environments. In International conference on advanced robotics (ICAR).

  • Lee, T., Leoky, M., & McClamroch, N. (2010). Geometric tracking control of a quadrotor UAV on se(3). In 49th IEEE conference on decision and control (CDC).

  • Leonard, N., & Fiorelli, E. (2001). Virtual leaders, artificial potentials and coordinated control of groups. In Proceedings of the 40th IEEE Conference on Decision and Control.

  • Liu, C., Chen, W.-H., & Andrews, J. (2011). Piecewise constant model predictive control for autonomous helicopters. Robotics and Autonomous Systems, 59(78), 571–579.

    Article  Google Scholar 

  • Liu, W., Winfield, A., Sa, J., Chen, J., & Dou, L. (2007). Strategies for energy optimisation in a swarm of foraging robots. Swarm Robotics, 4433, 14–26.

    Article  Google Scholar 

  • Marjovi, A., & Marques, L. (2013). Optimal spatial formation of swarm robotic gas sensors in odor plume finding. Autonomous Robots, 35(2–3), 93–109.

    Article  Google Scholar 

  • Masselli, A., & Zell, A. (2012). A novel marker based tracking method for position and attitude control of MAVs. In Proceedings of international micro air vehicle conference and flight competition.

  • Michael, N., Mellinger, D., Lindsey, Q., & Kumar, V. (2010). The grasp multiple micro-UAV testbed. IEEE Robotics Automation Magazine, 17(3), 56–65.

    Article  Google Scholar 

  • Multimedia. (2015). Various experiments with multi-MAV system verifying the proposed approach. http://mrs.felk.cvut.cz/data/mavgroups/ Retrieved from 8 August 2015.

  • No, T. S., Kim, Y., Tahk, M.-J., & Jeon, G.-E. (2011). Cascade-type guidance law design for multiple-UAV formation keeping. Aerospace Science and Technology, 15(6), 431–439.

    Article  Google Scholar 

  • Rad, A. A., Faez, K., & Qaragozlou, N. (2003). Fast circle detection using gradient pair vectors. In DICTA.

  • Saska, M., Chudoba, J., Preucil, L., Thomas, J., Loianno, G., Tresnak, A., Vonasek, V., & Kumar, V. (2014a). Autonomous deployment of swarms of micro-aerial vehicles in cooperative surveillance. In Proceedings of 2014 international conference on unmanned aircraft systems (ICUAS).

  • Saska, M., Hess, M., & Schilling, K. (2007). Hierarchical spline path planning method for complex environments. In Proceedings of the 4th international conference on informatics in control, automation and robotics. Angers, France.

  • Saska, M., Kasl, Z., Preucil, L. (2014b). Motion planning and control of formations of micro aerial vehicles. In Proceedings of the 19th world congress of the international federation of automatic control.

  • Saska, M., Mejia, J. S., Stipanovic, D. M., Schilling, K. (2009). Control and navigation of formations of car-like robots on a receding horizon. In Proceedings of 3rd IEEE multi-conference on systems and control.

  • Saska, M., Vonasek, V., & Preucil, L. (2010). Control of ad-hoc formations for autonomous airport snow shoveling. In IEEE/RSJ international conference on intelligent robots and systems (Vol. 1, pp. 4995–5000). Taipei: IEEE Industrial Electronics Society.

  • Saska, M., Krajnik, T., Vonasek, V., Kasl, Z., Spurny, V., & Preucil, L. (2014c). Fault-tolerant formation driving mechanism designed for heterogeneous MAVs-UGVs groups. Journal of Intelligent and Robotic Systems, 73(1–4), 603–622.

  • Saska, M., Mejia, J., Stipanovic, D., Vonasek, V., Schilling, K., & Preucil, L. (2013a). Control and navigation in manoeuvres of formations of unmanned mobile vehicles. European Journal of Control, 19(2), 157–171.

    Article  MathSciNet  MATH  Google Scholar 

  • Saska, M., Spurny, V., & Vonasek, V. (2016). Predictive control and stabilization of nonholonomic formations with integrated spline-path planning. Robotics and Autonomous Systems, 75, 379–397.

    Article  Google Scholar 

  • Saska, M., Vonasek, V., Krajnik, T., & Preucil, L. (2014d). Coordination and navigation of heterogeneous MAV&UGV formations localized by a hawk-eye-like approach under a model predictive control scheme. International Journal of Robotics Research, 33(10), 1393–1412.

  • Saska, M., Vonasek, V., & Preucil, L. (2013b). Trajectory planning and control for airport snow sweeping by autonomous formations of ploughs. Journal of Intelligent and Robotic Systems, 72(2), 239–261.

    Article  Google Scholar 

  • Schmickl, T., & Crailsheim, K. (2008). Trophallaxis within a robotic swarm: bio-inspired communication among robots in a swarm. Autonomous Robots, 25, 171–188.

    Article  Google Scholar 

  • Sharma, R. K., & Ghose, D. (2009). Collision avoidance between UAV clusters using swarm intelligence techniques. International Journal of Systems Science, 40, 521–538.

    Article  MathSciNet  MATH  Google Scholar 

  • Teacy, W., Nie, J., McClean, S., & Parr, G. (2010). Maintaining connectivity in UAV swarm sensing. In IEEE GLOBECOM Workshops.

  • Trianni, V. (2008). Evolutionary swarm robotics. New York: Springer.

    Book  Google Scholar 

  • Turpin, M., Michael, N., & Kumar, V. (2012). Trajectory design and control for aggressive formation flight with quadrotors. Autonomous Robots, 33(1–2), 143–156.

    Article  Google Scholar 

  • Winfield, A., Liu, W., Nembrini, J., & Martinoli, A. (2008). Modelling a wireless connected swarm of mobile robots. Swarm Intelligence, 2, 241–266.

    Article  Google Scholar 

  • Yang, S., Scherer, S. A., & Zell, A. (2012). An onboard monocular vision system for autonomous takeoff, hovering and landing of a micro aerial vehicle. In International conference on unmanned aircraft systems (ICUAS’12).

  • Yang, S., Scherer, S., & Zell, A. (2012). An Onboard monocular vision system for autonomous takeoff, hovering and landing of a micro aerial vehicle. Journal of Intelligent & Robotic Systems, 69(1–4), 499–515.

    Google Scholar 

  • Yu, H., & Beard, R. (2013). A vision-based collision avoidance technique for micro air vehicles using local-level frame mapping and path planning. Autonomous Robots, 34(1–2), 93–109.

    Article  Google Scholar 

Download references

Acknowledgments

This work has been supported by the Ministry of Education of the Czech Republic under project no. LH11053 and the experimental works required for paper revisions by Project No. HS 13167/830/8301616C000 founded by Khalifa University for the MBZIRC competition, both projects supporting the joint research of the Czech Technical University in Prague and the University of Pennsylvania. In addition, Martin Saska has been supported by the Grant Agency of the Czech Republic under postdoc Grant No. P103-12/P756. The work of Jan Faigl has been also partially supported by the Czech Science Foundations (GACR) under the research Project No. 13-18316P. Tomas Krajnik has been supported by the EU project STRANDS (ICT-600623). Tomas Baca has been supported by CTU grant no. SGS15/157/OHK3/2T/13. Final experiments done by Martin Saska for revisions of the paper have been supported by the Czech Science Foundations (GACR) under the research Project No. 16-24206S.

Author information

Authors and Affiliations

  1. Dept. of Cybernetics, Faculty of Electrical Engineering, Czech Technical University in Prague, Technicka 2, 166 27, Prague, Czech Republic

    Martin Saska, Tomas Baca, Jan Chudoba & Libor Preucil

  2. GRASP Lab, University of Pennsylvania, 3330 Walnut Street, 19104, Philadelphia, PA, USA

    Justin Thomas, Giuseppe Loianno & Vijay Kumar

  3. Lincoln Centre for Autonomous Systems, University of Lincoln, Brayford Pool, Lincoln, UK

    Tomas Krajnik

  4. Dept. of Computer Science, Faculty of Electrical Engineering Czech Technical University in Prague, Technicka 2, 166 27, Prague, Czech Republic

    Jan Faigl

Authors
  1. Martin Saska
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Tomas Baca
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Justin Thomas
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jan Chudoba
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Libor Preucil
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Tomas Krajnik
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Jan Faigl
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Giuseppe Loianno
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Vijay Kumar
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Martin Saska.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Saska, M., Baca, T., Thomas, J. et al. System for deployment of groups of unmanned micro aerial vehicles in GPS-denied environments using onboard visual relative localization. Auton Robot 41, 919–944 (2017). https://doi.org/10.1007/s10514-016-9567-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10514-016-9567-z

Keywords

Search

Navigation