Skip to main content
SpringerLink
Log in
Book cover

International Symposium on Experimental Robotics

ISER 2016: 2016 International Symposium on Experimental Robotics pp 775–786Cite as

Experimental Comparison of Open Source Vision-Based State Estimation Algorithms

  • Conference paper
  • First Online:

Part of the book series: Springer Proceedings in Advanced Robotics ((SPAR,volume 1))

Abstract

The problem of state estimation using primarily visual data has received a lot of attention in the last decade. Several open source packages have appeared addressing the problem, each supported by impressive demonstrations. Applying any of these packages on a new dataset however, has been proven extremely challenging. Suboptimal performance, loss of localization, and challenges in customization have not produced a clear winner. Several other research groups have presented superb performance without releasing the code, sometimes materializing as commercial products. In this paper, ten of the most promising open source packages are evaluated, by cross validating them on the datasets provided for each package and by testing them on eight different datasets collected over the years in our laboratory. Indoor and outdoor, terrestrial and flying vehicles, in addition to underwater robots, cameras, and buoys were used to collect data. An analysis on the motions required for the different approaches and an evaluation of their performance is presented.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   259.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   329.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD   329.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Notes

  1. 1.

    http://wiki.ros.org/Bags.

  2. 2.

    http://wiki.ros.org/gmapping.

References

  1. Agarwal, S., Mierle, K., Others: Ceres Solver (2015). http://ceres-solver.org

  2. Ball, D., Heath, S., Wiles, J., Wyeth, G., Corke, P., Milford, M.: OpenRatSLAM: an open source brain-based SLAM system. Auton. Robot. 34(3), 149–176 (2013)

    Article  Google Scholar 

  3. Boydstun, D., Farich III., M., J.M., Rubinson, S., Smith, Z., Rekleitis, I.: Drifter sensor network for environmental monitoring. In: 12th Conference on Computer Robot Vision, pp. 16–22, June 2015

    Google Scholar 

  4. Ceriani, S., Fontana, G., Giusti, A., Marzorati, D., Matteucci, M., Migliore, D., Rizzi, D., Sorrenti, D.G., Taddei, P.: RAWSEEDS ground truth collection systems for indoor self-localization and mapping. Auton Robot. 27(4), 353–371 (2009)

    Article  Google Scholar 

  5. Civera, J., Grasa, O.G., Davison, A.J., Montiel, J.M.M.: 1Point RANSAC for extended kalman filtering: application to real-time structure from motion and visual odometry. J. Field Robot. 27(5), 609–631 (2010)

    Article  Google Scholar 

  6. Concha, A., Civera, J.: DPPTAM: dense piecewise planar tracking and mapping from a monocular sequence. In: IEEE/RSJ International Conference on Intelligent Robots and Systems (2015)

    Google Scholar 

  7. Davison, A., Reid, I., Molton, N., Stasse, O.: MonoSLAM: real-time single camera SLAM. IEEE Trans. Pattern Anal. Mach. Intell. 29(6), 1052–1067 (2007)

    Article  Google Scholar 

  8. Dellaert, F., Kaess, M.: Square root SAM: simultaneous localization and mapping via square root information smoothing. Int. J. Robot. Res. 25(12), 1181–1203 (2006)

    Article  MATH  Google Scholar 

  9. Engel, J., Schöps, T., Cremers, D.: LSD-SLAM: large-scale direct monocular SLAM. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8690, pp. 834–849. Springer, Heidelberg (2014). doi:10.1007/978-3-319-10605-2_54

    Google Scholar 

  10. Forster, C., Pizzoli, M., Scaramuzza, D.: SVO: fast semi-direct monocular visual odometry. In: IEEE International Conference on Robotics and Automation, pp. 15–22 (2014)

    Google Scholar 

  11. Fraundorfer, F., Scaramuzza, D.: Visual odometry: part II: matching, robustness, optimization, and applications. IEEE Robot. Autom. Mag. 19(2), 78–90 (2012)

    Article  Google Scholar 

  12. Fuentes-Pacheco, J., Ruiz-Ascencio, J., Rendón-Mancha, J.M.: Visual simultaneous localization and mapping: a survey. Artif. Intell. Rev. 43, 55–81 (2015)

    Article  Google Scholar 

  13. Furgale, P.T., Barfoot, T.D.: Stereo mapping and localization for long-range path following on rough terrain. In: ICRA, pp. 4410–4416 (2010)

    Google Scholar 

  14. Geiger, A., Lenz, P., Stiller, C., Urtasun, R.: Vision meets robotics: the KITTI dataset. Int. J. Robot. Res. 32(11), 1231–1237 (2013)

    Article  Google Scholar 

  15. Geiger, A., Ziegler, J., Stiller, C.: Stereoscan: dense 3D reconstruction in real-time. In: Intelligent Vehicles Symposium (IV) (2011)

    Google Scholar 

  16. Hesch, J., Kottas, D., Bowman, S., Roumeliotis, S.: Consistency analysis and improvement of vision-aided inertial navigation. IEEE Trans. Robot. 30(1), 158–176 (2014)

    Article  Google Scholar 

  17. Jones, E.S., Soatto, S.: Visual-inertial navigation, mapping and localization: a scalable real-time causal approach. Int. J. Robot. Res. 30(4), 407–430 (2011)

    Article  Google Scholar 

  18. Kelly, J., Sukhatme, G.S.: Visual-inertial sensor fusion: localization, mapping and sensor-to-sensor self-calibration. Int. J. Robot. Res. 30(1), 56–79 (2011)

    Article  Google Scholar 

  19. Klein, G., Murray, D.: Parallel tracking and mapping for small ar workspaces. In: IEEE and ACM International Symposium on Mixed and Augmented Reality, pp. 225–234 (2007)

    Google Scholar 

  20. Konolige, K., Agrawal, M., Solà, J.: Large scale visual odometry for rough terrain. In: International Symposium on Research in Robotics (ISRR), November 2007

    Google Scholar 

  21. Kummerle, R., Grisetti, G., Strasdat, H., Konolige, K., Burgard, W.: \(g^2o\): a general framework for graph optimization. In: IEEE International Conference on Robotics and Automation, pp. 3607–3613 (2011)

    Google Scholar 

  22. Kümmerle, R., Steder, B., Dornhege, C., Ruhnke, M., Grisetti, G., Stachniss, C., Kleiner, A.: On measuring the accuracy of SLAM algorithms. Auton. Robot. 27(4), 387–407 (2009)

    Article  Google Scholar 

  23. Leutenegger, S., Lynen, S., Bosse, M., Siegwart, R., Furgale, P.: Keyframe-based visual-inertial odometry using nonlinear optimization. Int. J. Robot. Res. 34(3), 314–334 (2015)

    Article  Google Scholar 

  24. Lourakis, M.A., Argyros, A.: SBA: a software package for generic sparse bundle adjustment. ACM Trans. Math. Softw. 36(1), 1–30 (2009)

    Article  MathSciNet  Google Scholar 

  25. Mourikis, A.I., Roumeliotis, S.I.: A multi-state constraint Kalman filter for vision-aided inertial navigation. In: IEEE International Conference on Robotics and Automation, pp. 3565–3572. IEEE (2007)

    Google Scholar 

  26. Mur-Artal, R., Montiel, J.M.M., Tardós, J.D.: ORB-SLAM: a versatile and accurate monocular SLAM system. IEEE Trans. Robot. 31(5), 1147–1163 (2015)

    Article  Google Scholar 

  27. Newcombe, R.A., Lovegrove, S.J., Davison, A.J.: DTAM: dense tracking and mapping in real-time. In: International Conference on Computer Vision. ICCV, Computer Society, pp. 2320–2327. IEEE, Washington, DC (2011)

    Google Scholar 

  28. Scaramuzza, D., Fraundorfer, F.: Visual odometry [tutorial]. IEEE Robot. Autom. Mag. 18(4), 80–92 (2011)

    Article  Google Scholar 

  29. Schönberger, J.L., Frahm, J.M.: Structure-from-motion revisited. In: IEEE Conference on Computer Vision and Pattern Recognition (2016)

    Google Scholar 

  30. Snavely, N., Seitz, S.M., Szeliski, R.: Modeling the world from internet photo collections. Int. J. Comput. Vis. 80(2), 189–210 (2008)

    Article  Google Scholar 

  31. Triggs, B., McLauchlan, P.F., Hartley, R.I., Fitzgibbon, A.W.: Bundle adjustment – a modern synthesis. In: Vision Algorithms: Theory and Practice: International Workshop on Vision Algorithms, Corfu, Greece, pp. 298–372 (2000)

    Google Scholar 

  32. Williams, B., Cummins, M., Neira, J., Newman, P., Reid, I., Tardós, J.: A comparison of loop closing techniques in monocular SLAM. Robot. Autonom. Syst. 57(12), 1188–1197 (2009)

    Article  Google Scholar 

  33. Zhao, L., Huang, S., Sun, Y., Yan, L., Dissanayake, G.: Parallaxba: bundle adjustment using parallax angle feature parametrization. Int. J. Robot. Res. 34(4–5), 493–516 (2015)

    Article  Google Scholar 

Download references

Acknowledgment

The authors would like to thank the generous support of the Google Faculty Research Award and the National Science Foundation grants (NSF 0953503, 1513203, 1526862, 1637876).

Author information

Authors and Affiliations

  1. Computer Science and Engineering Department, University of South Carolina, 315 Main, Columbia, South Carolina, 29208, USA

    Alberto Quattrini Li, A. Coskun, S. M. Doherty, S. Ghasemlou, A. S. Jagtap, M. Modasshir, S. Rahman, A. Singh, M. Xanthidis, J. M. O’Kane & I. Rekleitis

Authors
  1. Alberto Quattrini Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. A. Coskun
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. S. M. Doherty
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. S. Ghasemlou
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. A. S. Jagtap
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. M. Modasshir
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. S. Rahman
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. A. Singh
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. M. Xanthidis
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. J. M. O’Kane
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. I. Rekleitis
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Alberto Quattrini Li .

Editor information

Editors and Affiliations

  1. Gra Sch of Info Sci&Tech,Dept of MechInf, The University of Tokyo Gra Sch of Info Sci&Tech,Dept of MechInf, Tokyo, Japan

    Dana Kulić

  2. Computer Science Department, Stanford University Computer Science Department, Stanford, California, USA

    Yoshihiko Nakamura

  3. Department of Electrical & Computer Engg, University of Waterloo Department of Electrical & Computer Engg, Waterloo, Ontario, Canada

    Oussama Khatib

  4. Department of Mechanical Systems Enginee, Tokyo University of Agriculture and Tech Department of Mechanical Systems Enginee, Tokyo, Japan

    Gentiane Venture

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing AG

About this paper

Cite this paper

Quattrini Li, A. et al. (2017). Experimental Comparison of Open Source Vision-Based State Estimation Algorithms. In: Kulić, D., Nakamura, Y., Khatib, O., Venture, G. (eds) 2016 International Symposium on Experimental Robotics. ISER 2016. Springer Proceedings in Advanced Robotics, vol 1. Springer, Cham. https://doi.org/10.1007/978-3-319-50115-4_67

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-50115-4_67

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-50114-7

  • Online ISBN: 978-3-319-50115-4

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation