Skip to main content
SpringerLink
Log in

Delayed fusion for real-time vision-aided inertial navigation

  • Special Issue Paper
  • Published:
Journal of Real-Time Image Processing Aims and scope Submit manuscript

Abstract

In this paper, we consider the effects of delay caused by real-time image acquisition and feature tracking in a previously documented vision-augmented inertial navigation system. At first, the paper illustrates how delay caused by image processing, if not explicitly taken into account, can lead to appreciable performance degradation of the estimator. Next, three different existing methods of delayed fusion and a novel combined one are considered and compared. Simulations and Monte Carlo analyses are used to assess the estimation errors and computational effort of the various methods. Finally, a best performing formulation is identified that properly handles the fusion of delayed measurements in the estimator without increasing the time burden of the filter.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

References

  1. Bonin-Fontand, F., Ortiz, A., Oliver G.: Visual navigation for mobile robots: a survey. J. Intell. Rob. Syst. 53(3), 263–296 (2008)

    Article  Google Scholar 

  2. Dalgleish, F.R., Tetlow, J.W., Allwood, R.L.: Vision-based navigation of unmanned underwater vehicles : a survey. part 2: Vision-based station-keeping and positioning. In: IMAREST Proceedings, Part B: Journal of Marine Design and Operations, vol. 8, pp. 13–19 (2005)

  3. Liu, Y.C., Dai, Q.H.: Vision aided unmanned aerial vehicle autonomy : an overview. In: Image and signal processing, 3th International Congress on, pp. 417–421 (2010)

  4. Taylor, C.N.: Enabling navigation of mavs through inertial, vision, and air pressure sensor fusion. In: Hahn, H., Ko, H., Lee, S. (eds.) Multisensor fusion and integration for intelligent systems, lecture notes in electrical engineering, vol. 35, pp. 143–158 (2009)

  5. Chun, L., Fagen, Z., Yiwei, S., Kaichang, D., Zhaoqin, L.: Stereo-image matching using a speeded up robust feature algorithm in an integrated vision navigation system. J. Navig. 65, 671–692 (2012) doi:10.1017/S0373463312000264

    Article  Google Scholar 

  6. Nister, D., Naroditsky, O., Bergen, J.: Visual odometry for ground vehicle applications. J. Field. Rob. 23(1), 3–20 (2006)

    Article  MATH  Google Scholar 

  7. Goedeme, T., Nuttin, M., Tuytelaars, T., Gool, L.V.: Omnidirectional vision based topological navigation. Int. J. Comput. Vision. 74(3), 219–236 (2007)

    Article  Google Scholar 

  8. Roumeliotis, S.I., Johnson, A.E., Montgomery, J.F.: Augmenting inertial navigation with image-based motion estimation. In: Robotics and automation, IEEE International Conference on, pp. 4326–4333 (2002)

  9. Qian, G., Chellappa, R., Zheng, Q.: Robust structure from motion estimation using inertial data. J. Opt. Soc. Am. 18(12), 2982–2997 (2001)

    Article  Google Scholar 

  10. Veth, M.J., Raquet, J.F., Pachter, M.: Stochastic constraints for efficient image correspondence search. J. IEEE. Trans. Aerosp. Electron. Syst. 42(3), 973–982 (2006)

    Article  Google Scholar 

  11. Mourikis, A.I., Roumeliotis, S.I.: A multi-state constraint Kalman filter for vision-aided inertial navigation. In: Robotics and automation, IEEE International Conference on, pp. 3565–3572 (2007)

  12. Corato, F., Innocenti, M., Pollini, L.: Robust vision-aided inertial navigation algorithm via entropy-like relative pose estimation. Gyrosco. Navig. 4(1), 1–13 (2013). doi:10.1134/S2075108713010033

    Article  Google Scholar 

  13. Tardif, J.P., George, M., Laverne, M., Kelly, A., Stentz, A.: A new approach to vision-aided inertial navigation. In: Intelligent robots and systems (IROS), IEEE/RSJ International Conference on, pp. 4161–4168 (2010). doi:10.1109/IROS.2010.5651059

  14. Bottasso, C.L., Leonello, D.: Vision-augmented inertial navigation by sensor fusion for an autonomous rotorcraft vehicle. In: Unmanned Rotorcraft, AHS International Specialists Meeting on, pp. 324–334 (2009)

  15. Jones, E.S., Soatto, S.: Visual-inertial navigation, mapping and localization: a scalable real-time causal approach. Int. J. Robot. Res. 30(4), 407–430 (2011)

    Google Scholar 

  16. Ferreira, F.J., Lobo, J., Dias, J.: Bayesian real-time perception algorithms on GPU–real-time implementation of Bayesian models for multimodal perception using CUDA. J. Real-Time. Image Proc. 6(3), 171–186 (2011)

    Article  Google Scholar 

  17. Pornsarayouth, S., Wongsaisuwan, M.: Sensor fusion of delay and non-delay signal using Kalman filter with moving covariance. In: Robotics and biomimetics, IEEE International Conference on, pp. 2045–2049 (2009)

  18. Alexander, H.L.: State estimation for distributed systems with sensing delay. pp. 103–111 (1991) SPIE . doi:10.1117/12.44843

  19. Larsen, T.D., Andersen, N.A., Ravn, O., Poulsen, N.: Incorporation of time delayed measurements in a discrete-time Kalman filter. In: Decision and Control, 37th IEEE Conference on, pp. 3972–3977 (1998)

  20. Challa, S., Legg, J.A., Wang, X.: Track-to-track fusion of out-of-sequence tracks. In: Information Fusion, 2002. Fifth International Conference on, vol. 2, pp. 919–926 (2002)

  21. Bar-Shalom, Y., Li, X.R.: Multitarget-MultisensorTracking: principles and Techniques. YBS Publishing (1995)

  22. Challa, S., Evans, R.J., Wang, X., Legg, J.: A fixed-lag smoothing solution to out-of-sequence information fusion problems. Commun. Inform. Syst. 2(4), 325–348 (2002)

    Article  MATH  Google Scholar 

  23. Van Der Merwe, R.: Sigma-point kalman filters for probabilistic inference in dynamic state-space models. In: Ph.D Thesis, OGI School of Science and Engineering, Oregon Health and Science University (2004)

  24. Roumeliotis, S., Burdick, J.: Stochastic cloning: a generalized framework for processing relative state measurements. In: Robotics and Automation, IEEE International Conference on, vol. 2, pp. 1788–1795 (2002)

  25. Gopalakrishnan, A., Kaisare, N., Narasimhan, S.: Incorporating delayed and infrequent measurements in extended Kalman filter based nonlinear state estimation. J. Proc. Control. 21(1),119–129 (2011)

    Article  Google Scholar 

  26. Tatiraju, S., Soroush, S., Ogunnaike, B.A.: Multirate nonlinear state estimation with application to a polymerization reactor. AIChE J 45(4), 769–780 (1999)

    Article  Google Scholar 

  27. Stanway, M.J.: Delayed-state sigma point Kalman filters for underwater navigation. In: Autonomous Underwater Vehicles, IEEE/OES Conference on, pp. 1–9 (2010)

  28. Asadi, E., Bottasso, C.L.: Handling delayed fusion in vision-augmented inertial navigation. In: Informatics in Control, Automation and Robotics, 9th International Conference on, pp. 394–401 (2012)

  29. Jianbo, S., Tomasi, C.: Good features to track. In: Computer Vision and Pattern Recognition, IEEE Computer Society Conference on, pp. 593–600 (1994)

  30. Calonder, M., Lepetit, V., Strecha, C., Fua, P.: Brief: Binary robust independent elementary features. In: Computer Vision, 11th European Conference on, vol. 6314(3), pp. 778–792. LNCS Springer (2010)

  31. Schmidt, S.F.: Applications of state space methods to navigation problems, C. T. Leondes, advanced control systems edn. Academic Press, New York (1996)

  32. Junker, G.: Pro OGRE 3D Programming. Springer-Verlag, New York (2006)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Aerospace Science and Technology, Politecnico di Milano, Milano, Italy

    Ehsan Asadi & Carlo L. Bottasso

  2. Wind Energy Institute, Technische Universität München, Garching bei München, Germany

    Carlo L. Bottasso

Authors
  1. Ehsan Asadi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Carlo L. Bottasso
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ehsan Asadi.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Asadi, E., Bottasso, C.L. Delayed fusion for real-time vision-aided inertial navigation. J Real-Time Image Proc 10, 633–646 (2015). https://doi.org/10.1007/s11554-013-0376-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11554-013-0376-8

Keywords

Search

Navigation