Skip to main content
SpringerLink
Log in

A tracking solution for mobile augmented reality based on sensor-aided marker-less tracking and panoramic mapping

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

This paper proposes a tracking system for outdoor augmented reality (AR) on handheld devices based on an integration of vision tracking and on-device sensor measurement. To deal with the unpredictable and complex visual information in an outdoor environment, two tracking schemes are proposed for both near-field and far-field tracking scenarios. A sensor-aided binary descriptor is combined with an intensity-based tracking algorithm to deliver a 3D tracking system for fronto-parallel planar surfaces in near-field tracking. In far-field tracking, a sensor-guided panoramic tracking and mapping approach is proposed which allows a creation of the panorama of distant scenes on the fly with camera rotation motion to be tracked at the same time. This implementation allows near real-time creation of panoramic maps on-device; therefore, the users are able to tag information on the training target instantly.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

References

  1. Bay H, Tuytelaars T, Gool LV (2008) SURF: speed up robust features. Comp Vision Image Underst (CVIU) 110(3):346–359

    Article  Google Scholar 

  2. Benhimane S, Malis E (2007) Homography-based 2D visual tracking and servoing. Int J Robot Res 26(7):661–676

    Article  Google Scholar 

  3. Bleser G, Stricker D (2008) Advanced tracking through efficient image processing and visual-inertial sensor fusion. In Virtual Reality Conference 2008 (VR’08), NE, USA, pp. 137–144

  4. Calonder MV, Lepetit V, Strecha C, Fua P (2010) BRIEF: binary robust independent elementary features. In 11th European Conference on Computer Vision, Heraklion, Greece, pp. 778–792

  5. Feiner S, Macintype B, Hollerer T, Webster T (1997) A touring machine: prototyping 3D mobile augmented reality systems for exploring the urban environment. In Proc. Of International Symposium on Wearable Computers (ISWC), Cambridge, Massachusetts, 13–14 Oct, pp. 74–81

  6. Fischler MA, Bolles RC (1981) Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun of ACM 24(6):381–395

    Article  MathSciNet  Google Scholar 

  7. Gammeter S, Gassmann A, Bossard L (2010) Server-side object recognition and client-side object tracking for mobile augmented reality. In 2010 I.E. Computer Society Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), San Francisco, USA, pp. 1–8

  8. Hwangbo M, Kim J, Kanade T (2009) Inertial-aided KLT feature tracking for a moving camera. In IEEE/RSJ International Conference on Intelligent Robots and Systems, 2009, St. Louis, USA, pp. 1909–1916

  9. Kurz D, Benhimane S (2011) Inertial sensor-aligned visual feature descriptors. In 2011 I.E. Conference on Computer Vision and Pattern Recognition (CVPR’11), Colorado, USA, pp. 161–166

  10. Lee W, Park Y, Lepetit V, Woo W (2010) Point-and-shoot for ubiquitous tagging on mobile phones. In IEEE International Symposium on Mixed and Augmented Reality (ISMAR’10), Seoul, Korea, pp. 57–64

  11. Lin L, Wang Y, Liu Y, Xiong C, Zeng K (2009) Marker-less registration based on template tracking for augmented reality. Multimed Tools Appl J(MMTA) 41(2):235–252

    Article  Google Scholar 

  12. Lowe DG (2004) Distinctive image features from scale-invariant keypoints. Int J Comput Vis 60(2):91–110

    Article  Google Scholar 

  13. Özuysal M, Fua P, Lepetit V (2007) Fast keypoint recognition in ten lines of code. In 2007 I.E. Conference on Computer Vision and Pattern Recognition (CVPR’07), Minneapolis, USA, pp. 1–8

  14. Reitmayr G, Drummond T (2006) Going out: robust model-based tracking for outdoor augmented reality. In Proc. Of International Symposium on Mixed and Augmented Reality (ISMAR ‘06), Santa Barbara, 22–25 Oct, pp. 109–118, 2006

  15. Rosten E, Drummond T (2006) Machine learning for high-speed corner detection. In Proceedings of European Conference on Computer Vision, Graz, Austria, pp 430–443

    Google Scholar 

  16. Rublee E, Rabaud V, Knolige K, Bradski G (2011) ORB: an efficient alternative to SITF or SURF. In 2011 I.E. International Conference on Computer Vision (ICCV’11), Barcelona, Spain, pp. 2564–2571

  17. Schall G, Wagner D, Reitmayr G, Taichmann E, Wieser M, Schmalstieg D, Hofmann WB (2009) Global pose estimation using multi-sensor fusion for outdoor augmented reality. In 2009 I.E. International Symposium on Mixed and Augmented Reality (ISMAR’09), Orlando, USA, pp. 153–162

  18. Silveira G, Malis E (2007) Real-time visual tracking under arbitrary illumination changes. Proceedings of IEEE Computer Vision and Pattern Recognition (CVPR), Minneapolis, pp 1–6

    Google Scholar 

  19. Ventura J, Hollerer T (2012) Wide-area scene mapping for mobile visual tracking. In IEEE International Symposium on Mixed and Augmented Reality (ISMAR’12), Atlanta, 5–8 Nov, pp 3–12

  20. Wagner D, Mulloni A, Langlotz T, Schmalstieg D (2010) Real-time panoramic mapping and tracking on mobile phones. In 2010 I.E. Virtual Reality Conference (VR’10), Massachusetts, USA, pp. 211–218

  21. Wagner D, Reitmayr G, Mulloni A, Drummond T, Schmalstieg D (2010) Real-time detection and tracking for augmented reality on mobile phones. IEEE Trans Vis Comput Graph 16(3):355–368

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Mechanical Engineering Department, National University of Singapore, 9 Engineering Drive 1, Singapore, 117576, Singapore

    L. Yu, S. K. Ong & A. Y. C. Nee

Authors
  1. L. Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. S. K. Ong
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. A. Y. C. Nee
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to S. K. Ong.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yu, L., Ong, S.K. & Nee, A.Y.C. A tracking solution for mobile augmented reality based on sensor-aided marker-less tracking and panoramic mapping. Multimed Tools Appl 75, 3199–3220 (2016). https://doi.org/10.1007/s11042-014-2430-3

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-014-2430-3

Keywords

Search

Navigation