Skip to main content
Log in

Accurate sensing of scene geo-context via mobile visual localization

  • Regular Paper
  • Published:
Multimedia Systems Aims and scope Submit manuscript

Abstract

Image geo-tagging has drawn a great deal of attention in recent years. The geographic information associated with images can be used to promote potential applications such as location recognition or virtual navigation. In this paper, we propose a novel approach for accurate mobile image geo-tagging in urban areas. The approach is able to provide a comprehensive set of geo-context information based on the current image, including the real location of the camera and the viewing angle, as well as the location of the captured scene. Moreover, the parsed building facades and their geometric structures can also be estimated. First, for the image to be geo-tagged, we perform partial duplicate image retrieval to filter crowd-sourced images capturing the same scene. We then employ the structure-from-motion technique to reconstruct a sparse 3D point cloud of the scene. Meanwhile, the geometric structure of the query image is analyzed to extract building facades. Finally, by combining the reconstructed 3D scene model and the extracted structure information, we can register the camera location and viewing direction to a real-world map. The captured building location and facade orientation are also aligned. The effectiveness of the proposed system is demonstrated by experiment results.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Notes

  1. http://www.stanford.edu/~dmchen/mvs.html.

References

  1. Avrithis, Y., Kalantidis, Y., Tolias, G., Spyrou, E.: Retrieving landmark and non-landmark images from community photo collections. In ACM Multimedia (2010)

  2. Bing street side http://www.microsoft.com/maps/streetside.aspx

  3. Chen, D., Baatz, G., Koser, K., Tsai, S., Vedantham, R., Pylvanainen, T., Roimela, K., Chen, X., Bach, J., Pollefeys, M., et al.: City-scale landmark identification on mobile devices. In CVPR, (2011)

  4. Cheng, Z., Ren, J., Shen, J., Miao, H.: Building a large scale test collection for effective benchmarking of mobile landmark search. In Advances in Multimedia Modeling, Springer, pp. 36–46, (2013)

  5. Flickr http://www.flickr.com/

  6. Girod, B., Chandrasekhar, V., Chen, D., Cheung, N., Grzeszczuk, R., Reznik, Y., Takacs, G., Tsai, S., Vedantham, R.: Mobile visual search. Signal Processing Magazine (2011)

  7. Google street view http://www.google.com/streetview

  8. Hays, J., Efros, A.: Im2gps: estimating geographic information from a single image. In CVPR (2008)

  9. Ji, R., Duan, L., Chen, J., Yao, H., Rui, Y., Chang, S., Gao, W.: Towards low bit rate mobile visual search with multiple-channel coding. In ACM Multimedia (2011)

  10. Ji, R., Duan, L., Chen, J., Yao, H., Yuan, J., Rui, Y., Gao, W.: Location discriminative vocabulary coding for mobile landmark search. IJCV (2012)

  11. Kroepfl, M., Wexler, Y., Ofek, E.: Efficiently locating photographs in many panoramas. In GIS (2010)

  12. Liu, H., Mei, T., Luo, J., Li, H., Li, S.: Finding perfect rendezvous on the go: accurate mobile visual localization and its applications to routing. In Proceedings of ACM Multimedia, pp. 9–18 (2012)

  13. Lowe, D.: Distinctive image features from scale-invariant keypoints. IJCV (2004)

  14. Luo, Z., Li, H., Tang, J., Hong, R., Chua, T.: Viewfocus: explore places of interests on Google maps using photos with view direction filtering. In ACM Multimedia (2009)

  15. Mastin, A., Kepner, J., Fisher, J.: Automatic registration of lidar and optical images of urban scenes. In Computer Vision and Pattern Recognition, 2009. CVPR 2009. IEEE Conference on, pp. 2639–2646. IEEE (2009)

  16. Muja M., Lowe, D.G.: Fast approximate nearest neighbors with automatic algorithm configuration. In VISSAPP’09, pp. 331–340, (2009)

  17. Nistér, D.: An efficient solution to the five-point relative pose problem. PAMI (2004)

  18. Nister, D., Stewenius, H.: Scalable recognition with a vocabulary tree. In CVPR (2006)

  19. Park, M., Luo, J., Collins, R., Liu, Y.: Beyond GPS: determining the camera viewing direction of a geotagged image. In ACM Multimedia (2010)

  20. Schindler, G., Brown, M., Szeliski, R.: City-scale location recognition. In CVPR, (2007)

  21. Snavely, N., Seitz, S., Szeliski, R.: Photo tourism: exploring photo collections in 3d. In TOG, (2006)

  22. Snavely N., Seitz S.M., Szeliski R. (2008) Modeling the world from internet photo collections. Intern J Comp Vision 80(2):189–210

    Article  Google Scholar 

  23. Szeliski, R.: Image alignment and stitching: a tutorial. Foundations and Trends® in Computer Graphics and Vision 2(1):1–104, (2006)

  24. Wang, S., Min, J., Yi, B.: Location based services for mobiles: technologies and standards. In IEEE International Conference on Communication (ICC), pp. 35–38 (2008)

  25. Wendel, A., Donoser, M., Bischof, H.: Unsupervised facade segmentation using repetitive patterns. Pattern Recognition, pp. 51–60 (2010)

  26. Wu, C., Frahm, J., Pollefeys, M.: Detecting large repetitive structures with salient boundaries. Computer Vision-ECCV 2010, pp. 142–155, (2010)

  27. Xu, X., Mei, T., Zeng, W., Yu, N., Luo, J.: Amigo: accurate mobile image geotagging. In Proceedings of International Conference on Internet Multimedia Computing and Service, pp. 11–14 (2012)

  28. Zhang, S., Yang, M., Cour, T., Yu, K., Metaxas, D.N.: Query specific fusion for image retrieval. In Computer Vision-ECCV 2012, Springer, pp. 660–673 (2012)

  29. Zhang, W., Kosecka, J.: Image based localization in urban environments. In 3DPVT (2006)

  30. Zhou, W., Lu, Y., Li, H., Song, Y., Tian, Q.: Spatial coding for large scale partial-duplicate web image search. In ACM Multimedia (2010)

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Heng Liu.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Liu, H., Li, H., Mei, T. et al. Accurate sensing of scene geo-context via mobile visual localization. Multimedia Systems 21, 255–265 (2015). https://doi.org/10.1007/s00530-013-0344-y

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00530-013-0344-y

Keywords

Navigation