Skip to main content
SpringerLink
Log in

Parallax-Aware Image Stitching Based on Homographic Decomposition

  • Conference paper
  • First Online:
Pattern Recognition (DAGM GCPR 2023)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 14264))

Included in the following conference series:

  • 101 Accesses

Abstract

Image stitching plays a crucial role for various computer vision applications, like panoramic photography, video production, medical imaging and satellite imagery. It makes it possible to align two images captured at different views onto a single image with a wider field of view. However, for 3D scenes with high depth complexity and images captured from two different positions, the resulting image pair may exhibit significant parallaxes. Stitching images with multiple or large apparent motion shifts remains a challenging task, and existing methods often fail in such cases. In this paper, a novel image stitching pipeline is introduced, addressing the aforementioned challenge: First, iterative dense feature matching is performed, which results in a multi-homography decomposition. Then, this output is used to compute a per-pixel multidimensional weight map of the estimated homographies for image alignment via weighted warping. Additionally, the homographic image space decomposition is exploited using combinatorial analysis to identify parallaxes, resulting in a parallax-aware overlapping region: Parallax-free overlapping areas only require weighted warping and blending. For parallax areas, these operations are omitted to avoid ghosting artifacts. Instead, histogram- and mask-based color mapping is performed to ensure visual color consistency. The presented experiments demonstrate that the proposed method provides superior results regarding precision and handling of parallaxes.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 79.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 99.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Acharya, T., Ray, A.K.: Image Processing - Principles and Applications (2005)

    Google Scholar 

  2. Chang, C.H., Sato, Y., Chuang, Y.Y.: Shape-preserving half-projective warps for image stitching. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 3254–3261 (2014)

    Google Scholar 

  3. Chen, Y.-S., Chuang, Y.-Y.: Natural image stitching with the global similarity prior. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9909, pp. 186–201. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46454-1_12

    Chapter  Google Scholar 

  4. Chen, Y., Xue, W., Chen, S.: Large parallax image stitching via structure preservation and multi-matching. In: Zhang, H., et al. (eds.) NCAA 2022. CCIS, vol. 1638, pp. 177–191. Springer, Singapore (2022). https://doi.org/10.1007/978-981-19-6135-9_14

    Chapter  Google Scholar 

  5. Gao, J., Kim, S.J., Brown, M.S.: Constructing image panoramas using dual-homography warping. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 49–56 (2011)

    Google Scholar 

  6. Gao, J., Li, Y., Chin, T.J., Brown, M.S.: Seam-driven image stitching. In: Eurographics (2013)

    Google Scholar 

  7. Hartley, R., Zisserman, A.: Multiple View Geometry in Computer Vision (2003)

    Google Scholar 

  8. Hirschmuller, H., Scharstein, D.: Evaluation of cost functions for stereo matching. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–8 (2007)

    Google Scholar 

  9. Jia, Q., Li, Z., Fan, X., Zhao, H., Teng, S., Ye, X., Latecki, L.J.: Leveraging line-point consistence to preserve structures for wide parallax image stitching. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 12181–12190 (2021)

    Google Scholar 

  10. Lee, K.Y., Sim, J.Y.: Stitching for multi-view videos with large parallax based on adaptive pixel warping. IEEE Access 6, 26904–26917 (2018)

    Article  Google Scholar 

  11. Lee, K.Y., Sim, J.Y.: Warping residual based image stitching for large parallax. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 8195–8203 (2020)

    Google Scholar 

  12. Li, J., Wang, Z., Lai, S., Zhai, Y., Zhang, M.: Parallax-tolerant image stitching based on robust elastic warping. IEEE Trans. Multimed. 20, 1672–1687 (2018)

    Article  Google Scholar 

  13. Li, N., Xu, Y., Wang, C.: Quasi-homography warps in image stitching. IEEE Trans. Multimed. 20, 1365–1375 (2017)

    Article  Google Scholar 

  14. Liao, T., Li, N.: Single-perspective warps in natural image stitching. IEEE Trans. Image Process. 29, 724–735 (2020)

    Article  MathSciNet  Google Scholar 

  15. Lin, C.C., Pankanti, S.U., Ramamurthy, K.N., Aravkin, A.Y.: Adaptive as-natural-as-possible image stitching. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1155–1163 (2015)

    Google Scholar 

  16. Lin, K., Jiang, N., Cheong, L.-F., Do, M., Lu, J.: SEAGULL: seam-guided local alignment for parallax-tolerant image stitching. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9907, pp. 370–385. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46487-9_23

    Chapter  Google Scholar 

  17. Liu, W.X., Chin, T.J.: Correspondence insertion for as-projective-as-possible image stitching. ArXiv (2016)

    Google Scholar 

  18. Lyu, W., Zhou, Z., Chen, L., Zhou, Y.: A survey on image and video stitching. Virtual Reality Intell. Hardw. 1, 55–83 (2019)

    Article  Google Scholar 

  19. Megha, V., Rajkumar, K.K.: Automatic satellite image stitching based on speeded up robust feature. In: International Conference on Artificial Intelligence and Machine Vision, pp. 1–6 (2021)

    Google Scholar 

  20. Nie, L., Lin, C., Liao, K., Liu, M., Zhao, Y.: A view-free image stitching network based on global homography. J. Vis. Commun. Image Representation 73, 102950 (2020)

    Article  Google Scholar 

  21. Nie, L., Lin, C., Liao, K., Liu, S., Zhao, Y.: Unsupervised deep image stitching: reconstructing stitched features to images. IEEE Trans. Image Process. 30, 6184–6197 (2021)

    Article  MathSciNet  Google Scholar 

  22. Raguram, R., Chum, O., Pollefeys, M., Matas, J., Frahm, J.M.: USAC: a universal framework for random sample consensus. IEEE Trans. Pattern Anal. Mach. Intell. 35, 2022–2038 (2013)

    Article  Google Scholar 

  23. Scharstein, D., Pal, C.: Learning conditional random fields for stereo. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–8 (2007)

    Google Scholar 

  24. Seibt, S., Von Rymon Lipinski, B., Latoschik, M.E.: Dense feature matching based on homographic decomposition. IEEE Access 10, 21236–21249 (2022)

    Article  Google Scholar 

  25. Szeliski, R.: Image alignment and stitching: a tutorial. Found. Trends Comput. Graph. Vision 2, 1–104 (2006)

    MathSciNet  Google Scholar 

  26. Wang, Z., Bovik, A., Sheikh, H., Simoncelli, E.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13, 600–612 (2004)

    Article  Google Scholar 

  27. Win, K.P., Kitjaidure, Y., Hamamoto, K.: Automatic stitching of medical images using feature based approach. Adv. Sci. Technol. Eng. Syst. J. 4, 127–133 (2019)

    Article  Google Scholar 

  28. Xue, W., Xie, W., Zhang, Y., Chen, S.: Stable linear structures and seam measurements for parallax image stitching. IEEE Trans. Circuits Syst. Video Technol. 32, 253–261 (2022)

    Article  Google Scholar 

  29. Zaragoza, J., Chin, T.J., Brown, M.S., Suter, D.: As-projective-as-possible image stitching with moving DLT. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 2339–2346 (2013)

    Google Scholar 

  30. Zhang, F., Liu, F.: Parallax-tolerant image stitching. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 3262–3269 (2014)

    Google Scholar 

  31. Zhang, G., He, Y., Chen, W., Jia, J., Bao, H.: Multi-viewpoint panorama construction with wide-baseline images. IEEE Trans. Image Process. 25, 3099–3111 (2016)

    Article  MathSciNet  Google Scholar 

  32. Zhao, Q., Ma, Y., Zhu, C., Yao, C., Feng, B., Dai, F.: Image stitching via deep homography estimation. Neurocomputing 450, 219–229 (2021)

    Article  Google Scholar 

  33. Zhao, Q., Wan, L., Feng, W., Zhang, J., Wong, T.T.: Cube2video: navigate between cubic panoramas in real-time. IEEE Trans. Multimed. 15, 1745–1754 (2013)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Game Tech Lab, Faculty of Computer Science, Nuremberg Institute of Technology, Nuremberg, Germany

    Simon Seibt, Michael Arold & Bartosz von Rymon Lipinski

  2. Institute for Applied Computer Science, Nuremberg Institute of Technology, Nuremberg, Germany

    Uwe Wienkopf

  3. Human-Computer Interaction Group, Institute of Computer Science, University of Wuerzburg, Wuerzburg, Germany

    Marc Erich Latoschik

Authors
  1. Simon Seibt
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Michael Arold
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Bartosz von Rymon Lipinski
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Uwe Wienkopf
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Marc Erich Latoschik
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Simon Seibt .

Editor information

Editors and Affiliations

  1. IWR, Heidelberg University, Heidelberg, Germany

    Ullrich Köthe

  2. IWR, Heidelberg University, Heidelberg, Germany

    Carsten Rother

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Seibt, S., Arold, M., von Rymon Lipinski, B., Wienkopf, U., Latoschik, M.E. (2024). Parallax-Aware Image Stitching Based on Homographic Decomposition. In: Köthe, U., Rother, C. (eds) Pattern Recognition. DAGM GCPR 2023. Lecture Notes in Computer Science, vol 14264. Springer, Cham. https://doi.org/10.1007/978-3-031-54605-1_13

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-54605-1_13

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-54604-4

  • Online ISBN: 978-3-031-54605-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation