Skip to main content
SpringerLink
Log in

Optimal seamline detection in dynamic scenes via graph cuts for image mosaicking

  • Original Paper
  • Published:
Machine Vision and Applications Aims and scope Submit manuscript

Abstract

In this paper, we present a novel method for creating a seamless mosaic from a set of geometrically aligned images captured from the scene with dynamic objects at different times. The artifacts caused by dynamic objects and geometric misalignments can be effectively concealed in our proposed seamline detection algorithm. In addition, we simultaneously compensate the image regions of dynamic objects based on the optimal seamline detection in the graph cuts energy minimization framework and create the mosaic with a relatively clean background. To ensure the high quality of the optimal seamline, the energy functions adopted in graph cuts combine the pixel-level similarities of image characteristics, including intensity and gradient, and the texture complexity. To successfully compensate the image regions covered by dynamic objects for creating a mosaic with a relatively clean background, we initially detect them in overlap regions between images based on pixel-level and region-level similarities, then refine them based on segments, and determine their image source in probability based on contour matching. We finally integrate all of these into the energy minimization framework to detect optimal seamlines. Experimental results on different dynamic scenes demonstrate that our proposed method is capable of generating high-quality mosaics with relatively clean backgrounds based on the detected optimal seamlines.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

Notes

  1. http://panotools.sourceforge.net/.

  2. http://www.ptgui.com/.

  3. http://hugin.sourceforge.net/.

  4. http://enblend.sourceforge.net/.

References

  1. Achanta, R., Shaji, A., Smith, K., Lucchi, A., Fua, P., Süsstrunk, S.: SLIC superpixels compared to state-of-the-art superpixel methods. IEEE Trans. Pattern Anal. Mach. Intell. 34(11), 2274–2282 (2012)

    Article  Google Scholar 

  2. Bellman, R.: Dynamic Programming. Princeton University Press, Princeton (1957)

    MATH  Google Scholar 

  3. Boutellier, J.J., Bordallo-Lopez, M., Silvén, O., Tico, M., Vehviläinen, M.: Creating panoramas on mobile phones. In: Proceeding of SPIE Electronic Imaging, vol. 6498, issue 7 (2007)

  4. Boykov, Y., Kolmogorov, V.: An experimental comparison of min-cut/max-flow algorithms for energy minimization in vision. IEEE Trans. Pattern Anal. Mach. Intell. 26(9), 1124–1137 (2004)

    Article  MATH  Google Scholar 

  5. Boykov, Y., Veksler, O., Zabih, R.: Fast approximate energy minimization via graph cuts. IEEE Trans. Pattern Anal. Mach. Intell. 23(11), 1222–1239 (2001)

    Article  Google Scholar 

  6. Brown, M., Lowe, D.G.: Automatic panoramic image stitching using invariant features. Int. J. Comput. Vis. 74(1), 59–73 (2007)

    Article  Google Scholar 

  7. Comaniciu, D., Meer, P.: Mean shift: a robust approach toward feature space analysis. IEEE Trans. Pattern Anal. Mach. Intell. 24(5), 603–619 (2002)

    Article  Google Scholar 

  8. Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR) (2005)

  9. Davis, J.: Mosaics of scenes with moving objects. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR) (1998)

  10. Dijkstra, E.W.: A note on two problems in connexion with graphs. Numer. Math. 1(1), 269–271 (1959)

    Article  MATH  MathSciNet  Google Scholar 

  11. Dissanayake, V., Herath, S., Rasnayaka, S., Seneviratne, S., Vidanaarachchi, R., Gamage, C.: Quantitative and qualitative evaluation of performance and robustness of image stitching algorithms. In: International Conference on Digital Image Computing: Techniques and Applications (DICTA) (2015)

  12. Hsieh, J.-W.: Fast stitching algorithm for moving object detection and mosaic construction. Image Vis. Comput. 22(4), 291–306 (2004)

    Article  Google Scholar 

  13. Kass, M., Witkin, A.: Snakes: active contour models. Int. J. Comput. Vis. 1(4), 321C–331 (1988)

    Article  Google Scholar 

  14. Kim, D.-W., Hong, K.-S.: Practical background estimation for mosaic blending with patch-based markov random fields. Pattern Recognit. 41(7), 2145–2155 (2008)

    Article  MATH  Google Scholar 

  15. Kolmogorov, V., Zabih, R.: Computing visual correspondence with occlusions using graph cuts. In: IEEE International Conference on Computer Vision (ICCV) (2001)

  16. Levin, A., Zomet, A., Peleg, S., Weiss, Y.: Seamless image stitching in the gradient domain. In: European Conference on Computer Vision (ECCV) (2004)

  17. Li, L., Yao, J., Liu, Y., Yuan, W., Shi, S., Yuan, S.: Optimal seamline detection for orthoimage mosaicking by combining deep convolutional neural network and graph cuts. Remote Sens. 9(7), 701 (2017a)

    Article  Google Scholar 

  18. Li, L., Yao, J., Xie, R., Xia, M., Zhang, W.: A unified framework for street-view panorama stitching. Sensors 17(1), 1 (2017b)

    Article  Google Scholar 

  19. López, M.B., Hannuksela, J., Silvén, O., Vehviläinen, M.: Interactive multi-frame reconstruction for mobile devices. Multimed. Tools Appl. 69(1), 31–51 (2014)

    Article  Google Scholar 

  20. Mills, A., Dudek, G.: Image stitching with dynamic elements. Image Vis. Comput. 27(10), 1593–1602 (2009)

    Article  Google Scholar 

  21. Philip, S., Summa, B., Tierny, J., Bremer, P.-T., Pascucci, V.: Distributed seams for gigapixel panoramas. IEEE Trans. Vis. Comput. Graph. 21(3), 350–362 (2015)

    Article  Google Scholar 

  22. Qureshi, H., Khan, M., Hafiz, R., Cho, Y., Cha, J.: Quantitative quality assessment of stitched panoramic images. IET Image Proc. 6(9), 1348–1358 (2012)

    Article  MathSciNet  Google Scholar 

  23. Rother, C., Kolmogorov, V., Blake, A.: Grabcut: Interactive foreground extraction using iterated graph cuts. In: ACM Transactions on Graphics (TOG) (Proceedings of SIGGRAPH 2004) (2004)

  24. Tao, C., Sun, H., Yang, C., Tian, J.: Efficient image stitching in the presence of dynamic objects and structure misalignment. J. Signal Inf. Proc. 2(3), 205 (2011)

    Google Scholar 

  25. Tao, M.W., Johnson, M.K., Paris, S.: Error-tolerant image compositing. Int. J. Comput. Vis. 103(2), 178–189 (2013)

    Article  MATH  Google Scholar 

  26. Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004)

    Article  Google Scholar 

  27. Xia, M., Yao, J., Xie, R., Li, L., Zhang, W.: Globally consistent alignment for planar mosaicking via topology analysis. Pattern Recognit. 66, 239–252 (2017)

    Article  Google Scholar 

  28. Xu, W., Mulligan, J.: Performance evaluation of color correction approaches for automatic multi-view image and video stitching. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2010)

  29. Zeng, L., Zhang, S., Zhang, J., Zhang, Y.: Dynamic image mosaic via SIFT and dynamic programming. Mach. Vis. Appl. 25(5), 1271–1282 (2014)

    Article  Google Scholar 

  30. Zhi, Q., Cooperstock, J.R.: Toward dynamic image mosaic generation with robustness to parallax. IEEE Trans. Image Process. 21(1), 366–378 (2012)

    Article  MATH  MathSciNet  Google Scholar 

  31. Zhou, Z., Li, S., Wang, B.: Multi-scale weighted gradient-based fusion for multi-focus images. Inf. Fusion 20(1), 60–72 (2014)

    Article  Google Scholar 

  32. Zhu, Q., Yeh, M.-C., Cheng, K.-T., Avidan, S.: Fast human detection using a cascade of histograms of oriented gradients. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR) (2006)

Download references

Acknowledgements

This work was partially supported by the National Natural Science Foundation of China (Project No. 41571436), the Hubei Province Science and Technology Support Program, China (Project No. 2015BAA027), the National Natural Science Foundation of China under Grant 91438203, LIESMARS Special Research Funding, and the South Wisdom Valley Innovative Research Team Program.

Author information

Authors and Affiliations

  1. Computer Vision and Remote Sensing (CVRS) Lab, School of Remote Sensing and Information Engineering, Wuhan University, Wuhan, Hubei, People’s Republic of China

    Li Li, Jian Yao, Haoang Li & Menghan Xia

  2. School of Control Science and Engineering, Shandong University, Jinan, Shandong, People’s Republic of China

    Wei Zhang

Authors
  1. Li Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jian Yao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Haoang Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Menghan Xia
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Wei Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jian Yao.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, L., Yao, J., Li, H. et al. Optimal seamline detection in dynamic scenes via graph cuts for image mosaicking. Machine Vision and Applications 28, 819–837 (2017). https://doi.org/10.1007/s00138-017-0874-y

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00138-017-0874-y

Keywords

Search

Navigation