Skip to main content
SpringerLink
Log in

A Variational Optical Flow Model for Accurate Motion Estimation from Rotational Image Sequences

  • Original Research
  • Published:
SN Computer Science Aims and scope Submit manuscript

Abstract

There are several optical flow models which aim to capture spatial characteristics of the flow such as divergence and curl. However accurate estimation is often a key challenge. In this context, we propose a variational optical flow model motivated by the harmonic-constraint based regularization for improving the estimation of rotation in motion. Our model is the standard Horn and Schunck model with an additional constraint penalizing the curl of the flow with an anisotropic weight term. Our implementation scheme is based on the robust Chambolle-Pock primal-dual algorithm. Further, to refine the flow-edges, we use the heuristic of weighted median filtering in our algorithm as a post-processing step. The results of our model indicate that it is comparable to, and in some cases, outperform some of the top-performing variational methods across multiple datasets in both angular and end-point errors.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Algorithm 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

Data Availability

The image sequences (data) used for this study are publicly available from the following sources: https://github.com/Tianshu-Liu/OpenOpticalFlow as a supplementary material to [21]. https://www.vision.middlebury.edu/flow as a supplementary material to [4]. http://visual.cs.ucl.ac.uk/pubs/flowConfidence/supp/ as a supplementary material to [1]. http://sintel.is.tue.mpg.de/downloads as a supplementary material to [10].

References

  1. Aoudha MO, Humayun A, Pollefeys M, Brostow GJ. Learning a confidence measure for optical flow. IEEE Trans Pattern Anal Mach Intell. 2013;35(5):1107–20. https://doi.org/10.1109/TPAMI.2012.171.

    Article  Google Scholar 

  2. Aoudha MO, Humayun A, Pollefeys M, Brostow GJ, UCL Datasets Project Page. http://visual.cs.ucl.ac.uk/pubs/flowConfidence/supp/. Accessed December 2023.

  3. Aubert G, Deriche R, Kornprobst P. Computing optical flow via variational techniques. SIAM J Appl Math. 1999;60:156–82. https://doi.org/10.1137/S0036139998340170.

    Article  MathSciNet  Google Scholar 

  4. Baker S, Scharstein D, Lewis J, Roth S, Black MJ, Szeliski R, A database and evaluation methodology for optical flow. In: IEEE international comference on computer vision. 2007, p. 1–8. https://www.doi.org/10.1109/ICCV.2007.4408903

  5. Baker S, Scharstein D, Lewis J, Roth S, Black MJ, Szeliski R, Middlebury stereo vision page. www.vision.middlebury.edu/flow. Accessed 21 Aug 2022.

  6. Barron JL, Fleet DJ, Beauchemin SS. Performance of optical flow techniques. Int J Comput Vis. 1994;12(1):43–77. https://doi.org/10.1007/BF01420984.

    Article  Google Scholar 

  7. Black MJ, Sapiro G, Marimont DH, Heeger D. Robust anisotropic diffusion. IEEE Trans Image Process. 1998;7(3):421–32.

    Article  Google Scholar 

  8. Brox T, Bruhn A, Papenberg N, Weickert J, High accuracy optical flow estimation based on a theory of warping. In: European conference on computer vision, Lecture notes in computer science. Berlin: Springer; 2004, Vol. 3024, p. 25–36. https://doi.org/10.1007/978-3-540-24673-2_3

  9. Bruhn A, Weickert J, Toward ultimate motion estimation: combining highest accuracy with real-time performance. In: IEEE international conference on computer vision, 2005; p. 749–55. https://doi.org/10.1109/ICCV.2005.240

  10. Buttler DJ, Wulff J, Stanley BG, Black MJ. A naturalistic open source movie for optical flow evaluation. Eur Conf Comput Vis. 2012;7577:611–25. https://doi.org/10.1007/978-3-642-33783-3_44.

    Article  Google Scholar 

  11. Buttler DJ, Wulff J, Stanley BG, Black MJ, MPI Sintel Datasets. http://sintel.is.tue.mpg.de/downloads. Accessed December 2023.

  12. Corpetti T, Mémin E, Pérez P, Estimating fluid optical flow. In: Proceedings of the 15th international conference on pattern recognition, 2000; vol. 3, p. 1033–6. https://doi.org/10.1109/ICPR.2000.903722

  13. Chambolle A, Pock T. A first-order primal-dual algorithm for convex problems with applications in imaging. J Math Imaging Vis. 2011;40:120–45. https://doi.org/10.1007/s10851-010-0251-1.

    Article  Google Scholar 

  14. Dirks H, Variational methods for joint motion estimation and image reconstruction, Ph.D. Thesis, Wilhems-Universität, 2015.

  15. Doshi H, Kiran NU. Nonlinear evolutionary PDE-based refinement of optical flow. Mach Graph Vis. 2021;30(1/4):45–65. https://doi.org/10.22630/MGV.2021.30.1.3.

    Article  Google Scholar 

  16. Hinterberger W, Scherzer O, Schnörr C, Weickert J. Analysis of optical flow models in the framework of calculus of variations. Numer Funct Anal Optim. 2002;23(1 &2):69–89. https://doi.org/10.1081/NFA-120004011.

    Article  MathSciNet  Google Scholar 

  17. Horn BKP, Schunck BG. Determining optical flow. Artif Intell. 1981;17:185–203. https://doi.org/10.1016/0004-3702(81)90024-2.

    Article  Google Scholar 

  18. Kawakami T, Yamaguchi T, Harada H, Estimation of rotation and divergence from optical flow constraint. In: International conference on control, automation and systems, 2008; p. 2538–42. https://doi.org/10.1109/ICCAS.2008.4694282

  19. Kohlberger T, Mémin E, Schnörr C. Variational dense motion estimation using Helmholtz Decomposition. In: International conference on scale-space theories in computer vision. 2003; vol. 2695, p. 432–48. https://doi.org/10.1007/3-540-44935-3_30.

  20. Li Y, Osher S. A new median formula with applications to PDE based denoising. Commun Math Sci. 2009;7(3):741–53. https://doi.org/10.4310/CMS.2010.v8.n4.a16.

    Article  MathSciNet  Google Scholar 

  21. Liu T. OpenOpticalFlow: an open source program for extraction of velocity fields from flow visualization images. J Open Res Softw. 2017;5:29. https://doi.org/10.5334/jors.168.

    Article  Google Scholar 

  22. Luttmann A, Bollt EM, Basnayake R, Kramer S, Tufillaro N. A framework for estimating potential flow from digital imagery. Chaos Interdiscip J Nonlinear Sci. 2013. https://doi.org/10.1063/1.4821188.

    Article  Google Scholar 

  23. McCane B, Novins K, Crannitch D, Galvin B. On benchmarking optical flow. Comput Vis Image Underst. 2001;84:126–43. https://doi.org/10.1006/cviu.2001.0930.

    Article  Google Scholar 

  24. Nagel HH, Enkelmann W. An investigation of the smoothness constraints for the estimation of displacement vector fields from image sequences. IEEE Trans Pattern Anal Mach Intell. 1986;8:565–93. https://doi.org/10.1109/TPAMI.1986.4767833.

    Article  Google Scholar 

  25. Otte M, Nagel HH. Estimation of optical flow based on higher-order spatiotemporal derivatives in interlaced and non-interlaced image sequences. Artif Intell. 1995;78:5–43. https://doi.org/10.1016/0004-3702(95)00033-X.

    Article  Google Scholar 

  26. Pan L, Liu M, Hartley R. Single image optical flow estimation with an event camera. Comput Vis Pattern Recognit. 2020. https://doi.org/10.1109/cvpr42600.2020.00174.

    Article  Google Scholar 

  27. Perona P, Malik J. Scale-space and edge detection using anisotropic diffusion. IEEE Trans Pattern Anal Mach Intell. 1990;12(7):629–39. https://doi.org/10.1109/34.56205.

    Article  Google Scholar 

  28. Sun D, Roth R, Black MJ, Secrets of optical flow estimation and their principles. In: IEEE computer society conference on computer vision and pattern recognition. 2010; p. 2432–9. https://doi.org/10.1109/CVPR.2010.5539939

  29. Sun D, Roth S, Black MJ. A quantitative analysis of current practices in optical flow estimation and the principles behind them. Int J Comput Vis. 2014;106(2):115–37. https://doi.org/10.1007/s11263-013-0644-x.

    Article  Google Scholar 

  30. Sun H, Tai X, Yuan J. Variational image motion estimation by preconditioned dual optimization. Inverse Probl Imaging. 2023;17(2):319–37. https://doi.org/10.3934/ipi.2022043.

    Article  MathSciNet  Google Scholar 

  31. Suter D, Motion estimation and vector splines. In: Computer vision and pattern recognition, 1994; p. 939-42. https://doi.org/10.1109/CVPR.1994.323929

  32. Tu Z, Poppe R, Veltkamp RC. Adaptive guided image filter for warping in variational optical flow computation. Signal Process. 2016;127:253–65. https://doi.org/10.1016/j.sigpro.2016.02.018.

    Article  Google Scholar 

  33. Tu Z, Xie W, Zhang D, Poppe R, Veltkamp RC, Li B, Yuan J. A survey of variational and CNN-based optical flow techniques. Signal Process Image Commun. 2019;72:9–24. https://doi.org/10.1016/j.image.2018.12.002.

    Article  Google Scholar 

  34. Wedel A, Pock T, Zach C, Cremers D, Bischof H. An improved algorithm for TV-\(L^1\) Optical Flow. Dagstuhl Motion Workshop. 2008. https://doi.org/10.1007/978-3-642-03061-1_2.

    Article  Google Scholar 

  35. Weickert J, Schnörr C. A theoritical framework for convex regularizers in PDE-based computation of image motion. Int J Comput Vis. 2001;45:245–64. https://doi.org/10.1023/A:1013614317973.

    Article  Google Scholar 

  36. Weickert J, Anisotropic diffusion in image processing, ser. ECMI Series. Stuttgart, Germany:Teubner-Verlag; 1998.

  37. Zhao J, Wang Y, Wang H. Optical flow with harmonic constraint and oriented smoothness. Int Conf Image Graph. 2011. https://doi.org/10.1109/ICIG.2011.122.

    Article  Google Scholar 

  38. Zhang C, Zhu L, Chen Z, Kong D, Shang X, An improved evaluation method for optial flow of endpoint error. In: International conference on computer networks and communication technology, 2016; p. 312–7. https://doi.org/10.2991/cnct-16.2017.44

  39. Zhang C, Ge L, Chen Z, Qin R, Li M, Liu W. Guided filtering: toward edge-preserving for optical flow. IEEE Access. 2018;6:26958–71. https://doi.org/10.1109/ACCESS.2018.2831920.

    Article  Google Scholar 

Download references

Acknowledgements

The authors express their deep sense of gratitude of Bhagawan Sri Sathya Sai Baba, Revered Founder Chancellor, SSSIHL.

Author information

Authors and Affiliations

  1. Department of Mathematics and Computer Science, Sri Sathya Sai Institute of Higher Learning, Puttaparthi, Andhra Pradesh, India

    Hirak Doshi & N. Uday Kiran

Authors
  1. Hirak Doshi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. N. Uday Kiran
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hirak Doshi.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Doshi, H., Kiran, N.U. A Variational Optical Flow Model for Accurate Motion Estimation from Rotational Image Sequences. SN COMPUT. SCI. 5, 359 (2024). https://doi.org/10.1007/s42979-024-02697-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s42979-024-02697-5

Keywords

Mathematics Subject Classification

Search

Navigation