Skip to main content
Log in

Lucas/Kanade Meets Horn/Schunck: Combining Local and Global Optic Flow Methods

  • Published:
International Journal of Computer Vision Aims and scope Submit manuscript

Abstract

Differential methods belong to the most widely used techniques for optic flow computation in image sequences. They can be classified into local methods such as the Lucas–Kanade technique or Bigün's structure tensor method, and into global methods such as the Horn/Schunck approach and its extensions. Often local methods are more robust under noise, while global techniques yield dense flow fields. The goal of this paper is to contribute to a better understanding and the design of novel differential methods in four ways; (i) We juxtapose the role of smoothing/regularisation processes that are required in local and global differential methods for optic flow computation. (ii) This discussion motivates us to describe and evaluate a novel method that combines important advantages of local and global approaches: It yields dense flow fields that are robust against noise. (iii) Spatiotemporal and nonlinear extensions as well as multiresolution frameworks are presented for this hybrid method. (iv) We propose a simple confidence measure for optic flow methods that minimise energy functionals. It allows to sparsify a dense flow field gradually, depending on the reliability required for the resulting flow. Comparisons with experiments from the literature demonstrate the favourable performance of the proposed methods and the confidence measure.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Alvarez, L., Esclarín, J., Lefébure, M., and Sánchez, J. 1999. A PDE model for computing the optical flow. In Proc. XVI Congreso de Ecuaciones Diferenciales y Aplicaciones, Las Palmas de Gran Canaria, Spain. pp. 1349–1356.

  • Alvarez, L., Weickert, J., and Sánchez., J. 2000. Reliable estimation of dense optical flowfields with large displacements. International Journal of Computer Vision, 39(1):41–56.

    Google Scholar 

  • Ananden., P. 1989. A computational framework and an algorithm for the measurement of visual motion. International Journal of Computer Vision, 2:283–310.

    Google Scholar 

  • Aubert, G., Deriche, R., and Kornprobst., P. 1999. Computing optical flow via variational techniques. SIAM Journal on Applied Mathematics, 60(1):156–182.

    Google Scholar 

  • Bab-Hadiashar, A. and Suter, D. 1998. Robust optic flow computation. International Journal of Computer Vision, 29(1):59–77.

    Google Scholar 

  • Bainbridge-Smith, A. and Lane, R.G. 1997. Determining optical flow using a differential method. Image and Vision Computing, 15(1):11–22.

    Google Scholar 

  • Barron, J.L., Fleet, D.J., and Beauchemin, S.S. 1994. Performance of optical flowtechniques. International Journal of Computer Vision, 12(1):43–77.

    Google Scholar 

  • Bertero, M., Poggio, T.A., and Torre, V. 1988. Ill-posed problems in early vision. Proceedings of the IEEE, 76(8):869–889.

    Google Scholar 

  • Bigün, J. and Granlund, G.H. 1988. Optical flow based on the inertia matrix in the frequency domain. In Proc. SSAB Symposium on Picture Processing, Lund, Sweden.

    Google Scholar 

  • Bigün, J., Granlund, G.H., and Wiklund, J. 1991. Multidimensional orientation estimation with applications to texture analysis and optical flow. IEEE Transactions on Pattern Analysis and Machine Intelligence, 13(8):775–790.

    Google Scholar 

  • Black, M.J. 1994. Recursive non-linear estimation of discontinuous flow fields. In Computer Vision, ECCV'94, J.-O. Eklundh (Ed.), vol. 800 of Lecture Notes in Computer Science, Springer: Berlin, pp. 138–145.

    Google Scholar 

  • Black, M.J. and Anandan, P. 1991. Robust dynamic motion estimation over time. In Proc. 1991 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, IEEE Computer Society Press: Maui, HI, pp. 292–302.

    Google Scholar 

  • Black, M.J. and Anandan, P. 1996. The robust estimation of multiple motions: Parametric and piecewise smooth flow fields. Computer Vision and Image Understanding, 63(1):75–104.

    Article  Google Scholar 

  • Black, M.J. and Jepson, A. 1996. Estimating optical flow in segmented images using variable-order parametric models with local deformations. IEEE Transactions on Pattern Analysis and Machine Intelligence, 18(10):972–986.

    Google Scholar 

  • Brox, T. and Weickert, J. 2002. Nonlinear matrix diffusion for optic flow estimation. In Pattern Recognition, L. Van Gool, (Ed.), vol. 2449 of Lecture Notes in Computer Science, Springer: Berlin, pp. 446–453.

    Google Scholar 

  • Bruhn, A. 2001. Regularization in motion estimation. Master's thesis, Department of Mathematics and Computer Science, University of Mannheim, Germany.

    Google Scholar 

  • Bruhn, A., Weickert, J., Feddern, C., Kohlberger, T., and Schnörr, C. 2003. Real-time optic flow computation with variational methods. In Computer Analysis of Images and Patterns, N. Petkov and M.A. Westberg (Eds.), vol. 2756 of Lecture Notes in Computer Science, Springer: Berlin, pp. 222–229.

    Google Scholar 

  • Bruhn, A., Weickert, J., and SchnÖrr, C. 2002. Combining the advantages of local and global optic flow methods. In Pattern Recognition, L. Van Gool, (Ed.), vol. 2449 of Lecture Notes in Computer Science, Springer: Berlin, pp. 454–462.

    Google Scholar 

  • Charbonnier, P., Blanc-Féraud, L. Aubert, G., and Barlaud, M. 1994. Two deterministic half-quadratic regularization algorithms for computed imaging. In Proc. 1994 IEEE International Conference on Image Processing, vol. 2. IEEE Computer Society Press: Austin TX, pp. 168–172.

    Google Scholar 

  • Cohen, I. 1993. Nonlinear variational method for optical flow computation. In Proc. Eighth Scandinavian Conference on Image Analysis, vol. 1, Tromsø, Norway, pp. 523–530.

    Google Scholar 

  • Courant, R. and Hilbert, D. 1953. Methods of Mathematical Physics, vol. 1. Interscience: New York.

    Google Scholar 

  • Elad, M. and Feuer, A. 1998. Recursive optical flow estimation—adaptive filtering approach. Journal of Visual Communication and Image Representation, 9(2):119–138.

    Google Scholar 

  • Elsgolc, L.E. 1961. Calculus of Variations. Pergamon: Oxford.

    Google Scholar 

  • Farnebäck, G. 2000. Fast and accurate motion estimation using orientation tensors and parametric motion models. In Proc. 15th International Conference on Pattern Recognition, vol. 1, Barcelona, Spain, pp, 135–139.

    Google Scholar 

  • Farnebäck, G. 2001. Very high accuracy velocity estimation using orientation tensors, parametric motion, and simultaneous segmentation of the motion field. In Proc. Eighth International Conference on Computer Vision, vol. 1, IEEE Computer Society Press: Vancouver, Canada, pp. 171–177.

    Google Scholar 

  • Fermüller, C., Shulman, D., and Aloimonos, Y. 2001. The statistics of optical flow. Computer Vision and Image Understanding, 82(1):1–32.

    Google Scholar 

  • Fleet, D.J. and Jepson, A.D. 1990. Computation of component image velocity from local phase information. International Journal of Computer Vision, 5(1):77–104.

    Google Scholar 

  • Galvin, B., McCane, B., Novins, K., Mason, D., and Mills, S. 1998. Recovering motion fields: An analysis of eight optical flow algorithms. In Proc. 1998 British Machine Vision Conference, Southampton, England.

    Google Scholar 

  • Hampel, F.R., Ronchetti, E.M., Rousseeuw, P.J., and Stahel, W.A. 1986. Robust Statistics: The Approach Based on Influence Functions. MIT Press: Cambridge, MA.

    Google Scholar 

  • Heitz, F. and Bouthemy, P. 1993. Multimodal estimation of discontinuous optical flow using Markov random fields. IEEE Transac tions on Pattern Analysis and Machine Intelligence, 15(12):1217–1232.

    Google Scholar 

  • Hinterberger, W., Scherzer, O., SchnÖrr, C., and Weickert, J. 2002. Analysis of optical flow models in the framework of calculus of variations. Numerical Functional Analysis and Optimization, 23(1/2):69–89.

    Google Scholar 

  • Horn, B. and Schunck, B. 1981. Determining optical flow. Artificial Intelligence, 17:185–203.

    Article  Google Scholar 

  • Huber, P.J. 1981. Robust Statistics. Wiley: New York.

    Google Scholar 

  • Jähne, B. 2001. Digitale Bildverarbeitung. Springer: Berlin.

    Google Scholar 

  • Ju, S., Black, M., and Jepson, A. 1996. Skin and bones: Multi-layer, locally affine, optical flow and regularization with transparency. In Proc. 1996 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, IEEE Computer Society Press: San Francisco, CA, pp. 307–314.

    Google Scholar 

  • Karlholm, J. 1998. local signal models for image sequence analysis. PhD thesis, LinkÖping University, Sweden, Dissertation No. 536.

    Google Scholar 

  • Kearney, J.K., Thompson, W.B., and Boley, D.L. 1987. Optical flow estimation: An error analysis of gradient-based methods with local optimization. IEEE Transactions on Pattern Analysis and Machine Intelligence, 9(2):229–244.

    Google Scholar 

  • Kumar, A., Tannenbaum, A.R., and Balas, G.J. 1996. Optic flow: A curve evolution approach. IEEE Transactions on Image Processing, 5(4):598–610.

    Google Scholar 

  • Lai, S.-H. and Vemuri, B.C. 1998. Reliable and efficient computation of optical flow. International Journal of Computer Vision, 29(2):87–105.

    Google Scholar 

  • Lauze, F.-B, Kornprobst, P., Lenglet, C., Deriche, R., and Nielsen, M. 2004. Sur quelques méthodes de calcul de flot optique à partir du tenseur de structure: Synthése et contribution. In Proc. 14th French Conference on Pattern Recognition and Artificial Intelligence, Toulouse, France.

  • Lucas, B. and Kanade, T. 1981. An iterative image registration technique with an application to stereo vision. In Proc. Seventh International Joint Conference on Artificial Intelligence, Vancouver, Canada, pp. 674–679.

  • Lucas, B.D. 1984. Generalized image matching by the method of differences. PhD thesis, School of Computer Science, Carnegie-Mellon University, Pittsburgh, PA.

    Google Scholar 

  • Mémin, E. and Pérez, P. 1998. Dense estimation and object-based segmentation of the optical flow with robust techniques. IEEE Transactions on Image Processing, 7(5):703–719.

    Google Scholar 

  • Mémin, E. and Pérez, P. 1998. A multigrid approach for hierarchical motion estimation. In Proc. 6th International Conference on Computer Vision, Bombay, India, pp. 933–938.

  • Mémin, E. and Pérez, P. 2002. Hierarchical estimation and segmentation of dense motion fields. International Journal of Computer Vision, 46(2):129–155.

    Google Scholar 

  • Mitiche, A. and Bouthemy, P. 1996. Computation and analysis of image motion: A synopsis of current problems and methods. International Journal of Computer Vision, 19(1):29–55.

    Google Scholar 

  • Murray, D.W. and Buxton, B.F. 1987. Scene segmentation from visual motion using global optimization. IEEE Transactions on Pattern Analysis and Machine Intelligence, 9(2):220–228.

    Google Scholar 

  • Nagel, H.-H. 1983. Constraints for the estimation of displacement vector fields from image sequences. In Proc. Eighth International Joint Conference on Artificial Intelligence, vol. 2, Karlsruhe, West Germany, pp. 945–951.

    Google Scholar 

  • Nagel, H.-H. 1990. Extending the ‘oriented smoothness constraint’ into the temporal domain and the estimation of derivatives of optical flow. In Computer Vision-ECCV '90, vol. 427 of Lecture Notes in Computer Science, O. Faugeras (Ed.), Springer: Berlin, pp. 139–148.

    Google Scholar 

  • Nagel, H.-H. and Gehrke, A. 1998. Spatiotemporally adaptive estimation and segmentation of OF-fields. In Computer Vision-ECCV '98, vol. 1407 of Lecture Notes in Computer Science, H. Burkhardt and B. Neumann, (Eds.), Springer: Berlin, pp. 86–102.

    Google Scholar 

  • Nesi, P. 1993. Variational approach to optical flowestimation managing discontinuities. Image and Vision Computing, 11(7):419–439.

    Google Scholar 

  • Ohta, N. 1996. Uncertainty models of the gradient constraint for optical flow computation. IEICE Transactions on Information and Systems, E79-D(7):958–962.

    Google Scholar 

  • Ong, E.P. and Spann, M. 1999. Robust optical flow computation based on least-median-of-squares regression. International Journal of Computer Vision, 31(1):51–82.

    Google Scholar 

  • Press, W.H., Teukolsky, S.A., Vetterling, W.T., and Flannery, B.P. 1992. Numerical Recipes in C, 2nd ed. Cambridge University Press: Cambridge, UK.

    Google Scholar 

  • Proesmans, M., Van Gool, L., Pauwels, E., and Oosterlinck, A. 1994. Determination of optical flow and its discontinuities using non-linear diffusion. In J.-O. Eklundh, (Ed.), Computer Vision-ECCV '94, volume 801 of Lecture Notes in Computer Science. Springer: Berlin, pp. 295–304.

    Google Scholar 

  • Schnörr, C. 1991. Determining optical flow for irregular domains by minimizing quadratic functionals of a certain class. International Journal of Computer Vision, 6(1):25–38.

    Google Scholar 

  • Schnörr, C. 1993. On functionals with greyvalue-controlled smoothness terms for determining optical flow. IEEE Transactions on Pattern Analysis and Machine Intelligence, 15:1074–1079.

    Google Scholar 

  • Schnörr, C. 1994. Segmentation of visual motion by minimizing convex non-quadratic functionals. In Proc. Twelfth International Conference on Pattern Recognition, vol. A, IEEE Computer Society Press: Jerusalem, Israel, pp. 661–663.

    Google Scholar 

  • Shulman, D. and Hervé, J. 1989. Regularization of discontinuous flow fields. In Proc. Workshop on Visual Motion, IEEE Computer Society Press: Irvine, CA, pp. 81–90.

    Google Scholar 

  • Simoncelli, E.P., Adelson, E.H., and Heeger, D.J. 1991. Probability distributions of optical flow. In Proc. 1991 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, IEEE Computer Society Press: Maui, HI, pp. 310–315.

    Google Scholar 

  • Stiller, C. and Konrad, J. 1999. Estimating motion in image sequences. IEEE Signal Processing Magazine, 16:70–91.

    Google Scholar 

  • Szeliski, R. and Coughlan, J. 1994. Hierarchical spline-based image registration. In Proc. 1994 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, IEEE Computer Society Press: Seattle, WA, pp. 194–201.

    Google Scholar 

  • Tretiak, O. and Pastor, L. 1984. Velocity estimation from image sequences with second order differential operators. In Proc. Seventh International Conference on Pattern Recognition, Montreal, Canada, pp. 16–19.

  • Uras, S., Girosi, F., Verri, A., and Torre, V.A., 1988. computational approach to motion perception. Biological Cybernetics, 60:79–87.

    Google Scholar 

  • Weber, J. and Malik, J. 1995. Robust computation of optical flow in a multi-scale differential framework. International Journal of Computer Vision, 14:67–81.

    Google Scholar 

  • Weickert, J. 1998. Anisotropic Diffusion in Image Processing. Teubner, Stuttgart.

  • Weickert, J. and SchnÖrr, C. 2001. A theoretical framework for convex regularizers in PDE-based computation of image motion. International Journal of Computer Vision, 45(3):245–264.

    Google Scholar 

  • Weickert, J. and SchnÖrr, C. 2001. Variational optic flow computation with a spatio-temporal smoothness constraint. Journal of Mathematical Imaging and Vision, 14(3):245–255.

    Google Scholar 

  • Yacoob, Y. and Davis, L.S. 1999. Temporal multi-scale models for flow and acceleration. International Journal of Computer Vision, 32(2):1–17.

    Google Scholar 

  • Young, D.M. 1971. Iterative Solution of Large Linear Systems. Academic Press: New York.

    Google Scholar 

  • Yuille, A.L. and Poggio, T.A. 1986. Scaling theorems for zero crossings. IEEE Transactions on Pattern Analysis and Machine Intelligence, 8(1):15–25.

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Rights and permissions

Reprints and permissions

About this article

Cite this article

Bruhn, A., Weickert, J. & Schnörr, C. Lucas/Kanade Meets Horn/Schunck: Combining Local and Global Optic Flow Methods. International Journal of Computer Vision 61, 211–231 (2005). https://doi.org/10.1023/B:VISI.0000045324.43199.43

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1023/B:VISI.0000045324.43199.43

Navigation