Abstract
Differential methods belong to the most widely used techniques for optic flow computation in image sequences. They can be classified into local methods such as the Lucas–Kanade technique or Bigün's structure tensor method, and into global methods such as the Horn/Schunck approach and its extensions. Often local methods are more robust under noise, while global techniques yield dense flow fields. The goal of this paper is to contribute to a better understanding and the design of novel differential methods in four ways; (i) We juxtapose the role of smoothing/regularisation processes that are required in local and global differential methods for optic flow computation. (ii) This discussion motivates us to describe and evaluate a novel method that combines important advantages of local and global approaches: It yields dense flow fields that are robust against noise. (iii) Spatiotemporal and nonlinear extensions as well as multiresolution frameworks are presented for this hybrid method. (iv) We propose a simple confidence measure for optic flow methods that minimise energy functionals. It allows to sparsify a dense flow field gradually, depending on the reliability required for the resulting flow. Comparisons with experiments from the literature demonstrate the favourable performance of the proposed methods and the confidence measure.
Similar content being viewed by others
References
Alvarez, L., Esclarín, J., Lefébure, M., and Sánchez, J. 1999. A PDE model for computing the optical flow. In Proc. XVI Congreso de Ecuaciones Diferenciales y Aplicaciones, Las Palmas de Gran Canaria, Spain. pp. 1349–1356.
Alvarez, L., Weickert, J., and Sánchez., J. 2000. Reliable estimation of dense optical flowfields with large displacements. International Journal of Computer Vision, 39(1):41–56.
Ananden., P. 1989. A computational framework and an algorithm for the measurement of visual motion. International Journal of Computer Vision, 2:283–310.
Aubert, G., Deriche, R., and Kornprobst., P. 1999. Computing optical flow via variational techniques. SIAM Journal on Applied Mathematics, 60(1):156–182.
Bab-Hadiashar, A. and Suter, D. 1998. Robust optic flow computation. International Journal of Computer Vision, 29(1):59–77.
Bainbridge-Smith, A. and Lane, R.G. 1997. Determining optical flow using a differential method. Image and Vision Computing, 15(1):11–22.
Barron, J.L., Fleet, D.J., and Beauchemin, S.S. 1994. Performance of optical flowtechniques. International Journal of Computer Vision, 12(1):43–77.
Bertero, M., Poggio, T.A., and Torre, V. 1988. Ill-posed problems in early vision. Proceedings of the IEEE, 76(8):869–889.
Bigün, J. and Granlund, G.H. 1988. Optical flow based on the inertia matrix in the frequency domain. In Proc. SSAB Symposium on Picture Processing, Lund, Sweden.
Bigün, J., Granlund, G.H., and Wiklund, J. 1991. Multidimensional orientation estimation with applications to texture analysis and optical flow. IEEE Transactions on Pattern Analysis and Machine Intelligence, 13(8):775–790.
Black, M.J. 1994. Recursive non-linear estimation of discontinuous flow fields. In Computer Vision, ECCV'94, J.-O. Eklundh (Ed.), vol. 800 of Lecture Notes in Computer Science, Springer: Berlin, pp. 138–145.
Black, M.J. and Anandan, P. 1991. Robust dynamic motion estimation over time. In Proc. 1991 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, IEEE Computer Society Press: Maui, HI, pp. 292–302.
Black, M.J. and Anandan, P. 1996. The robust estimation of multiple motions: Parametric and piecewise smooth flow fields. Computer Vision and Image Understanding, 63(1):75–104.
Black, M.J. and Jepson, A. 1996. Estimating optical flow in segmented images using variable-order parametric models with local deformations. IEEE Transactions on Pattern Analysis and Machine Intelligence, 18(10):972–986.
Brox, T. and Weickert, J. 2002. Nonlinear matrix diffusion for optic flow estimation. In Pattern Recognition, L. Van Gool, (Ed.), vol. 2449 of Lecture Notes in Computer Science, Springer: Berlin, pp. 446–453.
Bruhn, A. 2001. Regularization in motion estimation. Master's thesis, Department of Mathematics and Computer Science, University of Mannheim, Germany.
Bruhn, A., Weickert, J., Feddern, C., Kohlberger, T., and Schnörr, C. 2003. Real-time optic flow computation with variational methods. In Computer Analysis of Images and Patterns, N. Petkov and M.A. Westberg (Eds.), vol. 2756 of Lecture Notes in Computer Science, Springer: Berlin, pp. 222–229.
Bruhn, A., Weickert, J., and SchnÖrr, C. 2002. Combining the advantages of local and global optic flow methods. In Pattern Recognition, L. Van Gool, (Ed.), vol. 2449 of Lecture Notes in Computer Science, Springer: Berlin, pp. 454–462.
Charbonnier, P., Blanc-Féraud, L. Aubert, G., and Barlaud, M. 1994. Two deterministic half-quadratic regularization algorithms for computed imaging. In Proc. 1994 IEEE International Conference on Image Processing, vol. 2. IEEE Computer Society Press: Austin TX, pp. 168–172.
Cohen, I. 1993. Nonlinear variational method for optical flow computation. In Proc. Eighth Scandinavian Conference on Image Analysis, vol. 1, Tromsø, Norway, pp. 523–530.
Courant, R. and Hilbert, D. 1953. Methods of Mathematical Physics, vol. 1. Interscience: New York.
Elad, M. and Feuer, A. 1998. Recursive optical flow estimation—adaptive filtering approach. Journal of Visual Communication and Image Representation, 9(2):119–138.
Elsgolc, L.E. 1961. Calculus of Variations. Pergamon: Oxford.
Farnebäck, G. 2000. Fast and accurate motion estimation using orientation tensors and parametric motion models. In Proc. 15th International Conference on Pattern Recognition, vol. 1, Barcelona, Spain, pp, 135–139.
Farnebäck, G. 2001. Very high accuracy velocity estimation using orientation tensors, parametric motion, and simultaneous segmentation of the motion field. In Proc. Eighth International Conference on Computer Vision, vol. 1, IEEE Computer Society Press: Vancouver, Canada, pp. 171–177.
Fermüller, C., Shulman, D., and Aloimonos, Y. 2001. The statistics of optical flow. Computer Vision and Image Understanding, 82(1):1–32.
Fleet, D.J. and Jepson, A.D. 1990. Computation of component image velocity from local phase information. International Journal of Computer Vision, 5(1):77–104.
Galvin, B., McCane, B., Novins, K., Mason, D., and Mills, S. 1998. Recovering motion fields: An analysis of eight optical flow algorithms. In Proc. 1998 British Machine Vision Conference, Southampton, England.
Hampel, F.R., Ronchetti, E.M., Rousseeuw, P.J., and Stahel, W.A. 1986. Robust Statistics: The Approach Based on Influence Functions. MIT Press: Cambridge, MA.
Heitz, F. and Bouthemy, P. 1993. Multimodal estimation of discontinuous optical flow using Markov random fields. IEEE Transac tions on Pattern Analysis and Machine Intelligence, 15(12):1217–1232.
Hinterberger, W., Scherzer, O., SchnÖrr, C., and Weickert, J. 2002. Analysis of optical flow models in the framework of calculus of variations. Numerical Functional Analysis and Optimization, 23(1/2):69–89.
Horn, B. and Schunck, B. 1981. Determining optical flow. Artificial Intelligence, 17:185–203.
Huber, P.J. 1981. Robust Statistics. Wiley: New York.
Jähne, B. 2001. Digitale Bildverarbeitung. Springer: Berlin.
Ju, S., Black, M., and Jepson, A. 1996. Skin and bones: Multi-layer, locally affine, optical flow and regularization with transparency. In Proc. 1996 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, IEEE Computer Society Press: San Francisco, CA, pp. 307–314.
Karlholm, J. 1998. local signal models for image sequence analysis. PhD thesis, LinkÖping University, Sweden, Dissertation No. 536.
Kearney, J.K., Thompson, W.B., and Boley, D.L. 1987. Optical flow estimation: An error analysis of gradient-based methods with local optimization. IEEE Transactions on Pattern Analysis and Machine Intelligence, 9(2):229–244.
Kumar, A., Tannenbaum, A.R., and Balas, G.J. 1996. Optic flow: A curve evolution approach. IEEE Transactions on Image Processing, 5(4):598–610.
Lai, S.-H. and Vemuri, B.C. 1998. Reliable and efficient computation of optical flow. International Journal of Computer Vision, 29(2):87–105.
Lauze, F.-B, Kornprobst, P., Lenglet, C., Deriche, R., and Nielsen, M. 2004. Sur quelques méthodes de calcul de flot optique à partir du tenseur de structure: Synthése et contribution. In Proc. 14th French Conference on Pattern Recognition and Artificial Intelligence, Toulouse, France.
Lucas, B. and Kanade, T. 1981. An iterative image registration technique with an application to stereo vision. In Proc. Seventh International Joint Conference on Artificial Intelligence, Vancouver, Canada, pp. 674–679.
Lucas, B.D. 1984. Generalized image matching by the method of differences. PhD thesis, School of Computer Science, Carnegie-Mellon University, Pittsburgh, PA.
Mémin, E. and Pérez, P. 1998. Dense estimation and object-based segmentation of the optical flow with robust techniques. IEEE Transactions on Image Processing, 7(5):703–719.
Mémin, E. and Pérez, P. 1998. A multigrid approach for hierarchical motion estimation. In Proc. 6th International Conference on Computer Vision, Bombay, India, pp. 933–938.
Mémin, E. and Pérez, P. 2002. Hierarchical estimation and segmentation of dense motion fields. International Journal of Computer Vision, 46(2):129–155.
Mitiche, A. and Bouthemy, P. 1996. Computation and analysis of image motion: A synopsis of current problems and methods. International Journal of Computer Vision, 19(1):29–55.
Murray, D.W. and Buxton, B.F. 1987. Scene segmentation from visual motion using global optimization. IEEE Transactions on Pattern Analysis and Machine Intelligence, 9(2):220–228.
Nagel, H.-H. 1983. Constraints for the estimation of displacement vector fields from image sequences. In Proc. Eighth International Joint Conference on Artificial Intelligence, vol. 2, Karlsruhe, West Germany, pp. 945–951.
Nagel, H.-H. 1990. Extending the ‘oriented smoothness constraint’ into the temporal domain and the estimation of derivatives of optical flow. In Computer Vision-ECCV '90, vol. 427 of Lecture Notes in Computer Science, O. Faugeras (Ed.), Springer: Berlin, pp. 139–148.
Nagel, H.-H. and Gehrke, A. 1998. Spatiotemporally adaptive estimation and segmentation of OF-fields. In Computer Vision-ECCV '98, vol. 1407 of Lecture Notes in Computer Science, H. Burkhardt and B. Neumann, (Eds.), Springer: Berlin, pp. 86–102.
Nesi, P. 1993. Variational approach to optical flowestimation managing discontinuities. Image and Vision Computing, 11(7):419–439.
Ohta, N. 1996. Uncertainty models of the gradient constraint for optical flow computation. IEICE Transactions on Information and Systems, E79-D(7):958–962.
Ong, E.P. and Spann, M. 1999. Robust optical flow computation based on least-median-of-squares regression. International Journal of Computer Vision, 31(1):51–82.
Press, W.H., Teukolsky, S.A., Vetterling, W.T., and Flannery, B.P. 1992. Numerical Recipes in C, 2nd ed. Cambridge University Press: Cambridge, UK.
Proesmans, M., Van Gool, L., Pauwels, E., and Oosterlinck, A. 1994. Determination of optical flow and its discontinuities using non-linear diffusion. In J.-O. Eklundh, (Ed.), Computer Vision-ECCV '94, volume 801 of Lecture Notes in Computer Science. Springer: Berlin, pp. 295–304.
Schnörr, C. 1991. Determining optical flow for irregular domains by minimizing quadratic functionals of a certain class. International Journal of Computer Vision, 6(1):25–38.
Schnörr, C. 1993. On functionals with greyvalue-controlled smoothness terms for determining optical flow. IEEE Transactions on Pattern Analysis and Machine Intelligence, 15:1074–1079.
Schnörr, C. 1994. Segmentation of visual motion by minimizing convex non-quadratic functionals. In Proc. Twelfth International Conference on Pattern Recognition, vol. A, IEEE Computer Society Press: Jerusalem, Israel, pp. 661–663.
Shulman, D. and Hervé, J. 1989. Regularization of discontinuous flow fields. In Proc. Workshop on Visual Motion, IEEE Computer Society Press: Irvine, CA, pp. 81–90.
Simoncelli, E.P., Adelson, E.H., and Heeger, D.J. 1991. Probability distributions of optical flow. In Proc. 1991 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, IEEE Computer Society Press: Maui, HI, pp. 310–315.
Stiller, C. and Konrad, J. 1999. Estimating motion in image sequences. IEEE Signal Processing Magazine, 16:70–91.
Szeliski, R. and Coughlan, J. 1994. Hierarchical spline-based image registration. In Proc. 1994 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, IEEE Computer Society Press: Seattle, WA, pp. 194–201.
Tretiak, O. and Pastor, L. 1984. Velocity estimation from image sequences with second order differential operators. In Proc. Seventh International Conference on Pattern Recognition, Montreal, Canada, pp. 16–19.
Uras, S., Girosi, F., Verri, A., and Torre, V.A., 1988. computational approach to motion perception. Biological Cybernetics, 60:79–87.
Weber, J. and Malik, J. 1995. Robust computation of optical flow in a multi-scale differential framework. International Journal of Computer Vision, 14:67–81.
Weickert, J. 1998. Anisotropic Diffusion in Image Processing. Teubner, Stuttgart.
Weickert, J. and SchnÖrr, C. 2001. A theoretical framework for convex regularizers in PDE-based computation of image motion. International Journal of Computer Vision, 45(3):245–264.
Weickert, J. and SchnÖrr, C. 2001. Variational optic flow computation with a spatio-temporal smoothness constraint. Journal of Mathematical Imaging and Vision, 14(3):245–255.
Yacoob, Y. and Davis, L.S. 1999. Temporal multi-scale models for flow and acceleration. International Journal of Computer Vision, 32(2):1–17.
Young, D.M. 1971. Iterative Solution of Large Linear Systems. Academic Press: New York.
Yuille, A.L. and Poggio, T.A. 1986. Scaling theorems for zero crossings. IEEE Transactions on Pattern Analysis and Machine Intelligence, 8(1):15–25.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Bruhn, A., Weickert, J. & Schnörr, C. Lucas/Kanade Meets Horn/Schunck: Combining Local and Global Optic Flow Methods. International Journal of Computer Vision 61, 211–231 (2005). https://doi.org/10.1023/B:VISI.0000045324.43199.43
Issue Date:
DOI: https://doi.org/10.1023/B:VISI.0000045324.43199.43