Skip to main content
SpringerLink
Log in

Unified Direct Visual Tracking of Rigid and Deformable Surfaces Under Generic Illumination Changes in Grayscale and Color Images

  • Published:
International Journal of Computer Vision Aims and scope Submit manuscript

Abstract

The fundamental task of visual tracking is considered in this work as an incremental direct image registration problem. Direct methods refer to those that exploit the pixel intensities without resorting to image features. We propose new transformation models and optimization methods for directly and robustly registering images (including color ones) of rigid and deformable objects, all in a unified manner. We also show that widely adopted models are in fact particular cases of the proposed ones. Indeed, the proposed general models combine various classes of image warps and ensure robustness to generic lighting changes. Finally, the proposed optimization method together with the exploitation of all possible image information allow the algorithm to achieve high levels of accuracy. Extensive experiments are reported to demonstrate that visual tracking can indeed be highly accurate and robust despite deforming objects and severe illumination changes.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Baker, S., & Matthews, I. (2001). Equivalence and efficiency of image alignment algorithms. In IEEE computer vision and pattern recognition (pp. 1090–1097).

  • Baker, S., & Matthews, I. (2004). Lucas-Kanade 20 years on: A unifying framework. International Journal of Computer Vision, 56(3), 221–255.

    Article  Google Scholar 

  • Bartoli, A. (2008). Groupwise geometric and photometric direct image registration. IEEE Transactions on Pattern Analysis and Machine Intelligence, 30(12), 2098–2108.

    Article  Google Scholar 

  • Bartoli, A., & Zisserman, A. (2004). Direct estimation of non-rigid registration. In Proc. of the British machine vision conference (pp. 899–908).

  • Benhimane, S., & Malis, E. (2007). Homography-based 2D visual tracking and servoing. International Journal of Robotics Research, 26(7), 661–676. Special Issue on Vision and Robotics joint with the International Journal of Computer Vision.

    Article  Google Scholar 

  • Black, M. J., Fleet, D. J., & Yacoob, Y. (2000). Robustly estimating changes in image appearance. Computer Vision and Image Understanding, 78, 8–31.

    Article  Google Scholar 

  • Blinn, J. F. (1977). Models of light reflection for computer synthesized pictures. In SIGGRAPH (pp. 192–198).

  • Brown, L. G. (1992). A survey of image registration techniques. ACM Computing Surveys, 24, 325–376.

    Article  Google Scholar 

  • Carr, J., Fright, W., & Beatson, R. (1997). Surface interpolation with Radial Basis Functions for medical imaging. IEEE Transactions on Medical Imaging, 16(1).

  • Comaniciu, D., Ramesh, V., & Meer, P. (2000). Real-time tracking of non-rigid objects using mean-shift. In IEEE computer vision and pattern recognition.

  • Cook, R., & Torrance, K. (1982). A reflectance model for computer graphics. ACM Transactions on Graphics, 1, 7–24.

    Article  Google Scholar 

  • Faugeras, O., Luong, Q.-T., & Papadopoulo, T. (2001). The geometry of multiple images. Cambridge: MIT.

    MATH  Google Scholar 

  • Finlayson, G., Drew, M., & Funt, B. (1994). Color constancy: Generalized diagonal transforms suffice. Journal of the Optical Society of America A, 11(11), 3011–3020.

    Article  Google Scholar 

  • Gouiffès, M., Collewet, C., Fernandez-Maloigne, C., & Trémeau, A. (2006). Feature points tracking using photometric model and colorimetric invariants. In Proc. Eur. conf. on colour in graph., imag., and vis. (pp. 18–23).

  • Hager, G., & Belhumeur, P. (1998). Efficient region tracking with parametric models of geometry and illumination. IEEE Transactions on Pattern Analysis and Machine Intelligence, 20(10), 1025–1039.

    Article  Google Scholar 

  • Hartley, R., & Zisserman, A. (2000). Multiple view geometry in computer vision. Cambridge: Cambridge Univ. Press.

    MATH  Google Scholar 

  • Haussecker, H. W., & Fleet, D. J. (2001). Computing optical flow with physical models of brightness variation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 23(6).

  • Horst, R., & Pardalos, P. M. (Eds.) (1995). Handbook of global optimization. Dordrecht: Kluwer.

    MATH  Google Scholar 

  • Huber, P. J. (1981). Robust statistics. New York: Wiley.

    Book  MATH  Google Scholar 

  • Irani, M., & Anandan, P. (1999). All about direct methods. In Proc. workshop on vision alg.: theory and practice.

  • Jin, H., Favaro, P., & Soatto, S. (2001). Real-time feature tracking and outlier rejection with changes in illumination. In Proc. of the IEEE international conference on computer vision (pp. 684–689).

  • Jin, H., Favaro, P., & Soatto, S. (2003). A semi-direct approach to structure from motion. The Visual Computer, 6, 377–394.

    Article  Google Scholar 

  • Jurie, F., & Dhome, M. (2002). Real time robust template matching. In Proc. of the British machine vision conference (pp. 123–131).

  • Klinker, G. J., Shafer, S. A., & Kanade, T. (1990). The measurement of highlights in color images. International Journal of Computer Vision, 2, 7–32.

    Article  Google Scholar 

  • La Cascia, M., Sclaroff, S., & Athitsos, V. (2000). Fast, reliable head tracking under varying illumination: An approach based on robust registration of texture-mapped 3d models. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22, 322–336.

    Article  Google Scholar 

  • Lai, S.-H., & Fang, M. (1999). Robust and efficient image alignment with spatially varying illumination models. In IEEE computer vision and pattern recognition.

  • Lucas, B., & Kanade, T. (1981). An iterative image registration technique with an application to stereo vision. In Proc. of the int. joint conf. on art. intell. (pp. 674–679).

  • Maintz, J. B., & Viergever, M. A. (1998). A survey of medical image registration. Medical Image Analysis, 2(1), 1–36.

    Article  Google Scholar 

  • Malis, E. (2004). Improving vision-based control using efficient second-order minimization techniques. In Proc. of the IEEE int. conf. on robotics and automation, USA.

  • Malis, E. (2007). An efficient unified approach to direct visual tracking of rigid and deformable surfaces. In Proc. of the IEEE/RSJ int. conf. on intell. robots and syst., USA.

  • Montesinos, P., Gouet, V., Deriche, R., & Pele, D. (1999). Matching color uncalibrated images using differential invariants. Image and Vision Computing, 18(9), 659–671.

    Article  Google Scholar 

  • Mégret, R., Authesserre, J.-B., & Berthoumieu, Y. (2008). The bi-directional framework for unifying parametric image alignment approaches. In Proc. of the European conference on computer vision.

  • Nastar, C., Moghaddam, B., & Pentland, A. (1996). Generalized image matching: Statistical learning of physically-based deformations. In Proc. Eur. conf. on comp. vision.

  • Negahdaripour, S. (1998). Revised definition of optical flow: Integration of radiometric and geometric cues for dynamic scene analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence, 20(9), 961–979.

    Article  Google Scholar 

  • Shum, H. Y., & Szeliski, R. (2000). Construction of panoramic image mosaics with global and local alignment. International Journal of Computer Vision, 16(1), 63–84.

    Google Scholar 

  • Silveira, G., & Malis, E. (2007a). Direct visual servoing with respect to rigid objects. In Proc. of the IEEE/RSJ int. conf. on intelligent robots and systems, USA.

  • Silveira, G., & Malis, E. (2007b). Real-time visual tracking under arbitrary illumination changes. In IEEE computer vision and pattern recognition, USA.

  • Silveira, G., Malis, E., & Rives, P. (2008). An efficient direct approach to visual SLAM. IEEE Transactions on Robotics, 24, 969–979.

    Article  Google Scholar 

  • Szeliski, R. (2005). Image alignment and stitching. In Paragios, N., Chen, Y., & Faugeras, O. (Eds.) Handbook of math. models in comp. vision (pp. 273–292). Berlin: Springer.

    Google Scholar 

  • Tan, R., & Ikeuchi, K. (2005). Separating reflection components of textured surfaces using a single image. IEEE Transactions on Pattern Analysis and Machine Intelligence, 27(2), 178–193.

    Article  Google Scholar 

  • Varadarajan, V. (1974). Lie groups, Lie algebras, and their representations. New York: Prentice-Hall.

    MATH  Google Scholar 

  • Warner, F. W. (1987). Foundations of differential manifolds and Lie groups. Berlin: Springer.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. CTI Renato Archer, Rod. Pedro I, km 143,6, Amarais, CEP 13.069-901, Campinas, SP, Brazil

    Geraldo Silveira

  2. INRIA Sophia-Antipolis, 2004 Route des Lucioles, BP 93, 06902, Sophia-Antipolis Cedex, France

    Geraldo Silveira & Ezio Malis

Authors
  1. Geraldo Silveira
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ezio Malis
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Geraldo Silveira.

Electronic Supplementary Material

Below is the link to the electronic supplementary material. (3577KB)

Below is the link to the electronic supplementary material. (24170KB)

Below is the link to the electronic supplementary material. (16757KB)

Below is the link to the electronic supplementary material. (20219KB)

Below is the link to the electronic supplementary material. (1139KB)

Below is the link to the electronic supplementary material. (6023KB)

Below is the link to the electronic supplementary material. (19308KB)

Below is the link to the electronic supplementary material. (3831KM)

Below is the link to the electronic supplementary material. (1357KB)

Rights and permissions

Reprints and permissions

About this article

Cite this article

Silveira, G., Malis, E. Unified Direct Visual Tracking of Rigid and Deformable Surfaces Under Generic Illumination Changes in Grayscale and Color Images. Int J Comput Vis 89, 84–105 (2010). https://doi.org/10.1007/s11263-010-0324-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11263-010-0324-z

Keywords

Search

Navigation