Abstract
Image alignment has been a long standing problem in computer vision. Parameterized Appearance Models (PAMs) such as the Lucas-Kanade method, Eigentracking, and Active Appearance Models are commonly used to align images with respect to a template or to a previously learned model. While PAMs have numerous advantages relative to alternate approaches, they have at least two drawbacks. First, they are especially prone to local minima in the registration process. Second, often few, if any, of the local minima of the cost function correspond to acceptable solutions. To overcome these problems, this paper proposes a method to learn a metric for PAMs that explicitly optimizes that local minima occur at and only at the places corresponding to the correct fitting parameters. To the best of our knowledge, this is the first paper to address the problem of learning a metric to explicitly model local properties of the PAMs’ error surface. Synthetic and real examples show improvement in alignment performance in comparison with traditional approaches. In addition, we show how the proposed criteria for a good metric can be used to select good features to track.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Baker, S., & Matthews, I. (2001). Equivalence and efficiency of image alignment algorithms. In Proceedings of IEEE conference on computer vision and pattern recognition.
Baker, S., & Matthews, I. (2004). Lucas-Kanade 20 years on: A unifying framework. International Journal of Computer Vision, 56(3), 221–255.
Bergen, J. R., Anandan, P., Hanna, K. J., & Hingorani, R. (1992). Hierarchical model-based motion estimation. In European conference on computer vision (pp. 237–252).
Black, M. J., & Jepson, A. D. (1998). Eigentracking: Robust matching and tracking of objects using view-based representation. International Journal of Computer Vision, 26(1), 63–84.
Blanz, V., & Vetter, T. (1999). A morphable model for the synthesis of 3D faces. In ACM SIGGRAPH.
Cootes, T., Edwards, G., & Taylor, C. (2001). Active appearance models. Pattern Analysis and Machine Intelligence, 23(6).
Cootes, T. F., & Taylor, C. (2001). Statistical models of appearance for computer vision (Tech. rep.). University of Manchester.
de la Torre, F., & Black, M. J. (2003). Robust parameterized component analysis: theory and applications to 2D facial appearance models. Computer Vision and Image Understanding, 91, 53–71.
de la Torre, F., & Nguyen, M. H. (2008). Parameterized kernel principal component analysis: Theory and applications to supervised and unsupervised image alignment. In Proceedings of IEEE conference on computer vision and pattern recognition.
de la Torre, F., Vitrià, J., Radeva, P., & Melenchón, J. (2000). Eigenfiltering for flexible eigentracking. In International conference on pattern recognition (pp. 1118–1121).
de la Torre, F., Collet, A., Cohn, J., & Kanade, T. (2007). Filtered component analysis to increase robustness to local minima in appearance models. In IEEE conference on computer vision and pattern recognition.
Gong, S., Mckenna, S., & Psarrou, A. (2000). Dynamic vision: from images to face recognition. Imperial College Press.
Grant, M., & Boyd, S. (2008a). CVX: Matlab software for disciplined convex programming (web page & software). http://stanford.edu/~boyd/cvx.
Grant, M., & Boyd, S. (2008b). Graph implementations for nonsmooth convex programs. In V. Blondel, S. Boyd, & H. Kimura (Eds.), Lecture notes in control and information sciences: Recent advances in learning and control (a tribute to M. Vidyasagar) (pp. 95–110). Berlin: Springer.
Gross, R., Matthews, I., Cohn, J., Kanade, T., & Baker, S. (2007). The CMU multi-pose, illumination, and expression (Multi-PIE) face database (Tech. rep. tR-07-08). Carnegie Mellon University.
Hager, G., & Belhumeur, P. (1998). Efficient region tracking with parametric models of geometry and illumination. Pattern Analysis and Machine Intelligence, 20, 1025–1039.
Jolliffe, I. (1986). Principal component analysis. New York: Springer.
Jones, M. J., & Poggio, T. (1998). Multidimensional morphable models. In International conference on computer vision (pp. 683–688).
Kanatani, K. (1996). Statistical optimization for geometric computations: theory and practice. New York: Elsevier Science.
Learned-Miller, E. G. (2006). Data driven image models through continuous joint alignment. IEEE Transactions on Pattern Analysis and Machine Intelligence, 28(2), 236–250.
Liu, X. (2007). Generic face alignment using boosted appearance model. In IEEE conference on computer vision and pattern recognition.
Lucas, B., & Kanade, T. (1981). An iterative image registration technique with an application to stereo vision. In Proceedings of imaging understanding workshop.
Matei, B. C., & Meer, P. (2006). Estimation of nonlinear errors-in-variables models for computer vision applications. IEEE Transactions on Pattern Analysis and Machine Intelligence, 28(10), 1537–1552.
Matthews, I., & Baker, S. (2004). Active appearance models revisited. International Journal of Computer Vision, 60(2), 135–164.
Matthews, I., Ishikawa, T., & Baker, S. (2004). The template update problem. IEEE Transactions on Pattern Analysis and Machine Intelligence, 26, 810–815.
Nayar, S. K., & Poggio, T. (1996). Early visual learning. Oxford: Oxford University Press.
Nguyen, M. H., & de la Torre, F. (2008a). Learning image alignment without local minima for face detection and tracking. In 8th IEEE international conference on automatic face and gesture recognition.
Nguyen, M. H., & de la Torre, F. (2008b). Local minima free parameterized appearance models. In Proceedings of IEEE conference on computer vision and pattern recognition.
Rudin, W. (1976). Principles of mathematical analysis (3rd ed.). New York: McGraw-Hill.
Saragih, J., & Goecke, R. (2007). A nonlinear discriminative approach to AAM fitting. In International conference on computer vision.
Shi, J., & Tomasi, C. (1994). Good features to track. In IEEE conference on computer vision and pattern recognition.
Taskar, B., Guestrin, C., & Koller, D. (2003). Max-margin Markov networks. In Advances in neural information processing systems.
Tomasi, C., & Kanade, T. (1991). Detection and tracking of point features (Tech. Rep. CMU-CS-91-132). Carnegie Mellon University.
Tsochantaridis, I., Joachims, T., Hofmann, T., & Altun, Y. (2005). Large margin methods for structured and interdependent output variables. Journal of Machine Learning Research, 6, 1453–1484.
Vapnik, V. (1998). Statistical learning theory. New York: Wiley.
Vetter, T. (1997). Learning novel views to a single face image. In International conference on automatic face and gesture recognition.
Wimmer, M., Stulp, F., Tschechne, S. J., & Radig, B. (2006). Learning robust objective functions for model fitting in image understanding applications. In Proceedings of British machine vision conference.
Wu, H., Liu, X., & Doretto, G. (2008). Face alignment via boosted ranking model. In Proceedings of IEEE conference on computer vision and pattern recognition.
Xiao, J., Baker, S., Matthews, I., & Kanade, T. (2004). Real-time combined 2D+3D active appearance models. In Conference on computer vision and pattern recognition (Vol. II, pp. 535–542).
Yang, L. (2006). Distance metric learning: A comprehensive survey. http://www.cse.msu.edu/~yangliu1/frame_survey_v2.pdf.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Nguyen, M.H., de la Torre, F. Metric Learning for Image Alignment. Int J Comput Vis 88, 69–84 (2010). https://doi.org/10.1007/s11263-009-0299-9
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11263-009-0299-9