Abstract
In this paper we provide an integrated approach for matching patterns in scenes combining 3D and visual information. For local definition of points we propose a descriptor based on the notion of covariance of features for fusion of shape and color information of 3D surfaces, so-called multi-scale covariance descriptor (MCOV). The intrinsic properties of this descriptor are many: it is invariant to spatial rigid transformations, and robust to noise and resolution changes; it can also be used for characteristic point detection; and lies on top of a manifold topology which allows the use of analytical metric properties. This descriptor is complemented with a game theoretic approach for solving the matching correspondences under global geometric constraints. This layer offers a comprehensive understanding of the scene and avoids possible mismatches due to repeated areas or symmetries—which would be impossibly identified by the detector solely at a local level. Our solution is able to accurately match different views of a scene even under spatial transformations, high noise levels and with small overlap between views, outperforming state-of-the-art approaches. Results are validated by comparing MCOV against other state-of-the-art 3D point descriptor methods, and matching complex 3D and color scenes under several challenging conditions.
Similar content being viewed by others
References
Albarelli, A., Rodola, E., & Torsello, A. (2010). A game-theoretic approach to fine surface registration without initial motionestimation. In International conference on computer vision and pattern recognition (CVPR) (pp. 430–437).
Albarelli, A., Rota Bulò, S., Torsello, A., & Pelillo, M. (2009). Matching as a non-cooperative game. In International conference on computer vision (ICCV) (pp. 1319–1326).
Arsigny, V., Fillard, P., Pennec, X., & Ayache, N. (2006). Log-Euclidean metrics for fast and simple calculus on diffusion tensors. Magnetic Resonance in Medicine, 56(2), 411–421.
Besl, P., & McKay, N. (1992). A method for registration of 3-d shapes. Transactions on Pattern Analysis and Machine Intelligence, PAMI, 14, 239–256.
Brusco, N., Andreetto, M., Giorgi, A., & Cortelazzo, G. M. (2005). 3d registration by textured spin-images. In International conference on 3D digital imaging and modeling (3DIM) (pp. 262–269).
Chen, C. S., Hung, Y. P., & Cheng, J. B. (1999). RANSAC-based darces: A new approach to fast automatic registration of partially overlapping range images. Transactions on Pattern Analysis and Machine Intelligence, PAMI, 21, 1229–1234.
Cherian, A., Sra, S., Banerjee, A., & Papanikolopoulos, N. (2011). Efficient similarity search for covariance matrices via the Jensen–Bregman logdet divergence. In International conference on computer vision (ICCV) (pp. 2399–2406). IEEE.
Chua, C., & Jarvis, R. (1997). Point signatures: A new representation for 3D object recognition. IJCV, 25, 63–85.
Chum, O. & Matas, J. (2005): Matching with PROSAC-progressive sample consensus. In International conference on computer vision and pattern recognition (CVPR) (Vol. 1, pp. 220–226).
Chum, O., & Matas, J. (2008). Optimal randomized RANSAC. Transactions on Pattern Analysis and Machine Intelligence, PAMI, 30, 1472–1482.
Chum, O., Matas, J., & Obdrzalek, S. (2004). Enhancing RANSAC by generalized model optimization. In Asian conference on computer vision (ACCV) (Vol. 2, pp. 812–817).
Fehr, D., Cherian, A., Sivalingam, R., Nickolay, S., Morellas, V., & Papanikolopoulos, N. (2012). Compact covariance descriptors in 3d point clouds for object recognition. In International conference on robotics and automation (ICRA) (pp. 1793–1798).
Fischler, M. A., & Bolles, R. C. (1981). Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography. Communications of the ACM, 24, 381–395.
Flint, A., Dick, A. R., & Van Den Hengel, A. (2007). Thrift: Local 3d structure recognition. Digital Image Computing Techniques and Applications, 7, 182–188.
Förstner, W. & Moonen, B. (1999). A metric for covariance matrices. Quo vadis geodesia (pp. 113–128).
Frome, A., Huber, D., Kolluri, R., Bülow, T., & Malik, J. (2004). Recognizing objects in range data using regional point descriptors. ECCV, 3023, 224–237.
Hartley, R., & Zisserman, A. (2003). Multiple view geometry in computer vision. Cambridge: Cambridge University Press.
Horn, B. K. P. (1987). Closed-form solution of absolute orientation using unit quaternions. Optical Society of America, 4, 629–642.
Johnson, A. (1997). Spin-images: A representation for 3-d surface matching. Ph.D. thesis. Pittsburgh, PA: Robotics Institute, Carnegie Mellon University.
Johnson, A., & Hebert, M. (1999). Using spin images for efficient object recognition in cluttered 3d scenes. Transactions on Pattern Analysis and Machine Intelligence, PAMI, 21, 433–449.
Kovnatsky, A., Bronstein, M. M., Bronstein, A. M., & Kimmel, R. (2012). Photometric heat kernel signatures. In Scale space and variational methods in computer vision (pp. 616–627).
Lai, K., Bo, L., Ren, X., & Fox, D. (2011). A large-scale hierarchical multi-view RGB-D object dataset. In International conference on robotics and automation (ICRA) (pp. 1817–1824).
Lowe, D. G. (2004). Distinctive image features from scale-invariant keypoints. IJCV, 60, 91–110.
Luebke, D. P. (2001). A developer’s survey of polygonal simplification algorithms. Computer Graphics and Applications, 21, 24–35.
Rodolà, E., Albarelli, A., Bergamasco, F., & Torsello, A. (2013). A scale independent selection process for 3d object recognition in cluttered scenes. International Journal of Computer Vision, IJCV, 102((1–3)), 129–145.
Rusu, R., Blodow, N., & Beetz, M. (2009). Fast point feature histograms (FPFH) for 3d registration. In International conference on robotics and automation (ICRA) (pp. 3212–3217).
Tombari, F., Salti, S., & Di Stefano, L. (2011). A combined texture-shape descriptor for enhanced 3d feature matching. In International conference on image processing (ICIP) (pp. 809–812).
Tombari, F., Salti, S., & Stefano, L. (2010). Unique signatures of histograms for local surface description. In European conference on computer vision (ECCV) (Vol. 6313, pp. 356–369).
Torsello, A., Rodolà, E., & Albarelli, A. (2011). Sampling relevant points for surface registration. In Conference on 3D imaging, modeling, processing, visualization and transmission (3DIMPVT) (pp. 290–295).
Tuzel, O., Porikli, F., & Meer, P. (2006). Region covariance: A fast descriptor for detection and classification. In European conference on computer vision (ECCV) (pp. 589–600).
Tuzel, O., Porikli, F., & Meer, P. (2007). Human detection via classification on Riemannian manifolds. International conference on computer vision and pattern recognition (CVPR) (pp. 1–8).
Tuzel, O., Porikli, F., & Meer, P. (2008). Pedestrian detection via classification on Riemannian manifolds. Transactions on Pattern Analysis and Machine Intelligence, PAMI, 30, 1713–1727.
Wilks, S. S. (1932). Certain generalizations in the analysis of variance. Biometrika, 24, 471–494.
Yao, J., Odobez, J., et al. (2008). Fast human detection from videos using covariance features. In International workshop on visual surveillance, VS2008.
Zaharescu, A., Boyer, E., & Horaud, R. (2012). Keypoints and local descriptors of scalar functions on 2d manifolds. International Journal of Computer Vision, IJCV, 100, 78–98.
Zaharescu, A., Boyer, E., Varanasi, K., & Horaud, R. (2009). Surface feature detection and description with applications to mesh matching. In CVPR (pp. 373–380).
Acknowledgments
This work has been supported in part by the following research Project Grants: TIN2012-39203 and IPT-2012-0630-020000 awarded by the Spanish Government Ministry of Economy and Competitivity.
Author information
Authors and Affiliations
Corresponding author
Additional information
Communicated by S. Soatto.
Rights and permissions
About this article
Cite this article
Cirujeda, P., Dicente Cid, Y., Mateo, X. et al. A 3D Scene Registration Method via Covariance Descriptors and an Evolutionary Stable Strategy Game Theory Solver. Int J Comput Vis 115, 306–329 (2015). https://doi.org/10.1007/s11263-015-0820-2
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11263-015-0820-2