Abstract
The Shape-from-Template (SfT) problem is to recover the 3D shape of a deformable object from a single image, given a 3D template and a deformation constraint. We propose Particle-SfT, a new SfT algorithm which handles isometric and non-isometric deformations. We build Particle-SfT upon a particle system guided by deformation and reprojection constraint projections. Reconstruction is achieved by evolving particles to a globally attractive equilibrium, while taking observable external forces such as gravity into account, if any. Particle-SfT may be used to refine an existing initial shape. However, in practice we simply use the template as initial guess. This is because, as opposed to the existing refining methods, Particle-SfT has an extremely wide convergence basin. Particle-SfT is also faster than the existing refining methods. This is because it moves pieces of the shape’s mesh independently to achieve larger step size by optimal constraint projection. We proved its convergence to a fixed-point. We experimented it with synthetic and real data. It has the same accuracy as the best performing isometric method and consistently outperforms all existing elastic methods in almost all cases, while being much faster.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Agarwal, R., Meehan, M., & O’Regan, D. (2001). Fixed point theory and applications. Cambridge: Cambridge University Press.
Agudo, A. & Moreno-Noguer F. (2015). Simultaneous pose and non-rigid shape with particle dynamics. In CVPR.
Agudo, A., Agapito, L., Calvo, B., & Montiel, J. (2014). Good vibrations: a modal analysis approach for sequential non-rigid structure from motion. In CVPR.
Agudo, A., Moreno-Noguer, F., Calvo, B., & Montiel, J. (2016). Sequential non-rigid structure from motion using physical priors. IEEE Transactions on Pattern Analysis and Machine Intelligence, 38(5), 979–994.
Alcantarilla, P. F., Bartoli, A., & Davison, A. J. (2012). KAZE features. In ECCV.
Bartoli, A., Gérard, Y., Chadebecq, F., Collins, T., & Pizarro, D. (2015). Shape-from-Template. IEEE Transactions on Pattern Analysis and Machine Intelligence, 37(10), 2099–2118.
Belluce, L., & Kirk, W. (1969). Some fixed point theorems in metric and banach spaces. Canadian Mathematical Bulletin, 12(4), 481–491.
Brouwer, L. (1910). ber abbildungen von mannigfaltigkeiten. Mathematische Annalen, 71, 97–115.
Brunet, F., Bartoli, A., & Hartley, R. (2014). Monocular template-based 3D surface reconstruction: convex inextensible and nonconvex isometric methods. Computer Vision and Image Understanding, 125, 138–154.
Chhatkuli, A., Pizarro, D. & Bartoli, A. (2014). Stable template-based isometric 3D reconstruction in all imaging conditions by linear least-squares. In CVPR.
Goldstein, H. (1980). Classical mechanics. Salt Lake City: Addison-Wesley.
Haouchine, N., Dequidt, J., Berger, M., & Cotin, S. (2014). Single view augmentation of 3D elastic objects. In ISMAR.
Ilić, S., & Fua, P. (2007). Non-linear beam model for tracking large deformation. In ICCV.
Kass, M., Witkin, A., & Terzopoulos, D. (1988). Snakes: active contour models. International Journal of Computer Vision, 1(4), 321–331.
Lowe, D. (1999). Object recognition from local scale-invariant features. In ICCV.
Lyapunov, A. M. (1892). The general problem of the stability of motion (in russian). Doctoral dissertation, Univ. Kharkov.
Malti, A., Hartley, R., Bartoli, A., & Kim, J. (2013) Monocular template-based 3D reconstruction of extensible surfaces with linear elasticity. In CVPR.
Malti, A., Bartoli, A., & Hartley, R. (2015). A linear least-squares solution to elastic shape-from-template. In CVPR.
Marton, Z., Rusu, R., & Beetz, M. (2009) On fast surface reconstruction methods for large and noisy point clouds. In ICRA.
Metaxas, D. (1993). Constrained deformable superquadratics and nonrigid motion tracking. IEEE Transactions on Pattern Analysis and Machine Intelligence, 15(6), 580–591.
Moreno-Noguer, F., Salzmann, M., Lepetit, V., & Fua P. (2009). Capturing 3D stretchable surfaces from single images in closed form. In CVPR.
Müller, M., Heidelberger, B., Hennix, M., & Ratcliff, J. (2006). Position based dynamics. In Workshop in virtual reality interactions and physical simulation VRIPHYS.
Östlund, J., Varol, A., Ngo, T.D., & Fua., P. (2012). Laplacian meshes for monocular 3D shape recovery. In ECCV.
Perriollat, M., Hartley, R., & Bartoli, A. (2011). Monocular template-based reconstruction of inextensible surfaces. International Journal of Computer Vision, 95(2), 124–137.
Pilet, J., Lepetit, V., & Fua, P. (2008). Fast non-rigid surface detection, registration and realistic augmentation. International Journal of Computer Vision, 76(2), 109–122.
Pizarro, D., & Bartoli, A. (2012). Feature-based deformable surface detection with self-occlusion reasoning. International Journal of Computer Vision, 97(1), 54–70.
Provot, X. (1996). Deformation constraints in a mass-spring model to describe rigid cloth behaviour. In Graphics interface, Canadian Information Processing Society , pp. 147–154.
Rhoades, B. (1977). A comparison of various definitions of contractive mappings. Transaction of the American Mathematical Society, 226, 257–290.
Salzmann, M. & Fua P. (2009). Reconstructing sharply folding surfaces: A convex formulation. In CVPR.
Salzmann, M., & Fua, P. (2011). Linear local models for monocular reconstruction of deformable surfaces. IEEE Transactions on Pattern Analysis and Machine Intelligence, 33(5), 931–944.
Salzmann, M. & Urtasun R. (2011). Physically-based motion models for 3D tracking: a convex formulation. In ICCV.
Salzmann, M., Moreno-Noguer, F., Lepetit, V., & Fua P. (2008). Closed-form solution to non-rigid 3D surface registration. In ECCV.
Varol, A., Salzmann, M., Fua, P., & Urtasun, R. (2012). A constrained latent variable model. In CVPR.
Acknowledgments
We thank the authors of (Brunet et al. 2014; Chhatkuli et al. 2014; Bartoli et al. 2015; Östlund et al. 2012; Salzmann and Fua 2011; Varol et al. 2012; Malti et al. 2015) for making their source codes and/or datasets available to us. This research has received funding from the EU’s FP7 through the ERC Research Grant 307483 FLEXABLE.
Author information
Authors and Affiliations
Corresponding author
Additional information
Communicated by V. Lepetit.
Rights and permissions
About this article
Cite this article
Özgür, E., Bartoli, A. Particle-SfT: A Provably-Convergent, Fast Shape-from-Template Algorithm. Int J Comput Vis 123, 184–205 (2017). https://doi.org/10.1007/s11263-016-0968-4
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11263-016-0968-4