Abstract
Establishing a correspondence between two non-rigidly deforming shapes is one of the most fundamental problems in visual computing. Existing methods often show weak resilience when presented with challenges innate to real-world data such as noise, outliers, self-occlusion etc. On the other hand, auto-decoders have demonstrated strong expressive power in learning geometrically meaningful latent embeddings. However, their use in shape analysis has been limited. In this paper, we introduce an approach based on an auto-decoder framework, that learns a continuous shape-wise deformation field over a fixed template. By supervising the deformation field for points on-surface and regularizing for points off-surface through a novel Signed Distance Regularization (SDR), we learn an alignment between the template and shape volumes. Trained on clean water-tight meshes, without any data-augmentation, we demonstrate compelling performance on compromised data and real-world scans (Our code is available at https://github.com/Sentient07/IFMatch).
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Adams, B., Ovsjanikov, M., Wand, M., Seidel, H.P., Guibas, L.J.: Meshless modeling of deformable shapes and their motion. In: Proceedings of the 2008 ACM SIGGRAPH/Eurographics Symposium on Computer Animation, pp. 77–86. SCA 2008, Eurographics Association, Goslar, DEU (2008)
Aflalo, Y., Kimmel, R.: Spectral multidimensional scaling. Proc. Natl. Acad. Sci. 110(45), 18052–18057 (2013)
Allen, B., Curless, B., Popović, Z.: Articulated body deformation from range scan data. ACM Trans. Graph. 21(3), 612–619 (2002). https://doi.org/10.1145/566654.566626
Allen, B., Curless, B., Popović, Z.: The space of human body shapes: reconstruction and parameterization from range scans. ACM Trans. Graph. 22(3), 587–594 (2003). https://doi.org/10.1145/882262.882311
Anguelov, D., Srinivasan, P., Koller, D., Thrun, S., Rodgers, J., Davis, J.: Scape: shape completion and animation of people. ACM Trans. Graph. 24(3), 408–416 (2005). https://doi.org/10.1145/1073204.1073207
Atzmon, M., Lipman, Y.: Sal: sign agnostic learning of shapes from raw data. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2020)
Atzmon, M., Novotny, D., Vedaldi, A., Lipman, Y.: Augmenting implicit neural shape representations with explicit deformation fields. arXiv preprint arXiv:2108.08931 (2021)
Bhatnagar, B.L., Sminchisescu, C., Theobalt, C., Pons-Moll, G.: Loopreg: self-supervised learning of implicit surface correspondences, pose and shape for 3D human mesh registration. In: Larochelle, H., Ranzato, M., Hadsell, R., Balcan, M.F., Lin, H. (eds.) Advances in Neural Information Processing Systems, vol. 33, pp. 12909–12922. Curran Associates, Inc. (2020). https://proceedings.neurips.cc/paper/2020/file/970af30e481057c48f87e101b61e6994-Paper.pdf
Biasotti, S., Cerri, A., Bronstein, A., Bronstein, M.: Recent trends, applications, and perspectives in 3D shape similarity assessment. Comput. Graph. Forum 35(6), 87–119 (2016)
Bogo, F., Romero, J., Loper, M., Black, M.J.: FAUST: dataset and evaluation for 3D mesh registration. In: Proceedings IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, Piscataway, NJ, USA (2014)
Bogo, F., Romero, J., Pons-Moll, G., Black, M.J.: Dynamic FAUST: registering human bodies in motion. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5573–5582 (2017). https://doi.org/10.1109/CVPR.2017.591
Boscaini, D., Masci, J., Rodolà, E., Bronstein, M.: Learning shape correspondence with anisotropic convolutional neural networks. In: Lee, D., Sugiyama, M., Luxburg, U., Guyon, I., Garnett, R. (eds.) Advances in Neural Information Processing Systems, vol. 29. Curran Associates, Inc. (2016). https://proceedings.neurips.cc/paper/2016/file/228499b55310264a8ea0e27b6e7c6ab6-Paper.pdf
Burghard, O., Dieckmann, A., Klein, R.: Embedding shapes with green’s functions for global shape matching. Comput. Graph. 68, 1–10 (2017)
Chen, Z., Zhang, H.: Learning implicit fields for generative shape modeling. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5939–5948 (2019)
Chen, Z., Zhang, H.: Learning implicit fields for generative shape modeling. In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5932–5941 (2019)
Corona, E., Pumarola, A., Alenyà, G., Pons-Moll, G., Moreno-Noguer, F.: Smplicit: topology-aware generative model for clothed people. In: CVPR (2021)
Deng, Y., Yang, J., Tong, X.: Deformed implicit field: modeling 3D shapes with learned dense correspondence. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10286–10296 (2021)
Deprelle, T., Groueix, T., Fisher, M., Kim, V.G., Russell, B.C., Aubry, M.: Learning elementary structures for 3D shape generation and matching. In: NeurIPS (2019)
Donati, N., Sharma, A., Ovsjanikov, M.: Deep geometric functional maps: robust feature learning for shape correspondence. In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE (2020). https://doi.org/10.1109/cvpr42600.2020.00862
Eisenberger, M., Lähner, Z., Cremers, D.: Smooth shells: multi-scale shape registration with functional maps. In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 12262–12271 (2020)
Eisenberger, M., et al.: Neuromorph: unsupervised shape interpolation and correspondence in one go. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7473–7483 (2021)
Eisenberger, M., Toker, A., Leal-Taixé, L., Cremers, D.: Deep shells: unsupervised shape correspondence with optimal transport. arXiv preprint arXiv:2010.15261 (2020)
Ezuz, D., Ben-Chen, M.: Deblurring and denoising of maps between shapes. In: Computer Graphics Forum, vol. 36, pp. 165–174. Wiley Online Library (2017)
Gao, L., et al.: SDM-NET: deep generative network for structured deformable mesh. ACM Trans. Graph. 38(6) (2019). https://doi.org/10.1145/3355089.3356488
Genova, K., Cole, F., Vlasic, D., Sarna, A., Freeman, W.T., Funkhouser, T.A.: Learning shape templates with structured implicit functions. In: 2019 IEEE/CVF International Conference on Computer Vision (ICCV), pp. 7153–7163 (2019)
Groueix, T., Fisher, M., Kim, V.G., Russell, B., Aubry, M.: 3D-coded : 3D correspondences by deep deformation. In: ECCV (2018)
Groueix, T., Fisher, M., Kim, V., Russell, B., Aubry, M.: Unsupervised cycle-consistent deformation for shape matching. In: Symposium on Geometry Processing (SGP) (2019)
Güler, R.A., Neverova, N., Kokkinos, I.: Densepose: dense human pose estimation in the wild. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7297–7306 (2018)
Ha, D., Dai, A., Le, Q.V.: Hypernetworks (2016)
Halimi, O., Litany, O., Rodola, E., Bronstein, A.M., Kimmel, R.: Unsupervised learning of dense shape correspondence. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4370–4379 (2019)
Hao, Z., Averbuch-Elor, H., Snavely, N., Belongie, S.: Dualsdf: semantic shape manipulation using a two-level representation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (2020)
Hardy, R.L.: Multiquadric equations of topography and other irregular surfaces. J. Geophys. Res. 76(8), 1905–1915 (1971). https://doi.org/10.1029/jb076i008p01905
Jiang, C.M., Huang, J., Tagliasacchi, A., Guibas, L.J.: Shapeflow: learnable deformations among 3D shapes. ArXiv abs/2006.07982 (2020)
Joo, H., et al.: Panoptic studio: a massively multiview system for social interaction capture. TPAMI (2017)
Kanazawa, A., Black, M.J., Jacobs, D.W., Malik, J.: End-to-end recovery of human shape and pose. In: Computer Vision and Pattern Regognition (CVPR) (2018)
Kim, V.G., Lipman, Y., Funkhouser, T.: Blended intrinsic maps. ACM Trans. Graph. 30(4), 1–12 (2011). https://doi.org/10.1145/2010324.1964974
Kovnatsky, A., Bronstein, M.M., Bronstein, A.M., Glashoff, K., Kimmel, R.: Coupled quasi-harmonic bases. In: Computer Graphics Forum, vol. 32, pp. 439–448. Wiley Online Library (2013)
Lang, I., Ginzburg, D., Avidan, S., Raviv, D.: DPC: unsupervised Deep Point Correspondence via Cross and Self Construction. In: Proceedings of the International Conference on 3D Vision (3DV), pp. 1442–1451 (2021)
Litany, O., Remez, T., Rodola, E., Bronstein, A., Bronstein, M.: Deep functional maps: structured prediction for dense shape correspondence. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 5659–5667 (2017)
Liu, S., Zhang, Y., Peng, S., Shi, B., Pollefeys, M., Cui, Z.: DIST: rendering deep implicit signed distance function with differentiable sphere tracing. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2019–2028 (2020)
Loper, M., Mahmood, N., Romero, J., Pons-Moll, G., Black, M.J.: SMPL: a skinned multi-person linear model. ACM Trans. Graph. (Proc. SIGGRAPH Asia) 34(6), 248:1–248:16 (2015)
Marin, R., Rakotosaona, M.J., Melzi, S., Ovsjanikov, M.: Correspondence learning via linearly-invariant embedding. Proc. NeurIPS (2020)
Masci, J., Boscaini, D., Bronstein, M., Vandergheynst, P.: Geodesic convolutional neural networks on riemannian manifolds. In: Proceedings of the IEEE International Conference on Computer Vision Workshops, pp. 37–45 (2015)
Melzi, S., Marin, R., Rodolà, E., Castellani, U., Ren, J., Poulenard, A., Wonka, P., Ovsjanikov, M.: Shrec 2019: matching humans with different connectivity. In: Eurographics Workshop on 3D Object Retrieval, vol. 7 (2019)
Melzi, S., Ren, J., Rodolà, E., Sharma, A., Wonka, P., Ovsjanikov, M.: Zoomout: spectral upsampling for efficient shape correspondence. ACM Trans. Graph. 38(6) (2019). https://doi.org/10.1145/3355089.3356524
Mescheder, L., Oechsle, M., Niemeyer, M., Nowozin, S., Geiger, A.: Occupancy networks: learning 3D reconstruction in function space. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4460–4470 (2019)
Mezghanni, M., Boulkenafed, M., Lieutier, A., Ovsjanikov, M.: Physically-aware generative network for 3D shape modeling. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 9330–9341 (2021)
Mildenhall, B., Srinivasan, P.P., Tancik, M., Barron, J.T., Ramamoorthi, R., Ng, R.: Nerf: representing scenes as neural radiance fields for view synthesis. In: ECCV (2020)
Monti, F., Boscaini, D., Masci, J., Rodola, E., Svoboda, J., Bronstein, M.M.: Geometric deep learning on graphs and manifolds using mixture model CNNs. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 5115–5124 (2017)
Niemeyer, M., Geiger, A.: Giraffe: representing scenes as compositional generative neural feature fields. In: Proceedings IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2021)
Niemeyer, M., Mescheder, L., Oechsle, M., Geiger, A.: Occupancy flow: 4D reconstruction by learning particle dynamics. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 5379–5389 (2019)
Omran, M., Lassner, C., Pons-Moll, G., Gehler, P.V., Schiele, B.: Neural body fitting: unifying deep learning and model-based human pose and shape estimation. Verona, Italy (2018)
Ovsjanikov, M., Ben-Chen, M., Solomon, J., Butscher, A., Guibas, L.: Functional maps: a flexible representation of maps between shapes. ACM Trans. Graph. (TOG) 31(4), 1–11 (2012)
Park, J.J., Florence, P., Straub, J., Newcombe, R., Lovegrove, S.: Deepsdf: learning continuous signed distance functions for shape representation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 165–174 (2019)
Paschalidou, D., Gool, L.V., Geiger, A.: Learning unsupervised hierarchical part decomposition of 3D objects from a single RGB image. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1060–1070 (2020)
Pons-Moll, G., Pujades, S., Hu, S., Black, M.: Clothcap: seamless 4D clothing capture and retargeting. ACM Trans. Graph. (Proc. SIGGRAPH) 36(4) (2017). https://doi.org/10.1145/3072959.3073711, two first authors contributed equally
Pons-Moll, G., Romero, J., Mahmood, N., Black, M.J.: Dyna: a model of dynamic human shape in motion 34(4) (2015). https://doi.org/10.1145/2766993
Poulenard, A., Ovsjanikov, M.: Multi-directional geodesic neural networks via equivariant convolution. ACM Trans. Graph. (TOG) 37(6), 1–14 (2018)
Ren, J., Poulenard, A., Wonka, P., Ovsjanikov, M.: Continuous and orientation-preserving correspondences via functional maps. ACM Trans. Graph. 37(6) (2018). https://doi.org/10.1145/3272127.3275040
Rodolà, E., Cosmo, L., Bronstein, M.M., Torsello, A., Cremers, D.: Partial functional correspondence. Comput. Graph. Forum 36(1), 222–236 (2016). https://doi.org/10.1111/cgf.12797
Rodolà, E., Cosmo, L., Bronstein, M.M., Torsello, A., Cremers, D.: Partial functional correspondence. In: Computer Graphics Forum, vol. 36, pp. 222–236. Wiley Online Library (2017)
Roufosse, J.M., Sharma, A., Ovsjanikov, M.: Unsupervised deep learning for structured shape matching. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 1617–1627 (2019)
Sahillioğlu, Y.: Recent advances in shape correspondence. Vis. Comput. 36(8), 1705–1721 (2020)
Schwarz, K., Liao, Y., Niemeyer, M., Geiger, A.: Graf: generative radiance fields for 3D-aware image synthesis. In: Advances in Neural Information Processing Systems (NeurIPS) (2020)
Sharp, N., Attaiki, S., Crane, K., Ovsjanikov, M.: Diffusionnet: discretization agnostic learning on surfaces (2021)
Sitzmann, V., Martel, J., Bergman, A., Lindell, D., Wetzstein, G.: Implicit neural representations with periodic activation functions. Adv. Neural Inf. Process. Syst. 33 (2020)
Sitzmann, V., Zollhöfer, M., Wetzstein, G.: Scene representation networks: continuous 3D-structure-aware neural scene representations. arXiv preprint arXiv:1906.01618 (2019)
Takikawa, T., et al.: Neural geometric level of detail: Real-time rendering with implicit 3D shapes. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 11358–11367 (2021)
Takikawa, T., et al.: Neural geometric level of detail: real-time rendering with implicit 3D shapes (2021)
Tam, G.K., et al.: Registration of 3D point clouds and meshes: a survey from rigid to nonrigid. IEEE Trans. Visual Comput. Graph. 19(7), 1199–1217 (2012)
Tancik, M., et al.: Fourier features let networks learn high frequency functions in low dimensional domains. NeurIPS (2020)
Tiwari, G., Sarafianos, N., Tung, T., Pons-Moll, G.: Neural-gif: neural generalized implicit functions for animating people in clothing. ArXiv abs/2108.08807 (2021)
Van Kaick, O., Zhang, H., Hamarneh, G., Cohen-Or, D.: A survey on shape correspondence. Comput. Graph. Forum 30(6), 1681–1707 (2011)
Varol, G., Laptev, I., Schmid, C., Zisserman, A.: Synthetic humans for action recognition from unseen viewpoints 129(7), 2264–2287 (2021). https://doi.org/10.1007/s11263-021-01467-7
Varol, G., et al.: Learning from synthetic humans. In: CVPR (2017)
Vasu, S., Talabot, N., Lukoianov, A., Baqué, P., Donier, J., Fua, P.: Hybridsdf: combining free form shapes and geometric primitives for effective shape manipulation. ArXiv abs/2109.10767 (2021)
Wang, W., Ceylan, D., Mech, R., Neumann, U.: 3DN: 3D deformation network. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2019)
Wiersma, R., Eisemann, E., Hildebrandt, K.: CNNs on surfaces using rotation-equivariant features. ACM Trans. Graph. (TOG) 39(4), 1–92 (2020)
Wu, R., Zhuang, Y., Xu, K., Zhang, H., Chen, B.: PQ-NET: A generative part seq2seq network for 3D shapes. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2020)
Xie, Y., et al.: Neural fields in visual computing and beyond (2021). https://neuralfields.cs.brown.edu/
Yu, T., et al.: Simulcap : single-view human performance capture with cloth simulation. In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5499–5509 (2019)
Zadeh, A., Lim, Y.C., Liang, P.P., Morency, L.P.: Variational auto-decoder. ArXiv abs/1903.00840 (2019)
Zeng, Y., Qian, Y., Zhu, Z., Hou, J., Yuan, H., He, Y.: CorrNet3D: unsupervised end-to-end learning of dense correspondence for 3D point clouds. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2021)
Zheng, Z., Yu, T., Dai, Q., Liu, Y.: Deep implicit templates for 3D shape representation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1429–1439 (2021)
Zuffi, S., Kanazawa, A., Black, M.J.: Lions and tigers and bears: capturing non-rigid, 3D, articulated shape from images. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE Computer Society (2018)
Zuffi, S., Kanazawa, A., Jacobs, D., Black, M.J.: 3D menagerie: modeling the 3D shape and pose of animals. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2017)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Sundararaman, R., Pai, G., Ovsjanikov, M. (2022). Implicit Field Supervision for Robust Non-rigid Shape Matching. In: Avidan, S., Brostow, G., Cissé, M., Farinella, G.M., Hassner, T. (eds) Computer Vision – ECCV 2022. ECCV 2022. Lecture Notes in Computer Science, vol 13663. Springer, Cham. https://doi.org/10.1007/978-3-031-20062-5_20
Download citation
DOI: https://doi.org/10.1007/978-3-031-20062-5_20
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-20061-8
Online ISBN: 978-3-031-20062-5
eBook Packages: Computer ScienceComputer Science (R0)