Skip to main content

Implicit Field Supervision for Robust Non-rigid Shape Matching

  • Conference paper
  • First Online:
Computer Vision – ECCV 2022 (ECCV 2022)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13663))

Included in the following conference series:

Abstract

Establishing a correspondence between two non-rigidly deforming shapes is one of the most fundamental problems in visual computing. Existing methods often show weak resilience when presented with challenges innate to real-world data such as noise, outliers, self-occlusion etc. On the other hand, auto-decoders have demonstrated strong expressive power in learning geometrically meaningful latent embeddings. However, their use in shape analysis has been limited. In this paper, we introduce an approach based on an auto-decoder framework, that learns a continuous shape-wise deformation field over a fixed template. By supervising the deformation field for points on-surface and regularizing for points off-surface through a novel Signed Distance Regularization (SDR), we learn an alignment between the template and shape volumes. Trained on clean water-tight meshes, without any data-augmentation, we demonstrate compelling performance on compromised data and real-world scans (Our code is available at https://github.com/Sentient07/IFMatch).

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 89.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 119.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Adams, B., Ovsjanikov, M., Wand, M., Seidel, H.P., Guibas, L.J.: Meshless modeling of deformable shapes and their motion. In: Proceedings of the 2008 ACM SIGGRAPH/Eurographics Symposium on Computer Animation, pp. 77–86. SCA 2008, Eurographics Association, Goslar, DEU (2008)

    Google Scholar 

  2. Aflalo, Y., Kimmel, R.: Spectral multidimensional scaling. Proc. Natl. Acad. Sci. 110(45), 18052–18057 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  3. Allen, B., Curless, B., Popović, Z.: Articulated body deformation from range scan data. ACM Trans. Graph. 21(3), 612–619 (2002). https://doi.org/10.1145/566654.566626

    Article  Google Scholar 

  4. Allen, B., Curless, B., Popović, Z.: The space of human body shapes: reconstruction and parameterization from range scans. ACM Trans. Graph. 22(3), 587–594 (2003). https://doi.org/10.1145/882262.882311

    Article  Google Scholar 

  5. Anguelov, D., Srinivasan, P., Koller, D., Thrun, S., Rodgers, J., Davis, J.: Scape: shape completion and animation of people. ACM Trans. Graph. 24(3), 408–416 (2005). https://doi.org/10.1145/1073204.1073207

    Article  Google Scholar 

  6. Atzmon, M., Lipman, Y.: Sal: sign agnostic learning of shapes from raw data. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2020)

    Google Scholar 

  7. Atzmon, M., Novotny, D., Vedaldi, A., Lipman, Y.: Augmenting implicit neural shape representations with explicit deformation fields. arXiv preprint arXiv:2108.08931 (2021)

  8. Bhatnagar, B.L., Sminchisescu, C., Theobalt, C., Pons-Moll, G.: Loopreg: self-supervised learning of implicit surface correspondences, pose and shape for 3D human mesh registration. In: Larochelle, H., Ranzato, M., Hadsell, R., Balcan, M.F., Lin, H. (eds.) Advances in Neural Information Processing Systems, vol. 33, pp. 12909–12922. Curran Associates, Inc. (2020). https://proceedings.neurips.cc/paper/2020/file/970af30e481057c48f87e101b61e6994-Paper.pdf

  9. Biasotti, S., Cerri, A., Bronstein, A., Bronstein, M.: Recent trends, applications, and perspectives in 3D shape similarity assessment. Comput. Graph. Forum 35(6), 87–119 (2016)

    Article  Google Scholar 

  10. Bogo, F., Romero, J., Loper, M., Black, M.J.: FAUST: dataset and evaluation for 3D mesh registration. In: Proceedings IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, Piscataway, NJ, USA (2014)

    Google Scholar 

  11. Bogo, F., Romero, J., Pons-Moll, G., Black, M.J.: Dynamic FAUST: registering human bodies in motion. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5573–5582 (2017). https://doi.org/10.1109/CVPR.2017.591

  12. Boscaini, D., Masci, J., Rodolà, E., Bronstein, M.: Learning shape correspondence with anisotropic convolutional neural networks. In: Lee, D., Sugiyama, M., Luxburg, U., Guyon, I., Garnett, R. (eds.) Advances in Neural Information Processing Systems, vol. 29. Curran Associates, Inc. (2016). https://proceedings.neurips.cc/paper/2016/file/228499b55310264a8ea0e27b6e7c6ab6-Paper.pdf

  13. Burghard, O., Dieckmann, A., Klein, R.: Embedding shapes with green’s functions for global shape matching. Comput. Graph. 68, 1–10 (2017)

    Article  Google Scholar 

  14. Chen, Z., Zhang, H.: Learning implicit fields for generative shape modeling. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5939–5948 (2019)

    Google Scholar 

  15. Chen, Z., Zhang, H.: Learning implicit fields for generative shape modeling. In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5932–5941 (2019)

    Google Scholar 

  16. Corona, E., Pumarola, A., Alenyà, G., Pons-Moll, G., Moreno-Noguer, F.: Smplicit: topology-aware generative model for clothed people. In: CVPR (2021)

    Google Scholar 

  17. Deng, Y., Yang, J., Tong, X.: Deformed implicit field: modeling 3D shapes with learned dense correspondence. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10286–10296 (2021)

    Google Scholar 

  18. Deprelle, T., Groueix, T., Fisher, M., Kim, V.G., Russell, B.C., Aubry, M.: Learning elementary structures for 3D shape generation and matching. In: NeurIPS (2019)

    Google Scholar 

  19. Donati, N., Sharma, A., Ovsjanikov, M.: Deep geometric functional maps: robust feature learning for shape correspondence. In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE (2020). https://doi.org/10.1109/cvpr42600.2020.00862

  20. Eisenberger, M., Lähner, Z., Cremers, D.: Smooth shells: multi-scale shape registration with functional maps. In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 12262–12271 (2020)

    Google Scholar 

  21. Eisenberger, M., et al.: Neuromorph: unsupervised shape interpolation and correspondence in one go. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7473–7483 (2021)

    Google Scholar 

  22. Eisenberger, M., Toker, A., Leal-Taixé, L., Cremers, D.: Deep shells: unsupervised shape correspondence with optimal transport. arXiv preprint arXiv:2010.15261 (2020)

  23. Ezuz, D., Ben-Chen, M.: Deblurring and denoising of maps between shapes. In: Computer Graphics Forum, vol. 36, pp. 165–174. Wiley Online Library (2017)

    Google Scholar 

  24. Gao, L., et al.: SDM-NET: deep generative network for structured deformable mesh. ACM Trans. Graph. 38(6) (2019). https://doi.org/10.1145/3355089.3356488

  25. Genova, K., Cole, F., Vlasic, D., Sarna, A., Freeman, W.T., Funkhouser, T.A.: Learning shape templates with structured implicit functions. In: 2019 IEEE/CVF International Conference on Computer Vision (ICCV), pp. 7153–7163 (2019)

    Google Scholar 

  26. Groueix, T., Fisher, M., Kim, V.G., Russell, B., Aubry, M.: 3D-coded : 3D correspondences by deep deformation. In: ECCV (2018)

    Google Scholar 

  27. Groueix, T., Fisher, M., Kim, V., Russell, B., Aubry, M.: Unsupervised cycle-consistent deformation for shape matching. In: Symposium on Geometry Processing (SGP) (2019)

    Google Scholar 

  28. Güler, R.A., Neverova, N., Kokkinos, I.: Densepose: dense human pose estimation in the wild. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7297–7306 (2018)

    Google Scholar 

  29. Ha, D., Dai, A., Le, Q.V.: Hypernetworks (2016)

    Google Scholar 

  30. Halimi, O., Litany, O., Rodola, E., Bronstein, A.M., Kimmel, R.: Unsupervised learning of dense shape correspondence. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4370–4379 (2019)

    Google Scholar 

  31. Hao, Z., Averbuch-Elor, H., Snavely, N., Belongie, S.: Dualsdf: semantic shape manipulation using a two-level representation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (2020)

    Google Scholar 

  32. Hardy, R.L.: Multiquadric equations of topography and other irregular surfaces. J. Geophys. Res. 76(8), 1905–1915 (1971). https://doi.org/10.1029/jb076i008p01905

    Article  Google Scholar 

  33. Jiang, C.M., Huang, J., Tagliasacchi, A., Guibas, L.J.: Shapeflow: learnable deformations among 3D shapes. ArXiv abs/2006.07982 (2020)

    Google Scholar 

  34. Joo, H., et al.: Panoptic studio: a massively multiview system for social interaction capture. TPAMI (2017)

    Google Scholar 

  35. Kanazawa, A., Black, M.J., Jacobs, D.W., Malik, J.: End-to-end recovery of human shape and pose. In: Computer Vision and Pattern Regognition (CVPR) (2018)

    Google Scholar 

  36. Kim, V.G., Lipman, Y., Funkhouser, T.: Blended intrinsic maps. ACM Trans. Graph. 30(4), 1–12 (2011). https://doi.org/10.1145/2010324.1964974

    Article  Google Scholar 

  37. Kovnatsky, A., Bronstein, M.M., Bronstein, A.M., Glashoff, K., Kimmel, R.: Coupled quasi-harmonic bases. In: Computer Graphics Forum, vol. 32, pp. 439–448. Wiley Online Library (2013)

    Google Scholar 

  38. Lang, I., Ginzburg, D., Avidan, S., Raviv, D.: DPC: unsupervised Deep Point Correspondence via Cross and Self Construction. In: Proceedings of the International Conference on 3D Vision (3DV), pp. 1442–1451 (2021)

    Google Scholar 

  39. Litany, O., Remez, T., Rodola, E., Bronstein, A., Bronstein, M.: Deep functional maps: structured prediction for dense shape correspondence. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 5659–5667 (2017)

    Google Scholar 

  40. Liu, S., Zhang, Y., Peng, S., Shi, B., Pollefeys, M., Cui, Z.: DIST: rendering deep implicit signed distance function with differentiable sphere tracing. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2019–2028 (2020)

    Google Scholar 

  41. Loper, M., Mahmood, N., Romero, J., Pons-Moll, G., Black, M.J.: SMPL: a skinned multi-person linear model. ACM Trans. Graph. (Proc. SIGGRAPH Asia) 34(6), 248:1–248:16 (2015)

    Google Scholar 

  42. Marin, R., Rakotosaona, M.J., Melzi, S., Ovsjanikov, M.: Correspondence learning via linearly-invariant embedding. Proc. NeurIPS (2020)

    Google Scholar 

  43. Masci, J., Boscaini, D., Bronstein, M., Vandergheynst, P.: Geodesic convolutional neural networks on riemannian manifolds. In: Proceedings of the IEEE International Conference on Computer Vision Workshops, pp. 37–45 (2015)

    Google Scholar 

  44. Melzi, S., Marin, R., Rodolà, E., Castellani, U., Ren, J., Poulenard, A., Wonka, P., Ovsjanikov, M.: Shrec 2019: matching humans with different connectivity. In: Eurographics Workshop on 3D Object Retrieval, vol. 7 (2019)

    Google Scholar 

  45. Melzi, S., Ren, J., Rodolà, E., Sharma, A., Wonka, P., Ovsjanikov, M.: Zoomout: spectral upsampling for efficient shape correspondence. ACM Trans. Graph. 38(6) (2019). https://doi.org/10.1145/3355089.3356524

  46. Mescheder, L., Oechsle, M., Niemeyer, M., Nowozin, S., Geiger, A.: Occupancy networks: learning 3D reconstruction in function space. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4460–4470 (2019)

    Google Scholar 

  47. Mezghanni, M., Boulkenafed, M., Lieutier, A., Ovsjanikov, M.: Physically-aware generative network for 3D shape modeling. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 9330–9341 (2021)

    Google Scholar 

  48. Mildenhall, B., Srinivasan, P.P., Tancik, M., Barron, J.T., Ramamoorthi, R., Ng, R.: Nerf: representing scenes as neural radiance fields for view synthesis. In: ECCV (2020)

    Google Scholar 

  49. Monti, F., Boscaini, D., Masci, J., Rodola, E., Svoboda, J., Bronstein, M.M.: Geometric deep learning on graphs and manifolds using mixture model CNNs. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 5115–5124 (2017)

    Google Scholar 

  50. Niemeyer, M., Geiger, A.: Giraffe: representing scenes as compositional generative neural feature fields. In: Proceedings IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2021)

    Google Scholar 

  51. Niemeyer, M., Mescheder, L., Oechsle, M., Geiger, A.: Occupancy flow: 4D reconstruction by learning particle dynamics. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 5379–5389 (2019)

    Google Scholar 

  52. Omran, M., Lassner, C., Pons-Moll, G., Gehler, P.V., Schiele, B.: Neural body fitting: unifying deep learning and model-based human pose and shape estimation. Verona, Italy (2018)

    Google Scholar 

  53. Ovsjanikov, M., Ben-Chen, M., Solomon, J., Butscher, A., Guibas, L.: Functional maps: a flexible representation of maps between shapes. ACM Trans. Graph. (TOG) 31(4), 1–11 (2012)

    Article  Google Scholar 

  54. Park, J.J., Florence, P., Straub, J., Newcombe, R., Lovegrove, S.: Deepsdf: learning continuous signed distance functions for shape representation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 165–174 (2019)

    Google Scholar 

  55. Paschalidou, D., Gool, L.V., Geiger, A.: Learning unsupervised hierarchical part decomposition of 3D objects from a single RGB image. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1060–1070 (2020)

    Google Scholar 

  56. Pons-Moll, G., Pujades, S., Hu, S., Black, M.: Clothcap: seamless 4D clothing capture and retargeting. ACM Trans. Graph. (Proc. SIGGRAPH) 36(4) (2017). https://doi.org/10.1145/3072959.3073711, two first authors contributed equally

  57. Pons-Moll, G., Romero, J., Mahmood, N., Black, M.J.: Dyna: a model of dynamic human shape in motion 34(4) (2015). https://doi.org/10.1145/2766993

  58. Poulenard, A., Ovsjanikov, M.: Multi-directional geodesic neural networks via equivariant convolution. ACM Trans. Graph. (TOG) 37(6), 1–14 (2018)

    Article  Google Scholar 

  59. Ren, J., Poulenard, A., Wonka, P., Ovsjanikov, M.: Continuous and orientation-preserving correspondences via functional maps. ACM Trans. Graph. 37(6) (2018). https://doi.org/10.1145/3272127.3275040

  60. Rodolà, E., Cosmo, L., Bronstein, M.M., Torsello, A., Cremers, D.: Partial functional correspondence. Comput. Graph. Forum 36(1), 222–236 (2016). https://doi.org/10.1111/cgf.12797

  61. Rodolà, E., Cosmo, L., Bronstein, M.M., Torsello, A., Cremers, D.: Partial functional correspondence. In: Computer Graphics Forum, vol. 36, pp. 222–236. Wiley Online Library (2017)

    Google Scholar 

  62. Roufosse, J.M., Sharma, A., Ovsjanikov, M.: Unsupervised deep learning for structured shape matching. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 1617–1627 (2019)

    Google Scholar 

  63. Sahillioğlu, Y.: Recent advances in shape correspondence. Vis. Comput. 36(8), 1705–1721 (2020)

    Article  Google Scholar 

  64. Schwarz, K., Liao, Y., Niemeyer, M., Geiger, A.: Graf: generative radiance fields for 3D-aware image synthesis. In: Advances in Neural Information Processing Systems (NeurIPS) (2020)

    Google Scholar 

  65. Sharp, N., Attaiki, S., Crane, K., Ovsjanikov, M.: Diffusionnet: discretization agnostic learning on surfaces (2021)

    Google Scholar 

  66. Sitzmann, V., Martel, J., Bergman, A., Lindell, D., Wetzstein, G.: Implicit neural representations with periodic activation functions. Adv. Neural Inf. Process. Syst. 33 (2020)

    Google Scholar 

  67. Sitzmann, V., Zollhöfer, M., Wetzstein, G.: Scene representation networks: continuous 3D-structure-aware neural scene representations. arXiv preprint arXiv:1906.01618 (2019)

  68. Takikawa, T., et al.: Neural geometric level of detail: Real-time rendering with implicit 3D shapes. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 11358–11367 (2021)

    Google Scholar 

  69. Takikawa, T., et al.: Neural geometric level of detail: real-time rendering with implicit 3D shapes (2021)

    Google Scholar 

  70. Tam, G.K., et al.: Registration of 3D point clouds and meshes: a survey from rigid to nonrigid. IEEE Trans. Visual Comput. Graph. 19(7), 1199–1217 (2012)

    Article  Google Scholar 

  71. Tancik, M., et al.: Fourier features let networks learn high frequency functions in low dimensional domains. NeurIPS (2020)

    Google Scholar 

  72. Tiwari, G., Sarafianos, N., Tung, T., Pons-Moll, G.: Neural-gif: neural generalized implicit functions for animating people in clothing. ArXiv abs/2108.08807 (2021)

    Google Scholar 

  73. Van Kaick, O., Zhang, H., Hamarneh, G., Cohen-Or, D.: A survey on shape correspondence. Comput. Graph. Forum 30(6), 1681–1707 (2011)

    Article  Google Scholar 

  74. Varol, G., Laptev, I., Schmid, C., Zisserman, A.: Synthetic humans for action recognition from unseen viewpoints 129(7), 2264–2287 (2021). https://doi.org/10.1007/s11263-021-01467-7

  75. Varol, G., et al.: Learning from synthetic humans. In: CVPR (2017)

    Google Scholar 

  76. Vasu, S., Talabot, N., Lukoianov, A., Baqué, P., Donier, J., Fua, P.: Hybridsdf: combining free form shapes and geometric primitives for effective shape manipulation. ArXiv abs/2109.10767 (2021)

    Google Scholar 

  77. Wang, W., Ceylan, D., Mech, R., Neumann, U.: 3DN: 3D deformation network. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2019)

    Google Scholar 

  78. Wiersma, R., Eisemann, E., Hildebrandt, K.: CNNs on surfaces using rotation-equivariant features. ACM Trans. Graph. (TOG) 39(4), 1–92 (2020)

    Article  Google Scholar 

  79. Wu, R., Zhuang, Y., Xu, K., Zhang, H., Chen, B.: PQ-NET: A generative part seq2seq network for 3D shapes. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2020)

    Google Scholar 

  80. Xie, Y., et al.: Neural fields in visual computing and beyond (2021). https://neuralfields.cs.brown.edu/

  81. Yu, T., et al.: Simulcap : single-view human performance capture with cloth simulation. In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5499–5509 (2019)

    Google Scholar 

  82. Zadeh, A., Lim, Y.C., Liang, P.P., Morency, L.P.: Variational auto-decoder. ArXiv abs/1903.00840 (2019)

    Google Scholar 

  83. Zeng, Y., Qian, Y., Zhu, Z., Hou, J., Yuan, H., He, Y.: CorrNet3D: unsupervised end-to-end learning of dense correspondence for 3D point clouds. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2021)

    Google Scholar 

  84. Zheng, Z., Yu, T., Dai, Q., Liu, Y.: Deep implicit templates for 3D shape representation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1429–1439 (2021)

    Google Scholar 

  85. Zuffi, S., Kanazawa, A., Black, M.J.: Lions and tigers and bears: capturing non-rigid, 3D, articulated shape from images. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE Computer Society (2018)

    Google Scholar 

  86. Zuffi, S., Kanazawa, A., Jacobs, D., Black, M.J.: 3D menagerie: modeling the 3D shape and pose of animals. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2017)

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Ramana Sundararaman .

Editor information

Editors and Affiliations

1 Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (pdf 5738 KB)

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Sundararaman, R., Pai, G., Ovsjanikov, M. (2022). Implicit Field Supervision for Robust Non-rigid Shape Matching. In: Avidan, S., Brostow, G., Cissé, M., Farinella, G.M., Hassner, T. (eds) Computer Vision – ECCV 2022. ECCV 2022. Lecture Notes in Computer Science, vol 13663. Springer, Cham. https://doi.org/10.1007/978-3-031-20062-5_20

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-20062-5_20

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-20061-8

  • Online ISBN: 978-3-031-20062-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics