Skip to main content
SpringerLink
Log in
Book cover

European Conference on Computer Vision

ECCV 2020: Computer Vision – ECCV 2020 pp 248–263Cite as

Ladybird: Quasi-Monte Carlo Sampling for Deep Implicit Field Based 3D Reconstruction with Symmetry

  • Conference paper
  • First Online:

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 12346))

Abstract

Deep implicit field regression methods are effective for 3D reconstruction from single-view images. However, the impact of different sampling patterns on the reconstruction quality is not well-understood. In this work, we first study the effect of point set discrepancy on the network training. Based on Farthest Point Sampling algorithm, we propose a sampling scheme that theoretically encourages better generalization performance, and results in fast convergence for SGD-based optimization algorithms. Secondly, based on the reflective symmetry of an object, we propose a feature fusion method that alleviates issues due to self-occlusions which makes it difficult to utilize local image features. Our proposed system Ladybird is able to create high quality 3D object reconstructions from a single input image. We evaluate Ladybird on a large scale 3D dataset (ShapeNet) demonstrating highly competitive results in terms of Chamfer distance, Earth Mover’s distance and Intersection Over Union (IoU).

Y. Xu and T. Fan—These two authors contribute equally.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Notes

  1. 1.

    ShapeNet data set is aligned, and most objects are symmetric about xy plane.

  2. 2.

    For two point set \(S_1\) and \(S_2\), CD is defined to be \(\displaystyle \sum _{x \in S_1} \min _{y \in S_2} \Vert x-y\Vert ^2_2 + \sum _{y \in S_2} \min _{ x \in S_1} \Vert x-y\Vert ^2_2\).

References

  1. Uni(corn—form) tool kit. https://utk-team.github.io/utk/

  2. Abadi, M., et al.: Tensorflow: a system for large-scale machine learning. In: 12th \(\{\)USENIX\(\}\) Symposium on Operating Systems Design and Implementation (\(\{\)OSDI\(\}\) 16), pp. 265–283 (2016)

    Google Scholar 

  3. Chang, A.X., et al.: Shapenet: An information-rich 3d model repository. arXiv preprint arXiv:1512.03012 (2015)

  4. Chen, Z., Tagliasacchi, A., Zhang, H.: BSP-Net: generating compact meshes via binary space partitioning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 45–54 (2020)

    Google Scholar 

  5. Chen, Z., Zhang, H.: Learning implicit fields for generative shape modeling. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5939–5948 (2019)

    Google Scholar 

  6. Choy, C.B., Xu, D., Gwak, J.Y., Chen, K., Savarese, S.: 3D-R2N2: a unified approach for single and multi-view 3D object reconstruction. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9912, pp. 628–644. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46484-8_38

    Chapter  Google Scholar 

  7. Eldar, Y., Lindenbaum, M., Porat, M., Zeevi, Y.Y.: The farthest point strategy for progressive image sampling. IEEE Trans. Image Process. 6(9), 1305–1315 (1997)

    Article  Google Scholar 

  8. Fan, H., Su, H., Guibas, L.J.: A point set generation network for 3D object reconstruction from a single image. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 605–613 (2017)

    Google Scholar 

  9. Gao, L., et al.: SDM-NET: deep generative network for structured deformable mesh. ACM Trans. Graph. (TOG) 38(6), 1–15 (2019)

    Google Scholar 

  10. Gkioxari, G., Malik, J., Johnson, J.: Mesh R-CNN. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 9785–9795 (2019)

    Google Scholar 

  11. Groueix, T., Fisher, M., Kim, V.G., Russell, B.C., Aubry, M.: Atlasnet: Apapier-m\(\backslash \hat{}\) ach\(\backslash \)’e approach to learning 3d surfacegeneration. arXiv preprint arXiv:1802.05384 (2018)

  12. Halton, J.H.: Algorithm 247: radical-inverse quasi-random point sequence. Commun. ACM 7(12), 701–702 (1964)

    Article  Google Scholar 

  13. Joe, S., Kuo, F.Y.: Constructing Sobol sequences with better two-dimensional projections. SIAM J. Sci. Comput. 30(5), 2635–2654 (2008)

    Article  MathSciNet  Google Scholar 

  14. Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arxiv:1412.6980 (2014)

  15. Kuipers, L., Niederreiter, H.: Uniform Distribution of Sequences. Courier Corporation, North Chelmsford (2012)

    MATH  Google Scholar 

  16. Li, K., Pham, T., Zhan, H., Reid, I.: Efficient dense point cloud object reconstruction using deformation vector fields. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 497–513 (2018)

    Google Scholar 

  17. Lin, C.H., Kong, C., Lucey, S.: Learning efficient point cloud generation for dense 3D object reconstruction. In: Thirty-Second AAAI Conference on Artificial Intelligence (2018)

    Google Scholar 

  18. Liu, S., Zhang, Y., Peng, S., Shi, B., Pollefeys, M., Cui, Z.: Dist: Rendering deep implicit signed distance function with differentiable sphere tracing. arXiv preprint arXiv:1911.13225 (2019)

  19. Liu, S., Li, T., Chen, W., Li, H.: Soft rasterizer: a differentiable renderer for image-based 3D reasoning. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 7708–7717 (2019)

    Google Scholar 

  20. Mescheder, L., Oechsle, M., Niemeyer, M., Nowozin, S., Geiger, A.: Occupancy networks: Learning 3D reconstruction in function space. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4460–4470 (2019)

    Google Scholar 

  21. Mitchell, E., Engin, S., Isler, V., Lee, D.D.: Higher-order function networks for learning composable 3d object representations. arXiv preprint arXiv:1907.10388 (2019)

  22. Niederreiter, H.: Low-discrepancy and low-dispersion sequences. J. Number Theor. 30(1), 51–70 (1988)

    Article  MathSciNet  Google Scholar 

  23. Pan, J., Han, X., Chen, W., Tang, J., Jia, K.: Deep mesh reconstruction from single RGB images via topology modification networks. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 9964–9973 (2019)

    Google Scholar 

  24. Pilleboue, A., Singh, G., Coeurjolly, D., Kazhdan, M., Ostromoukhov, V.: Variance analysis for Monte Carlo integration. ACM Trans. Graph. (Proc. SIGGRAPH) 34(4), 124:1–124:14 (2015)

    Article  Google Scholar 

  25. Singh, G., et al.: Analysis of sample correlations for Monte Carlo rendering. Comput. Graph. Forum 38(2), 473–491 (2019)

    Article  Google Scholar 

  26. Sun, X., et al.: Pix3D: Dataset and methods for single-image 3D shape modeling. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2974–2983 (2018)

    Google Scholar 

  27. Villani, C.: Optimal Transport: Old and New, vol. 338. Springer, Heidelberg (2008). https://doi.org/10.1007/978-3-540-71050-9

    Book  MATH  Google Scholar 

  28. Wang, N., Zhang, Y., Li, Z., Fu, Y., Liu, W., Jiang, Y.G.: Pixel2mesh: generating 3D mesh models from single RGB images. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 52–67 (2018)

    Google Scholar 

  29. Wang, W., Ceylan, D., Mech, R., Neumann, U.: 3DN: 3D deformation network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1038–1046 (2019)

    Google Scholar 

  30. Xie, H., Yao, H., Sun, X., Zhou, S., Zhang, S.: Pix2Vox: context-aware 3D reconstruction from single and multi-view images. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2690–2698 (2019)

    Google Scholar 

  31. Xu, H., Barbič, J.: Signed distance fields for polygon soup meshes. Proc. Graph. Interface 2014, 35–41 (2014)

    Google Scholar 

  32. Xu, Q., Wang, W., Ceylan, D., Mech, R., Neumann, U.: Disn: Deep implicit surface network for high-quality single-view 3d reconstruction. arXiv preprint arXiv:1905.10711 (2019)

  33. Yao, Y., Schertler, N., Rosales, E., Rhodin, H., Sigal, L., Sheffer, A.: Front2back: single view 3d shape reconstruction via front to back prediction. arXiv preprint arXiv:1912.10589 (2019)

Download references

Acknowledgement

We would like to thank the anonymous reviewers for their helpful feedback and suggestions. We would like to thank Zilei Huang for his help in accelerating the data processing and debugging.

Author information

Authors and Affiliations

  1. Netease Fuxi AI Lab, Hangzhou, China

    Yifan Xu & Yi Yuan

  2. Saarland Informatics Campus, Saarbruecken, Germany

    Tianqi Fan

  3. MPI for Informatics, Saarbruecken, Germany

    Tianqi Fan & Gurprit Singh

Authors
  1. Yifan Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Tianqi Fan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yi Yuan
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Gurprit Singh
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yi Yuan .

Editor information

Editors and Affiliations

  1. University of Oxford, Oxford, UK

    Andrea Vedaldi

  2. Graz University of Technology, Graz, Austria

    Horst Bischof

  3. University of Freiburg, Freiburg im Breisgau, Germany

    Thomas Brox

  4. University of North Carolina at Chapel Hill, Chapel Hill, NC, USA

    Jan-Michael Frahm

1 Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (pdf 2991 KB)

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Xu, Y., Fan, T., Yuan, Y., Singh, G. (2020). Ladybird: Quasi-Monte Carlo Sampling for Deep Implicit Field Based 3D Reconstruction with Symmetry. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, JM. (eds) Computer Vision – ECCV 2020. ECCV 2020. Lecture Notes in Computer Science(), vol 12346. Springer, Cham. https://doi.org/10.1007/978-3-030-58452-8_15

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-58452-8_15

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-58451-1

  • Online ISBN: 978-3-030-58452-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation