Anpei Chen
Skip Abstract Section
Abstract
A surface light field represents the radiance of rays originating from any points on the surface in any directions. Traditional approaches require ultra-dense sampling to ensure the rendering quality. In this paper, we present a novel neural network based technique called deep surface light field or DSLF to use only moderate sampling for high fidelity rendering. DSLF automatically fills in the missing data by leveraging different sampling patterns across the vertices and at the same time eliminates redundancies due to the network's prediction capability. For real data, we address the image registration problem as well as conduct texture-aware remeshing for aligning texture edges with vertices to avoid blurring. Comprehensive experiments show that DSLF can further achieve high data compression ratio while facilitating real-time rendering on the GPU.
- Chris Buehler, Michael Bosse, Leonard Mcmillan, Steven Gortler, and Michael Cohen. 2001. Unstructured lumigraph rendering. In Conference on Computer Graphics and Interactive Techniques. 425--432. Google Scholar
Digital Library
- Wei-Chao Chen, Jean-Yves Bouguet, Michael H Chu, and Radek Grzeszczuk. 2002. Light field mapping: efficient representation and hardware rendering of surface light fields. ACM Transactions on Graphics (TOG) 21, 3 (2002), 447--456. Google ScholarDigital Library
- Paul Debevec, Christoph Bregler, Michael Cohen, and Leonard Mcmillan. 1998. Image-based modeling and rendering. 299.Google Scholar
- Martin A. Fischler and Robert C. Bolles. 1987. Random Sample Consensus: A Paradigm for Model Fitting with Applications to Image Analysis and Automated Cartography. Readings in Computer Vision (1987), 726--740. Google ScholarDigital Library
- Xiao Shan Gao, Xiao Rong Hou, Jianliang Tang, and Hang Fei Cheng. 2003. Complete solution classification for the perspective-three-point problem. IEEE Transactions on Pattern Analysis and Machine Intelligence 25, 8 (2003), 930--943. Google ScholarDigital Library
- M.W Gardner and S.R Dorling. 1998. Artificial neural networks (the multilayer perceptron)-a review of applications in the atmospheric sciences. Atmospheric Environment 32, 14-15 (1998), 2627--2636.Google ScholarCross Ref
- Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2015. Deep Residual Learning for Image Recognition. (2015), 770--778.Google Scholar
- Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition. 770--778.Google ScholarCross Ref
- Kurt Hornik, Maxwell Stinchcombe, and Halbert White. 1989. Multilayer feedforward networks are universal approximators. Neural networks 2, 5 (1989), 359--366. Google ScholarDigital Library
- Liang Hu, Pedro V. Sander, and Hugues Hoppe. 2010. Parallel View-Dependent Level-of-Detail Control. IEEE Transactions on Visualization and Computer Graphics 16, 5 (2010), 718--728. Google ScholarDigital Library
- Tomasz Jannson. 1980. Radiance transfer function. Journal of the Optical Society of America 70, 12 (1980), 1544--1549.Google ScholarCross Ref
- Diederik Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014).Google Scholar
- Solomon Kullback and Richard A Leibler. 1951. On information and sufficiency. The annals of mathematical statistics 22, 1 (1951), 79--86.Google Scholar
- Vincent Lepetit, Francesc Moreno-Noguer, and Pascal Fua. 2009. EPnP: An Accurate O(n) Solution to the PnP Problem. International Journal of Computer Vision 81, 2 (2009), 155--166. Google ScholarDigital Library
- Marc Levoy. 1996. Light field rendering. In Conference on Computer Graphics and Interactive Techniques. 31--42. Google ScholarDigital Library
- M. Lounsbery, T. Derose, and J. Warren. 1997. Multersolution surfaces of arbitrary topological type. (1997).Google Scholar
- Yi Ma, Stefano Soatto, Jana KoEck, and S. Shankar Sastry. 2004. An invitation to 3-D vision:. Springer,. 526 pages.Google Scholar
- Marcus Magnor and Bernd Girod. 1999. Adaptive Block-based Light Field Coding. In PROC. 3RD INTERNATIONAL WORKSHOP ON SYNTHETIC AND NATURAL HYBRID CODING AND THREE-DIMENSIONAL IMAGING IWSNHC3DI;99, SANTORINI, GREECE. 140--143.Google Scholar
- Leonard Mcmillan and Gary Bishop. 1995. Plenoptic modeling: an image-based rendering system. In Conference on Computer Graphics and Interactive Techniques. 39--46. Google ScholarDigital Library
- Ehsan Miandji, Joel Kronander, and Jonas Unger. 2013. Learning based compression of surface light fields for real-time rendering of global illumination scenes. In SIGGRAPH Asia 2013 Technical Briefs. ACM, 24. Google ScholarDigital Library
- Gavin Miller, Steven Rubin, and Dulce Ponceleon. 1998. Lazy decompression of surface light fields for precomputed global illumination. In Rendering Techniques '98, Proceedings of the Eurographics Workshop in Vienna, Austria, June 29-July. 281--292.Google ScholarCross Ref
- Jorge J More. 1977. The Levenberg-Marquardt algorithm: Implementation and theory. Lecture Notes in Mathematics 630 (1977), 105--116.Google ScholarCross Ref
- Vinod Nair and Geoffrey E. Hinton. 2010. Rectified linear units improve restricted boltzmann machines. In International Conference on International Conference on Machine Learning. 807--814. Google ScholarDigital Library
- Damien Porquet, Jean-Michel Dischler, and Djamchid Ghazanfarpour. 2005. Real-time high-quality view-dependent texture mapping using per-pixel visibility. In Proceedings of the 3rd international conference on Computer graphics and interactive techniques in Australasia and South East Asia. ACM, 213--220. Google ScholarDigital Library
- Alec Radford, Luke Metz, and Soumith Chintala. 2015. Unsupervised representation learning with deep convolutional generative adversarial networks. arXiv preprint arXiv:1511.06434 (2015).Google Scholar
- Peiran Ren, Yue Dong, Stephen Lin, Xin Tong, and Baining Guo. 2015. Image based relighting using neural networks. ACM Transactions on Graphics (TOG) 34, 4 (2015), 111. Google ScholarDigital Library
- Peiran Ren, Jiaping Wang, Minmin Gong, Stephen Lin, Xin Tong, and Baining Guo. 2013. Global illumination with radiance regression functions. ACM Transactions on Graphics (TOG) 32, 4 (2013), 130. Google ScholarDigital Library
- Johannes L. Schonberger, Enliang Zheng, Jan Michael Frahm, and Marc Pollefeys. 2016. Pixelwise View Selection for Unstructured Multi-View Stereo. In European Conference on Computer Vision. 501--518.Google Scholar
- Michael Van den Bergh, Xavier Boix, Gemma Roig, Benjamin de Capitani, and Luc Van Gool. 2012. SEEDS: Superpixels extracted via energy-driven sampling. In European conference on computer vision. Springer, 13--26. Google ScholarDigital Library
- Daniel N Wood, Daniel I Azuma, Ken Aldinger, Brian Curless, Tom Duchamp, David H Salesin, and Werner Stuetzle. 2000. Surface light fields for 3D photography. In Proceedings of the 27th annual conference on Computer graphics and interactive techniques. ACM Press/Addison-Wesley Publishing Co., 287--296. Google ScholarDigital Library
Index Terms
- Deep Surface Light Fields
Recommendations
Surface light fields for 3D photography
SIGGRAPH '00: Proceedings of the 27th annual conference on Computer graphics and interactive techniquesA surface light field is a function that assigns a color to each ray originating on a surface. Surface light fields are well suited to constructing virtual images of shiny objects under complex lighting conditions. This paper presents a framework for ...
Opacity light fields: interactive rendering of surface light fields with view-dependent opacity
I3D '03: Proceedings of the 2003 symposium on Interactive 3D graphicsWe present new hardware-accelerated techniques for rendering surface light fields with opacity hulls that allow for interactive visualization of objects that have complex reflectance properties and elaborate geometrical details. The opacity hull is a ...
Rendering for an interactive 360° light field display
SIGGRAPH '07: ACM SIGGRAPH 2007 papersWe describe a set of rendering techniques for an autostereoscopic light field display able to present interactive 3D graphics to multiple simultaneous viewers 360 degrees around the display. The display consists of a high-speed video projector, a ...
Comments