Inseung Hwang
Min H. Kim
Skip Abstract Section
Skip Supplemental Material SectionAbstract
Ellipsometry techniques allow to measure polarization information of materials, requiring precise rotations of optical components with different configurations of lights and sensors. This results in cumbersome capture devices, carefully calibrated in lab conditions, and in very long acquisition times, usually in the order of a few days per object. Recent techniques allow to capture polarimetric spatially-varying reflectance information, but limited to a single view, or to cover all view directions, but limited to spherical objects made of a single homogeneous material. We present sparse ellipsometry, a portable polarimetric acquisition method that captures both polarimetric SVBRDF and 3D shape simultaneously. Our handheld device consists of off-the-shelf, fixed optical components. Instead of days, the total acquisition time varies between twenty and thirty minutes per object. We develop a complete polarimetric SVBRDF model that includes diffuse and specular components, as well as single scattering, and devise a novel polarimetric inverse rendering algorithm with data augmentation of specular reflection samples via generative modeling. Our results show a strong agreement with a recent ground-truth dataset of captured polarimetric BRDFs of real-world objects.
Supplemental Material
- Neil Alldrin, Todd Zickler, and David Kriegman. 2008. Photometric stereo with non-parametric and spatially-varying reflectance. In Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR) 2008. 1--8.Google Scholar
Cross Ref
- Gary Atkinson and E.R. Hancock. 2006. Recovery of surface orientation from diffuse polarization. IEEE Transactions on Image Processing 15, 6 (jun 2006), 1653--1664. Google ScholarDigital Library
- RMA Azzam. 1978. Photopolarimetric measurement of the Mueller matrix by Fourier analysis of a single detected signal. Optics Letters 2, 6 (1978), 148--150.Google ScholarCross Ref
- RMA Azzam. 2016. Stokes-vector and Mueller-matrix polarimetry. JOSA A 33, 7 (2016), 1396--1408.Google ScholarCross Ref
- Yunhao Ba, Alex Gilbert, Franklin Wang, Jinfa Yang, Rui Chen, Yiqin Wang, Lei Yan, Boxin Shi, and Achuta Kadambi. 2020. Deep Shape from Polarization. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) 12369 LNCS (2020), 554--571.Google ScholarDigital Library
- Seung-Hwan Baek and Felix Heide. 2021. Polarimetric Spatio-Temporal Light Transport Probing. ACM Transactions on Graphics (Proc. SIGGRAPH Asia) 40, 6 (2021).Google Scholar
- Seung-Hwan Baek, Daniel S. Jeon, Xin Tong, and Min H. Kim. 2018. Simultaneous Acquisition of Polarimetric SVBRDF and Normals. ACM Transactions on Graphics (Proc. SIGGRAPH Asia 2018) 36, 6 (2018), 268:1--15. Google ScholarDigital Library
- Seung-Hwan Baek, Tizian Zeltner, Hyun Jin Ku, Inseung Hwang, Xin Tong, Wenzel Jakob, and Min H. Kim. 2020. Image-Based Acquisition and Modeling of Polarimetric Reflectance. ACM Transactions on Graphics (Proc. SIGGRAPH 2020) 39, 4 (2020).Google Scholar
- Mark Boss, Raphael Braun, Varun Jampani, Jonathan T Barron, Ce Liu, and Hendrik Lensch. 2021. Nerd: Neural reflectance decomposition from image collections. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 12684--12694.Google ScholarCross Ref
- Guojun Chen, Yue Dong, Pieter Peers, Jiawan Zhang, and Xin Tong. 2014. Reflectance scanning: estimating shading frame and BRDF with generalized linear light sources. ACM Transactions on Graphics (TOG) 33, 4 (2014), 117:1--11.Google ScholarDigital Library
- Edward Collett. 2005. Field guide to polarization. Spie Bellingham, WA.Google Scholar
- Robert L. Cook and Kenneth E. Torrance. 1982. A Reflectance Model for Computer Graphics. ACM Transactions on Graphics (TOG) 1, 1 (1982), 7--24. Google ScholarDigital Library
- Zhaopeng Cui, Jinwei Gu, Boxin Shi, Ping Tan, and Jan Kautz. 2017. Polarimetric Multi-View Stereo. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 1558--1567.Google ScholarCross Ref
- Zhaopeng Cui, Viktor Larsson, and Marc Pollefeys. 2019. Polarimetric relative pose estimation. Proceedings of the IEEE International Conference on Computer Vision 2019-October (2019), 2671--2680. Google ScholarCross Ref
- Paul E. Debevec and Jitendra Malik. 1997. Recovering High Dynamic Range Radiance Maps from Photographs. In Proc. ACM SIGGRAPH '97. 369--378.Google Scholar
- Fernando del Molino and Adolfo Muñoz. 2019. Polarization mapping. Computers & Graphics 83 (2019), 42 -- 50. Google ScholarDigital Library
- Valentin Deschaintre, Yiming Lin, and Abhijeet Ghosh. 2021. Deep polarization imaging for 3D shape and SVBRDF acquisition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).Google ScholarCross Ref
- Yuqi Ding, Yu Ji, Mingyuan Zhou, Sing Bing Kang, and Jinwei Ye. 2021. Polarimetric Helmholtz Stereopsis. In 2021 IEEE/CVF International Conference on Computer Vision (ICCV). 5017--5026. Google ScholarCross Ref
- Kenneth K Ellis. 1996. Polarimetric bidirectional reflectance distribution function of glossy coatings. JOSA A 13, 8 (1996), 1758--1762.Google ScholarCross Ref
- Yoshiki Fukao, Ryo Kawahara, Shohei Nobuhara, and Ko Nishino. 2021. Polarimetric Normal Stereo. In 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 682--690. Google ScholarCross Ref
- Abhijeet Ghosh, Tongbo Chen, Pieter Peers, Cyrus A. Wilson, and Paul Debevec. 2010. Circularly polarized spherical illumination reflectometry. ACM Transactions on Graphics 29, 6 (2010). Google ScholarDigital Library
- Abhijeet Ghosh, Graham Fyffe, Borom Tunwattanapong, Jay Busch, Xueming Yu, and Paul Debevec. 2011. Multiview face capture using polarized spherical gradient illumination. Proceedings of the 2011 SIGGRAPH Asia Conference on - SA '11 30, 6 (2011), 1. Google ScholarDigital Library
- Abhijeet Ghosh, Tim Hawkins, Pieter Peers, Sune Frederiksen, and Paul Debevec. 2008. Practical Modeling and Acquisition of Layered Facial Reflectance. ACM Trans. Graph. 27, 5, Article 139 (dec 2008), 10 pages. Google ScholarDigital Library
- Paulo Gotardo, Jérémy Riviere, Derek Bradley, Abhijeet Ghosh, and Thabo Beeler. 2018. Practical Dynamic Facial Appearance Modeling and Acquisition. ACM Trans. Graph. 37, 6, Article 232 (dec 2018), 13 pages. Google ScholarDigital Library
- Pat Hanrahan and Wolfgang Krueger. 1993. Reflection from layered surfaces due to subsurface scattering. In Proceedings of the 20th annual conference on Computer graphics and interactive techniques. 165--174.Google ScholarDigital Library
- Eric Heitz. 2014. Understanding the Masking-Shadowing Function in Microfacet-Based BRDFs. J. Comput. Graph. Techn. 3, 2 (2014), 48--107.Google Scholar
- Louis G Henyey and Jesse Leonard Greenstein. 1941. Diffuse radiation in the galaxy. The Astrophysical Journal 93 (1941), 70--83.Google ScholarCross Ref
- Cong Phuoc Huynh, Antonio Robles-Kelly, and Edwin R. Hancock. 2013. Shape and refractive index from single-view spectro-polarimetric images. Vol. 101. 64--94 pages. Google ScholarDigital Library
- MW Hyde IV, JD Schmidt, and MJ Havrilla. 2009. A geometrical optics polarimetric bidirectional reflectance distribution function for dielectric and metallic surfaces. Optics express 17, 24 (2009), 22138--22153.Google Scholar
- Adrian Jarabo and Victor Arellano. 2017. Bidirectional Rendering of Vector Light Transport. Computer Graphics Forum (12 2017).Google Scholar
- Achuta Kadambi, Vage Taamazyan, Boxin Shi, and Ramesh Raskar. 2015. Polarized 3D: High-Quality Depth Sensing with Polarization Cues. In Proc. ICCV. IEEE Computer Society, 3370--3378.Google ScholarDigital Library
- Christos Kampouris, Stefanos Zafeiriou, and Abhijeet Ghosh. 2018. Diffuse-Specular Separation using Binary Spherical Gradient Illumination.. In EGSR (EI&I). 1--10.Google Scholar
- Michael Kazhdan and Hugues Hoppe. 2013. Screened poisson surface reconstruction. ACM Transactions on Graphics (TOG) 32, 3 (2013), 29.Google ScholarDigital Library
- Jason Lawrence, Aner Ben-Artzi, Christopher DeCoro, Wojciech Matusik, Hanspeter Pfister, Ravi Ramamoorthi, and Szymon Rusinkiewicz. 2006. Inverse shade trees for non-parametric material representation and editing. ACM Transactions on Graphics (TOG) 25, 3 (2006), 735--745.Google ScholarDigital Library
- Joo Ho Lee, Adrian Jarabo, Daniel S. Jeon, Diego Gutierrez, and Min H. Kim. 2018. Practical Multiple Scattering for Rough Surfaces. ACM Transactions on Graphics (TOG) 37, 6 (2018).Google ScholarDigital Library
- Chenyang Lei, Chenyang Qi, Jiaxin Xie, Na Fan, Vladlen Koltun, and Qifeng Chen. 2021. Shape from Polarization for Complex Scenes in the Wild. (2021). arXiv:2112.11377 http://arxiv.org/abs/2112.11377Google Scholar
- Hendrik P. A. Lensch, Jan Kautz, Michael Goesele, Wolfgang Heidrich, and Hans-Peter Seidel. 2003. Image-based Reconstruction of Spatial Appearance and Geometric Detail. ACM Transactions on Graphics (TOG) 22, 2 (2003), 234--257.Google ScholarDigital Library
- Fei Liu, Lei Cao, Xiaopeng Shao, Pingli Han, and Xiangli Bin. 2015. Polarimetric dehazing utilizing spatial frequency segregation of images. Appl. Opt. 54, 27 (Sep 2015), 8116--8122. Google ScholarCross Ref
- Wan-Chun Ma, Tim Hawkins, Pieter Peers, Charles-Felix Chabert, Malte Weiss, and Paul Debevec. 2007. Rapid Acquisition of Specular and Diffuse Normal Maps from Polarized Spherical Gradient Illumination. In Proceedings of the 18th Eurographics Conference on Rendering Techniques (Grenoble, France) (EGSR'07). Eurographics Association, Goslar, DEU, 183--194.Google ScholarDigital Library
- Xiaohe Ma, Kaizhang Kang, Ruisheng Zhu, Hongzhi Wu, and Kun Zhou. 2021. Free-Form Scanning of Non-Planar Appearance with Neural Trace Photography. ACM Trans. Graph. 40, 4, Article 124 (jul 2021), 13 pages. Google ScholarDigital Library
- JR Maxwell, J Beard, S Weiner, Da Ladd, and S Ladd. 1973. Bidirectional Reflectance Model Validation and Utilization. Technical Report. Environmental Research Inst Of Michiganann Arbor Infrared And Optics Div.Google Scholar
- Ben Mildenhall, Pratul P. Srinivasan, Matthew Tancik, Jonathan T. Barron, Ravi Ramamoorthi, and Ren Ng. 2020. NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis. In ECCV.Google Scholar
- Daisuke Miyazaki, Robby T. Tan, Kenji Hara, and Katsushi Ikeuchi. 2003. Polarization-based inverse rendering from a single view. Proceedings of the IEEE International Conference on Computer Vision 2, Iccv (2003), 982--987. Google ScholarCross Ref
- Michal Mojzik, Tomas Skrivan, Alexander Wilkie, and Jaroslav Krivanek. 2016. Bi-Directional Polarised Light Transport. In Eurographics Symposium on Rendering.Google Scholar
- Giljoo Nam, Joo Ho Lee, Diego Gutierrez, and Min H. Kim. 2018. Practical SVBRDF acquisition of 3D objects with unstructured flash photography. ACM Transactions on Graphics 37, 6 (2018), 1--12. Google ScholarDigital Library
- Giljoo Nam, Joo Ho Lee, Hongzhi Wu, Diego Gutierrez, and Min H. Kim. 2016. Simultaneous Acquisition of Microscale Reflectance and Normals. ACM Transactions on Graphics (Proc. SIGGRAPH Asia 2016) 35, 6 (2016). Google ScholarDigital Library
- Geoffrey Oxholm and Ko Nishino. 2014. Multiview shape and reflectance from natural illumination. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (2014), 2163--2170. Google ScholarDigital Library
- Richard G Priest and Thomas A Gerner. 2000. Polarimetric BRDF in the microfacet model: Theory and measurements. Technical Report. NAVAL RESEARCH LAB WASHINGTON DC.Google Scholar
- Jérémy Riviere, Paulo Gotardo, Derek Bradley, Abhijeet Ghosh, and Thabo Beeler. 2020. Single-Shot High-Quality Facial Geometry and Skin Appearance Capture. 39, 4 (2020). Google ScholarDigital Library
- Jérémy Riviere, Ilya Reshetouski, Luka Filipi, and Abhijeet Ghosh. 2017. Polarization imaging reflectometry in the wild. ACM Transactions on Graphics 36, 6 (2017). Google ScholarDigital Library
- Johannes L Schönberger, Enliang Zheng, Jan-Michael Frahm, and Marc Pollefeys. 2016. Pixelwise view selection for unstructured multi-view stereo. In European Conference on Computer Vision. Springer, 501--518.Google ScholarCross Ref
- Pratul P. Srinivasan, Boyang Deng, Xiuming Zhang, Matthew Tancik, Ben Mildenhall, and Jonathan T. Barron. 2021. NeRV: Neural Reflectance and Visibility Fields for Relighting and View Synthesis. In CVPR.Google Scholar
- Yinlong Sun. 2007. Statistical ray method for deriving reflection models of rough surfaces. JOSA A 24, 3 (2007), 724--744.Google ScholarCross Ref
- Kenneth E Torrance and Ephraim M Sparrow. 1967. Theory for off-specular reflection from roughened surfaces. JOSA 57, 9 (1967), 1105--1112.Google ScholarCross Ref
- Silvia Tozza, William A.P. Smith, Dizhong Zhu, Ravi Ramamoorthi, and Edwin R. Hancock. 2017. Linear Differential Constraints for Photo-Polarimetric Height Estimation. Proceedings of the IEEE International Conference on Computer Vision 2017-October (2017), 2298--2306. Google ScholarCross Ref
- Borom Tunwattanapong, Graham Fyffe, Paul Graham, Jay Busch, Xueming Yu, Abhijeet Ghosh, and Paul Debevec. 2013. Acquiring Reflectance and Shape from Continuous Spherical Harmonic Illumination. ACM Trans. Graph. 32, 4, Article 109 (jul 2013), 12 pages. Google ScholarDigital Library
- Bruce Walter, Stephen R Marschner, Hongsong Li, and Kenneth E Torrance. 2007. Microfacet models for refraction through rough surfaces. In Eurographics conference on Rendering Techniques. Eurographics Association, 195--206.Google ScholarDigital Library
- Alexander Wilkie and Andrea Weidlich. 2010. A standardised polarisation visualisation for images. In Proceedings of the 26th Spring Conference on Computer Graphics. 43--50.Google ScholarDigital Library
- Alexander Wilkie and Andrea Weidlich. 2012. Polarised light in computer graphics. In SIGGRAPH Asia 2012 Courses. 1--87.Google Scholar
- Hongzhi Wu, Zhaotian Wang, and Kun Zhou. 2016. Simultaneous Localization and Appearance Estimation with a Consumer RGB-D Camera. IEEE Transactions on Visualization and Computer Graphics 22, 8 (2016), 2012--2023. Google ScholarDigital Library
- Hongzhi Wu and Kun Zhou. 2015. Appfusion: Interactive appearance acquisition using a kinect sensor. In Computer Graphics Forum, Vol. 34. Wiley Online Library, 289--298.Google Scholar
- Rui Xia, Yue Dong, Pieter Peers, and Xin Tong. 2016. Recovering shape and spatially-varying surface reflectance under unknown illumination. ACM Transactions on Graphics 35, 6 (2016). Google ScholarDigital Library
- Fanchao Yang, Xingjia Tang, Bingliang Hu, Ruyi Wei, Liang Kong, and Yong Li. 2016. A Method of Removing Reflected Highlight on Images Based on Polarimetric Imaging. Journal of Sensors 2016 (2016).Google Scholar
- Z. Y. Zhang. 2000. A Flexible New Technique for Camera Calibration. IEEE Trans. Pattern Analysis and Machine Intelligence 22, 11 (Nov. 2000), 1330--1334.Google ScholarDigital Library
- Jinyu Zhao, Yusuke Monno, and Masatoshi Okutomi. 2020. Polarimetric Multi-view Inverse Rendering. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) 12369 LNCS (2020), 85--102. Google ScholarDigital Library
- Zhiming Zhou, Guojun Chen, Yue Dong, David Wipf, Yong Yu, John Snyder, and Xin Tong. 2016. Sparse-as-possible SVBRDF acquisition. ACM Transactions on Graphics (TOG) 35, 6 (2016), 189.Google ScholarDigital Library
- Zhenglong Zhou, Zhe Wu, and Ping Tan. 2013. Multi-view photometric stereo with spatially varying isotropic materials. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (2013), 1482--1489. Google ScholarDigital Library
- Dizhong Zhu and William AP Smith. 2019. Depth from a polarisation + RGB stereo pair. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).Google ScholarCross Ref
Index Terms
- Sparse ellipsometry: portable acquisition of polarimetric SVBRDF and shape with unstructured flash photography
Recommendations
Rendering specular microgeometry with wave optics
Simulation of light reflection from specular surfaces is a core problem of computer graphics. Existing solutions either make the approximation of providing only a large-area average solution in terms of a fixed BRDF (ignoring spatial detail), or are ...
The Role of Subsurface Scattering in Glossiness Perception
This study investigates the potential impact of subsurface light transport on gloss perception for the purposes of broadening our understanding of visual appearance in computer graphics applications. Gloss is an important attribute for characterizing ...
Interactive editing and modeling of bidirectional texture functions
While measured Bidirectional Texture Functions (BTF) enable impressive realism in material appearance, they offer little control, which limits their use for content creation. In this work, we interactively manipulate BTFs and create new BTFs from flat ...
Comments