Skip to main content
SpringerLink
Log in

DEMC: A Deep Dual-Encoder Network for Denoising Monte Carlo Rendering

  • Regular Paper
  • Published:
Journal of Computer Science and Technology Aims and scope Submit manuscript

Abstract

In this paper, we present DEMC, a deep dual-encoder network to remove Monte Carlo noise efficiently while preserving details. Denoising Monte Carlo rendering is different from natural image denoising since inexpensive by-products (feature buffers) can be extracted in the rendering stage. Most of them are noise-free and can provide sufficient details for image reconstruction. However, these feature buffers also contain redundant information. Hence, the main challenge of this topic is how to extract useful information and reconstruct clean images. To address this problem, we propose a novel network structure, dual-encoder network with a feature fusion sub-network, to fuse feature buffers firstly, then encode the fused feature buffers and a noisy image simultaneously, and finally reconstruct a clean image by a decoder network. Compared with the state-of-the-art methods, our model is more robust on a wide range of scenes, and is able to generate satisfactory results in a significantly faster way.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Kajiya J T. The rendering equation. In Proc. the 13th Annual Conference on Computer Graphics and Interactive Techniques, August 1986, pp.143-150.

  2. Rousselle F, Manzi M, Zwicker M. Robust denoising using feature and color information. Computer Graphics Forum, 2013, 32(7): 121-130.

    Article  Google Scholar 

  3. Kalantari N K, Bako S, Sen P. A machine learning approach for filtering Monte Carlo noise. ACM Trans. Graph., 2015, 34(4): Article No. 122.

  4. Bitterli B, Rousselle F, Moon B, Guitián J A, Adler D, Mitchell K, Jarosz W, Novák J. Nonlinearly weighted first-order regression for denoising Monte Carlo renderings. Computer Graphics Forum, 2016, 35(4): 107-117.

    Article  Google Scholar 

  5. Bako S, Vogels T, McWilliams B, Meyer M, Novák J, Harvill A, Sen P, Derose T, Rousselle F. Kernel-predicting convolutional networks for denoising Monte Carlo renderings. ACM Trans. Graph., 2017, 36(4): Article No. 97.

  6. Moon B, Carr N, Yoon S E. Adaptive rendering based on weighted local regression. ACM Transactions on Graphics, 2017, 33(5): Article No. 170.

  7. Cook R L, Porter T, Carpenter L. Distributed ray tracing. In Proc. the 11th Annual Conference on Computer Graphics and Interactive Techniques, July 1984, pp.137-145.

  8. Zwicker M, Jarosz W, Lehtinen J, Moon B, Ramamoorthi R, Rousselle F, Sen P, Soler C, Yoon S E. Recent advances in adaptive sampling and reconstruction for Monte Carlo rendering. Computer Graphics Forum, 2015, 34(2): 667-681.

    Article  Google Scholar 

  9. Ramamoorthi R, Mahajan D, Belhumeur P. A first-order analysis of lighting, shading, and shadows. ACM Transactions on Graphics, 2007, 26(1): Article No. 2.

  10. Jarosz W, Schönefeld V, Kobbelt L, Jensen H W. Theory, analysis and applications of 2D global illumination. ACM Transactions on Graphics, 2012, 31(5): Article No. 125.

  11. Bauszat P, Eisemann M, Magnor M. Guided image filtering for interactive high-quality global illumination. Computer Graphics Forum, 2011, 30(4): 1361-1368.

    Article  Google Scholar 

  12. Bauszat P, Eisemann M, Eisemann E, Magnor M. General and robust error estimation and reconstruction for Monte Carlo rendering. Computer Graphics Forum, 2015, 34(2): 597-608.

    Article  Google Scholar 

  13. Yang X, Wang D, Hu W, Zhao L, Piao X, Zhou D, Zhang Q, Yin B, Cai Q, Wei X. Fast reconstruction for Monte Carlo rendering using deep convolutional networks. IEEE Access, 2019, 7: 21177-21187.

    Article  Google Scholar 

  14. Chaitanya C R A, Kaplanyan A S, Schied C, Salvi M, Lefohn A, Nowrouzezahrai D, Aila T. Interactive reconstruction of Monte Carlo image sequences using a recurrent denoising autoencoder. ACM Transactions on Graphics, 2017, 36(4): Article No. 98.

  15. Zhang K, Zuo W, Chen Y, Meng D, Zhang L. Beyond a Gaussian denoiser: Residual learning of deep CNN for image denoising. IEEE Transactions on Image Processing, 2017, 26(7): 3142-3155.

    Article  MathSciNet  Google Scholar 

  16. Gharbi M, Chaurasia G, Paris S, Durand F. Deep joint demosaicking and denoising. ACM Transactions on Graphics, 2016, 35(6): Article No. 191.

  17. Mao X, Shen C, Yang Y B. Image restoration using very deep convolutional encoder-decoder networks with symmetric skip connections. In Proc. the 2016 Annual Conference on Neural Information Processing Systems, December 2016, pp.2802-2810.

  18. Hinton G E, Salakhutdinov R R. Reducing the dimensionality of data with neural networks. Science, 2016, 313(5786): 504-507.

    Article  MathSciNet  Google Scholar 

  19. Balduzzi D, McWilliams B, Butler-Yeoman T. Neural Taylor approximations: Convergence and exploration in rectifier networks. In Proc. the 34th International Conference on Machine Learning, August 2017, pp.351-360.

    Google Scholar 

  20. He K, Zhang X, Ren S, Sun J. Deep residual learning for image recognition. In Proc. the 2016 IEEE Conference on Computer Vision and Pattern Recognition, June 2016, pp.770-778.

  21. Wang Z, Bovik A C, Sheikh H R, Simoncelli E P. Image quality assessment: From error visibility to structural similarity. IEEE Transactions on Image Processing, 2004, 13(4): 600-612.

    Article  Google Scholar 

  22. Pharr M, Jakob W, Humphreys G. Physically Based Rendering: From Theory to Implementation (3rd edition). Morgan Kaufmann, 2016.

  23. Eilertsen G, Kronander J, Denes G, Mantiuk R K, Unger J. HDR image reconstruction from a single exposure using deep CNNs. ACM Transactions on Graphics, 2107, 36(6): Article No. 178.

  24. Rousselle F, Knaus C, Zwicker. Adaptive sampling and reconstruction using greedy error minimization. ACM Transactions on Graphics, 2011, 30(6): Article No. 159.

  25. Glorot X, Bengio Y. Understanding the difficulty of training deep feedforward neural networks. In Proc. the 13th International Conference on Artificial Intelligence and Statistics, May 2010, pp.249-256

  26. Abadi M, Barham P, Chen J et al. Tensorflow: A system for large-scale machine learning. In Proc. the 12th USENIX Symposium on Operating Systems Design and Implementation, November 2016, pp.265-283.

  27. Kingma D P, Ba J. Adam: A method for stochastic optimization. arXiv:1412.6980, 2014. https://arxiv.org/abs/1412.6980, May 2019.

  28. Boughida M, Boubekeur T. Bayesian collaborative denoising for Monte Carlo rendering. Computer Graphics Forum, 2017, 36(4): 137-153.

    Article  Google Scholar 

  29. Lang M, Wang O, Aydin T, Smolic A, Gross M. Practical temporal consistency for image-based graphics applications. ACM Transactions on Graphics, 2012, 31(4): Article No. 34.

  30. Bonneel N, Tompkin J, Sunkavalli K, Sun D, Paris S, Pfister H. Blind video temporal consistency. ACM Transactions on Graphics, 2015, 34(6): Article No. 196.

  31. Lai W S, Huang J B, Wang O, Shechtman E, Yumer E, Yang M H. Learning blind video temporal consistency. In Proc. the 15th European Conference on Computer Vision, September 2018, pp.179-185.

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science and Technology, Dalian University of Technology, Dalian, 116024, China

    Xin Yang, Li-Jing Zhao, Bao-Cai Yin, Qiang Zhang & Xiao-Peng Wei

  2. Department of Computer Science, The University of Hong Kong, Pokfulam, Hong Kong, China

    Dawei Wang

  3. Department of Computer Science and Engineering, The Chinese University of Hong Kong, Shatin, Hong Kong, China

    Wenbo Hu

  4. School of Creative Media, City University of Hong Kong, Kowloon, Hong Kong, China

    Hongbo Fu

Authors
  1. Xin Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Dawei Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Wenbo Hu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Li-Jing Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Bao-Cai Yin
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Qiang Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Xiao-Peng Wei
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Hongbo Fu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xiao-Peng Wei.

Electronic supplementary material

ESM 1

(PDF 1391 kb)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yang, X., Wang, D., Hu, W. et al. DEMC: A Deep Dual-Encoder Network for Denoising Monte Carlo Rendering. J. Comput. Sci. Technol. 34, 1123–1135 (2019). https://doi.org/10.1007/s11390-019-1964-2

Download citation

  • Received:

  • Revised:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11390-019-1964-2

Keywords

Search

Navigation