Skip to main content
SpringerLink
Log in

Photometric Stereo with Non-Lambertian Preprocessing and Hayakawa Lighting Estimation for Highly Detailed Shape Reconstruction

  • Conference paper
  • First Online:
Mathematical Methods for Objects Reconstruction (INdAM 2021)

Part of the book series: Springer INdAM Series ((SINDAMS,volume 54))

  • 179 Accesses

Abstract

In many realistic scenarios, the use of highly detailed photometric 3D reconstruction techniques is hindered by several challenges in given imagery. Especially, the light sources are often unknown and need to be estimated, and the light reflectance is often non-Lambertian. In addition, when approaching the problem to apply photometric techniques at real-world imagery, several parameters appear that need to be fixed in order to obtain high-quality reconstructions. In this chapter, we attempt to tackle these issues by combining photometric stereo with non-Lambertian preprocessing and Hayakawa lighting estimation. At hand of a dedicated study, we discuss the applicability of these techniques for their use in automated 3D geometry recovery for 3D printing.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 139.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
USD 179.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Ahmed, A.H., Farag, A.A.: A new formulation for shape from shading for non-Lambertian surfaces. In: Proceedings of the 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06), vol. 2, pp. 17–22. IEEE, Piscataway (2006)

    Google Scholar 

  2. Baglama, J., Reichel, L.: Augmented implicitly restarted Lanczos bidiagonalization methods. SIAM J. Sci. Comput. 27(1), 19–42 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  3. Baglama, J., Reichel, L.: An implicitly restarted block Lanczos bidiagonalization method using Leja shifts. BIT Numer. Math. 53, 285–310 (2012)

    MathSciNet  MATH  Google Scholar 

  4. Bähr, M., Breuß, M., Quéau, Y., Boroujerdi, A.S., Durou, J.D.: Fast and accurate surface normal integration on non-rectangular domains. Comput. Vis. Media 3(2), 107–129 (2017)

    Article  MATH  Google Scholar 

  5. Barsky, S., Petrou, M.: The 4-source photometric stereo technique for three-dimensional surfaces in the presence of highlights and shadows. IEEE Trans. Pattern Anal. Mach. Intell. 25(10), 1239–1252 (2003)

    Article  Google Scholar 

  6. Basri, R., Jacobs, D., Kemelmacher, I.: Photometric stereo with general, unknown lighting. Int. J. Comput. Vis. 72, 239–257 (2007)

    Article  Google Scholar 

  7. Belhumeur, P., Kriegman, D., Yuille, A.: The bas-relief abiguity. In: Proceedings of the 1997 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'97), pp. 1060–1066. IEEE, Piscataway (1997)

    Google Scholar 

  8. Björck, A.: Numerical Methods for Least Squares Problems. SIAM, Philadelphia (1996)

    Book  MATH  Google Scholar 

  9. Breuß, M., Yarahmadi, A.M.: Perspective shape from shading. In: Advances in Photometric 3D-Reconstruction. pp. 31–72 (2020)

    Google Scholar 

  10. Byrd, R.H., Gilbert, J.C., Nocedal, J.: A trust region method based on interior point techniques for nonlinear programming. Math. Program. 89(1), 149–185 (2000)

    Article  MathSciNet  MATH  Google Scholar 

  11. Chen, C.P., Chen, C.S.: The 4-source photometric stereo under general unknown lighting. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) Computer Vision – ECCV 2006, pp. 72–83. Springer, Berlin (2006)

    Chapter  Google Scholar 

  12. Chen, G., Han, K., Wong, K.Y: PS-FCN: A flexible learning framework for photometric stereo. In: Proceedings of the European Conference on Computer Vision (ECCV), 16pp (2018)

    Google Scholar 

  13. Chen, G., Han, K., Shi, B., Matsushita, Y., Wong, K.: Self-calibrating deep photometric stereo networks. In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8731–8739 (2019)

    Google Scholar 

  14. Chen, G., Waechter, M., Shi, B., Wong, K.Y.K., Matsushita, Y.: What is learned in deep uncalibrated photometric stereo? In: European Conference on Computer Vision (2020)

    Google Scholar 

  15. Concas, A., Dessì, R., Fenu, C., Rodriguez, G., Vanzi, M.: Identifying the lights position in photometric stereo under unknown lighting. In: 2021 21st International Conference on Computational Science and Its Applications (ICCSA), pp. 10–20, Cagliari, Italy, September 2021

    Google Scholar 

  16. Durou, J.D., Falcone, M., Sagona, M.: Numerical methods for shape-from-shading: a new survey with benchmarks. Comput. Vis. Image Underst. 109, 22–43 (2008)

    Article  Google Scholar 

  17. Fox, J.: Applied Regression Analysis and Generalized Linear Models. SAGE Publishing, Thousand Oaks (2008)

    Google Scholar 

  18. Golub, G.H., Van Loan, C.F.: Matrix Computations. Johns Hopkins University Press, Baltimore (1996)

    MATH  Google Scholar 

  19. Haefner, B., Ye, Z., Gao, M., Wu, T., Quéau, Y., Cremers, D.: Variational uncalibrated photometric stereo under general lighting. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 8539–8548 (2019)

    Google Scholar 

  20. Haralick, R.M., Sternberg, S.R., Zhuang, X.: Image analysis using mathematical morphology. IEEE Trans. Pattern Anal. Mach. Intell. PAMI-9(4), 532–550 (1987)

    Article  Google Scholar 

  21. Hayakawa, H.: Photometric stereo under a light source with arbitrary motion. J. Opt. Soc. Am. A 11(11), 3079–3089 (1994)

    Article  MathSciNet  Google Scholar 

  22. Hold-Geoffroy, Y., Zhang, J., Gotardo, P.F.U., Lalonde, J.F.: What is a good day for outdoor photometric stereo? In: International Conference on Computational Photography (2015)

    Google Scholar 

  23. Hold-Geoffroy, Y., Gotardo, P., Lalonde, J.F.: Single day outdoor photometric stereo. IEEE Trans. Pattern Anal. Mach. Intell. 43(6), 2062–2074 (2021). https://doi.org/10.1109/tpami.2019.2962693

    Article  Google Scholar 

  24. Horn, B.K.P. (ed.): Robot Vision. MIT Press, Cambridge, USA (1986)

    Google Scholar 

  25. Horn, B.: Obtaining shape from shading information. In: Shape from Shading, pp. 123–171 (1989)

    Google Scholar 

  26. Ikehata, S.: CNN-PS: CNN-based photometric stereo for general non-convex surfaces. In: Proceedings of the European Conference on Computer Vision (ECCV), 16pp (2018)

    Google Scholar 

  27. Ju, Y., Tozza, S., Breuß, M., Bruhn, A., Kleefeld, A.: Generalised perspective shape from shading with oren-nayar reflectance. In: Proceedings of the 24th British Machine Vision Conference, pp. 42.1–42.11. BMVA Press, Durham (2013)

    Google Scholar 

  28. Khanian, M., Boroujerdi, A.S., Breuß, M.: Photometric stereo for strong specular highlights. Comput. Vis. Media 4, 83–102 (2018)

    Article  Google Scholar 

  29. Kimmel, R., Siddiqi, K., Kimia, B.B., Bruckstein, A.M.: Shape from shading: level set propagation and viscosity solutions. Int. J. Comput. Vis. 16(2), 107–133 (1995)

    Article  Google Scholar 

  30. Kozera, R.: Existence and uniqueness in photometric stereo. Appl. Math. Comput. 44(1), 1–103 (1991)

    MathSciNet  MATH  Google Scholar 

  31. Lambert, J.H., Klett, M.J., Detlefsen, C.P.: Photometria Sive De Mensura Et Gradibus Luminis, Colorum Et Umbrae. Klett, Augustae Vindelicorum, Augustae Vindelicorum (1760)

    Google Scholar 

  32. Mecca, R., Falcone, M.: Uniqueness and approximation of a photometric shape-from-shading model. SIAM J. Imag. Sci. 6(1), 616–659 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  33. Mecca, R., Tankus, A., Wetzler, A., Bruckstein, A.M.: A direct differential approach to photometric stereo with perspective viewing. SIAM J. Imag. Sci. 7(2), 579–612 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  34. Oren, M., Nayar, S.K.: Generalization of the Lambertian model and implications for machine vision. Int. J. Comput. Vis. 14(3), 227–251 (1995)

    Article  Google Scholar 

  35. Phong, B.T.: Illumination for computer generated pictures. Commun. ACM 18, 311–317 (1975)

    Article  Google Scholar 

  36. Quéau, Y., Mecca, R., Durou, J.D., Descombes, X.: Photometric stereo with only two images: a theoretical study and numerical resolution. Image Vis. Comput. 57, 175–191 (2017)

    Article  Google Scholar 

  37. Quéau, Y., Durou, J.D., Aujol, J.F.: Normal integration: a survey. J. Math. Imaging Vision 60(4), 576–593 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  38. Radow, G., Hoeltgen, L., Quéau, Y., Breuß, M.: Optimisation of classic photometric stereo by non-convex variational minimisation. J. Math. Imaging Vision 61(1), 84–105 (2019)

    Article  MathSciNet  MATH  Google Scholar 

  39. Ragheb, H., Hancock, E.R.: Surface radiance correction for shape from shading. Pattern Recogn. 38(10), 1574–1595 (2005)

    Article  Google Scholar 

  40. Santo, H., Samejima, M., Sugano, Y., Shi, B., Matsushita, Y.: Deep photometric stereo network. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 501–509 (2017)

    Google Scholar 

  41. Shi, B., Mo, Z., Wu, Z., Duan, D., Yeung, S.K., Tan, P.: A benchmark dataset and evaluation for non-Lambertian and uncalibrated photometric stereo. IEEE Trans. Pattern Anal. Mach. Intell. 41(2), 271–284 (2019)

    Article  Google Scholar 

  42. Stocchino, G.: Mathematical Models and Numerical Algorithms for Photometric Stereo (Modelli Matematici e Algoritmi Numerici per la Photometric Stereo). Bachelor’s Thesis in Mathematics, University of Cagliari (2015). Available at http://bugs.unica.it/gppe/did/tesi/15stocchino.pdf

    Google Scholar 

  43. Taniai, T., Maehara, T.: Neural inverse rendering for general reflectance photometric stereo (2018)

    Google Scholar 

  44. Tankus, A., Kiryati, N.: Photometric stereo under perspective projection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 611–616 (2005). https://doi.org/10.1109/ICCV.2005.190

  45. Torrance, K.E., Sparrow, E.M.: Theory for off-specular reflection from roughened surfaces. J. Opt. Soc. Am. 57(9), 1105–1114 (1967). http://www.osapublishing.org/abstract.cfm?URI=josa-57-9-1105

    Article  Google Scholar 

  46. Tozza, S., Falcone, M.: A comparison of non-lambertian models for the shape-from-shading problem. In: Perspectives in Shape Analysis, pp. 15–42. Mathematics and Visualization, Springer (2016)

    Google Scholar 

  47. Tozza, S., Mecca, R., Duocastella, M., Bue, A.D.: Direct differential photometric stereo shape recovery of diffuse and specular surfaces. J. Math. Imaging Vision 56(1), 57–76 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  48. Vanzi, M., Mannu, C., Dessí, R., Tanda, G.: Photometric stereo for 3D mapping of carvings and relieves. Case studies on prehistorical art in Sardinia. In: Proceedings of Atti del XVII MAÇAO’s Intemational Rock ArtSeminar (2016)

    Google Scholar 

  49. Woodham, R.J.: Photometric stereo: a reflectance map technique for determining surface orientation from image intensity. In: Optics & Photonics (1979)

    Google Scholar 

  50. Woodham, R.J.: Photometric method for determining surface orientation from multiple images. Opt. Eng. 19(1) (1980)

    Google Scholar 

  51. Yi, J., Ni, H., Wen, Z., Liu, B., Tao, J.: CTC regularized model adaptation for improving LSTM RNN based multi-accent Mandarin speech recognition. In: 2016 10th International Symposium on Chinese Spoken Language Processing (ISCSLP), pp. 1–5 (2016)

    Google Scholar 

  52. Zhang, R., Tsai, P.S., Cryer, J., Shah, M.: Shape from shading: a survey. IEEE Trans. Pattern Anal. Mach. Intell. 21, 690–706 (1999)

    Article  MATH  Google Scholar 

Download references

Acknowledgements

The work of Georg Radow was supported by the Deutsche Forschungsgemeinschaft, grant number BR 2245/4-1.

The work of Giuseppe Rodriguez was partially supported by the Regione Autonoma della Sardegna research project “Algorithms and Models for Imaging Science [AMIS]” (RASSR57257, intervento finanziato con risorse FSC 2014-2020—Patto per lo Sviluppo della Regione Sardegna), and the INdAM-GNCS research project “Tecniche numeriche per l’analisi delle reti complesse e lo studio dei problemi inversi.”

The work of Ashkan Mansouri Yarahmadi and Michael Breuß was partially supported by the European Regional Development Fund, EFRE 85037495.

Author information

Authors and Affiliations

  1. BTU Cottbus-Senftenberg, Applied Mathematics Group, Institute for Mathematics, Cottbus, Germany

    Georg Radow, Ashkan Mansouri Yarahmadi & Michael Breuß

  2. University of Cagliari, Department of Mathematics and Computer Science, Cagliari, Italy

    Giuseppe Rodriguez

Authors
  1. Georg Radow
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Giuseppe Rodriguez
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ashkan Mansouri Yarahmadi
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Michael Breuß
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Giuseppe Rodriguez .

Editor information

Editors and Affiliations

  1. Istituto per le Applicazioni del Calcolo “M.Picone”, Consiglio Nazionale delle Ricerche, Rome, Italy

    Emiliano Cristiani

  2. Department of Mathematics, Sapienza University of Rome, Rome, Italy

    Maurizio Falcone †

  3. Department of Mathematics, University of Bologna, Bologna, Italy

    Silvia Tozza

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Radow, G., Rodriguez, G., Yarahmadi, A.M., Breuß, M. (2023). Photometric Stereo with Non-Lambertian Preprocessing and Hayakawa Lighting Estimation for Highly Detailed Shape Reconstruction. In: Cristiani, E., Falcone †, M., Tozza, S. (eds) Mathematical Methods for Objects Reconstruction. INdAM 2021. Springer INdAM Series, vol 54. Springer, Singapore. https://doi.org/10.1007/978-981-99-0776-2_2

Download citation

Publish with us

Policies and ethics

Search

Navigation