Abstract
A method that integrates shape from shading and stereo is reported for Lambertian objects. A rectification is proposed to convert any lighting direction from oblique to orthographic. A sparse stereo method is reported that directly uses depth information and has no foreshortening problem. The method completely solves three difficult problems in stereo, namely, recovering depth at occlusion; matching at places with similar shading and matching at smooth silhouettes. The method has been tested on both synthetic and real images. It shows superior performance compared with two recent stereo algorithms. It is also a method based on the physics of image formation.
Similar content being viewed by others
References
Aloimonos, J., & Shulman, D. (1989). Integration of visual modules: an extension of the Marr paradigm. New York: Academic Press.
Carr, J. C., Fright, W. R., & Beatson, R. K. (1997). Surface interpolation with radial basis functions for medical imaging. IEEE Transactions on Medical Imaging, 16(1), 96–107.
Chow, C. K., & Yuen, S. Y. (2007a). Equivalence of oblique and frontal illumination in perspective shape from shading. In Proceedings international conference on machine vision (pp. 7–11).
Chow, C.K., Yuen, S. Y. (2007b). Lighting direction estimation in perspective shape from shading by genetic algorithm. In Proceedings 4th Canadian conference on computer and robot vision (pp. 289–296).
Courteille, F., Crouzil, A., Durou, J.-D., & Gurdjod, P. (2004). Towards shape from shading under realistic photographic conditions. In Proceedings of the of IEEE international conference on pattern recognition (Vol. 2 pp. 277–280).
Cryer, J. E., Tsai, P. S., & Shah, M. (1995). Integration of shape from shading and stereo. Pattern Recognition, 28(7), 1033–1043.
Devernay, F., & Faugeras, O. (1994). Computing differential properties of 3D shapes from stereopsis without 3D models. In Proceedings of computer vision and pattern recognition (pp. 208–213).
Durou, J.-D., Falcone, M., & Sagona, M. (2004). A survey of numerical methods for shape from shading (IRIT Technical Report 2004-2-R).
Faugeras, O., & Keriven, R. (1998). Variational principles, surface evolution, PDE’s, level set methods, and the stereo problem. IEEE Transactions on Image Processing, 7(3), 336–344.
Forsyth, D. A., & Ponce, J. (2003). Computer vision, a modern approach. New York: Prentice Hall.
Frankot, R. T., & Chellappa, R. (1988). A method for enforcing integrability in shape from shading algorithm. IEEE Transactions on Pattern Analysis and Machine Intelligence, 10(4), 439–451.
Hartt, K., & Carlotto, M. (1989). A Method for shape-from-shading using multiple images acquired under different viewing and lighting conditions. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 53–60).
Hougen, D. R., & Ahuja, N. (1993)l. Estimation of the light source direction and its use in integrated shape recovery from stereo and shading. In Proceedings of international conference on computer vision (pp. 148–155).
Ikeuchi, K. (1983). Constructing a depth map from images (AI Memo No. 744). MIT Artificial Intelligence Lab., Cambridge, MA.
Jin, H., Soatto, S., & Yezzi, A. J. (2005). Multi-view stereo reconstruction of dense shape and complex appearance. International Journal of Computer Vision, 63(3), 175–189.
Jin, H., Cremers, D., Yezzi, A. J., & Soatto, S. (2004). Shedding light on stereoscopic segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition (Vol. 1, pp. 36–42).
Jin, H., Cremers, D., Wang, D., Prados, E., Yezzi, A., & Soatto, S. (2008). 3-d reconstruction of shaded objects from multiple images under unknown illumination. International Journal of Computer Vision, 76, 245–256.
Kimmel, R., & Sethian, J. A. (1998). Computing geodesic paths on manifolds. Proceedings of the National Academy of Sciences of the United States of America, 95, 8431–8435.
Kimmel, R., & Sethian, J. A. (2001). Optimal algorithm for shape from shading and path planning. Journal of Mathematical Imaging and Vision, 14(3), 237–244.
Kobayashi, F., Fukuda, Y., Shimojima, K., & Takusagawa, T. (1998). Shape reconstruction system integrating stereo vision and shape-from-shading with evolutionary programming. Journal of Electronic Imaging, 7(1), 8–15.
Kolmogorov, V., & Zabih, R. (2001). Computing visual correspondence with occlusions using graph cuts. In Proceedings of international conference on computer vision (Vol. 2, pp. 508–515).
Kolmogorov, V., & Zabih, R. (2002). Multi-camera scene reconstruction via graph cuts. Lecture Notes in Computer Science, 2352, 82–96.
Leclerc, Y. G., & Bobick, A. F. (1991). The direct computation of height from shading. In Proceedings of computer vision and pattern recognition (pp. 552–558).
Mostafa, M. G.-H., Yamany, S. M., & Farag, A. A. (1999). Integrating stereo and shape from shading. In Proceedings of the international conference on image processing (Vol. 3, pp. 130–134).
Prados, E., & Faugeras, O. (2005a). Shape from shading: a well-posed problem? In Proceedings of computer vision and pattern recognition (Vol. 2 pp. 870–877).
Prados, E., & Faugeras, O. (2005b). A generic and provably convergent shape-from-shading method for orthographic and pinhole cameras. International Journal on Computer Vision, 65(1–2), 97–125.
Prados, E., & Soatto, S. (2005). Fast marching method for generic shape from shading. Lecture Notes in Computer Science, 3752, 320–331.
Prados, E., Camilli, F., & Faugeras, O. (2006). A unifying and rigorous shape from shading method adapted to realistic data and applications. Journal of Mathematical Imaging and Vision, 25, 307–328.
Samaras, D., Metaxas, D., Fua, P., & Leclerc, Y. G. (2000). Variable albedo surface reconstruction from stereo and shape from shading. In Proceedings of IEEE conference on computer vision and pattern recognition (Vol. 1, pp. 480–487).
Scharstein, D., & Szeliski, R. (2002). A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. International Journal of Computer Vision, 47(1–3), 7–42. See also http://vision.middlebury.edu/stereo/.
Sethian, J. A., & Vladimirsky, A. (2000). Fast methods for the Eikonal and related Hamilton-Jacobi equations on unstructured meshes. Proceedings of the National Academy of Sciences of the United States of America, 97, 5699–5703.
Sun, C. (2002). Fast stereo matching using rectangular subregioning and 3d maximum-surface techniques. International Journal of Computer Vision, 47, 99–117.
Sun, J., Zheng, N. N., & Shum, H. Y. (2003). Stereo matching using belief propagation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 25(7), 1–14.
Tankus, A., Sochen, N., & Yeshurun, Y. (2005). Shape-from-shading under perspective projection. International Journal on Computer Vision, 63(1), 21–43.
Tappen, M. F., Freeman, W. T., & Adelson, E. H. (2005). Recovering intrinsic images from a single image. IEEE Transactions on Pattern Analysis and Machine Intelligence, 27(9), 1459–1472.
Trucco, E., & Verri, A. (1998). Introductory techniques for 3-D computer vision. New York: Prentice Hall.
Verbeek, P. W., & Verwer, B. J. H. (1990). Shape from shape, the eikonal equation solved by gray-weighted distance transform. Pattern Recognition Letters, 11, 681–690.
Yuen, S. Y., Tsui, Y. Y., & Chow, C. K. (2007). A fast marching formulation of perspective shape from shading under frontal illumination. Pattern Recognition Letters, 28, 806–824.
Zhang, Z. (1999). Flexible camera calibration by viewing a plane from unknown orientation. In Proceedings of IEEE international conference on computer vision (pp. 666–673).
Zhang, R., Tsai, P. S., Cryer, J. E., & Shah, M. (1999). Shape from shading: a survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, 21(8), 690–706.
Zheng, Q., & Chellappa, R. (1991). Estimation of illuminant direction, albedo, and shape from shading. IEEE Transactions on Pattern Analysis and Machine Intelligence, 13(7), 680–702.
Author information
Authors and Affiliations
Corresponding author
Additional information
The work described in this article was fully supported by a grant from CityU (7002128).
Rights and permissions
About this article
Cite this article
Chow, C.K., Yuen, S.Y. Recovering Shape by Shading and Stereo Under Lambertian Shading Model. Int J Comput Vis 85, 58–100 (2009). https://doi.org/10.1007/s11263-009-0240-2
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11263-009-0240-2