Skip to main content
SpringerLink
Log in

Object-centered surface reconstruction: Combining multi-image stereo and shading

  • Published:
International Journal of Computer Vision Aims and scope Submit manuscript

Abstract

Our goal is to reconstruct both the shape and reflectance properties of surfaces from multiple images. We argue that an object-centered representation is most appropriate for this purpose because it naturally accommodates multiple sources of data, multiple images (including motion sequences of a rigid object), and self-occlusions. We then present a specific object-centered reconstruction method and its implementation. The method begins with an initial estimate of surface shape provided, for example, by triangulating the result of conventional stereo. The surface shape and reflectance properties are then iteratively adjusted to minimize an objective function that combines information from multiple input images. The objective function is a weighted sum of stereo, shading, and smoothness components, where the weight varies over the surface. For example, the stereo component is weighted more strongly where the surface projects onto highly textured areas in the images, and less strongly otherwise. Thus, each component has its greatest influence where its accuracy is likely to be greatest. Experimental results on both synthetic and real images are presented.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Abbot, A.L. and Ahuja, N. 1990. Active surface reconstruction by integrating focus, vergence, stereo, and camera calibration. InInternational Conference on Computer Vision, pp. 489–492.

  • Aloimonos, J.Y. 1989. Unification and integration of visual modules: an extension of the Marr paradigm. In {btARPA Image Understanding Workshop}, pp. 507–551.

  • Asada, M., Kimura, M., Taniguchi, Y., and Shirai, Y. 1992. Dynamic integration of height maps into a 3D world representation from range image sequences.International Journal of Computer Vision, 9(1):31–54.

    Google Scholar 

  • Baltsavias, E.P. 1991.Multiphoto Geometrically Constrained Matching. Ph.D. thesis, Institute for Geodesy and Photogrammetry, ETH Zurich.

    Google Scholar 

  • Barnard, S. 1989. Stochastic stereo matching over scale.International Journal of Computer Vision, 3(1):17–32.

    Google Scholar 

  • Barrow, H.G. and Tenenbaum, J.M. 1978. Recovering intrinsic scene characteristics from images. InComputer Vision Systems, Academic Press, New York, New York, pp. 3–26.

    Google Scholar 

  • Blake, A., Zisserman, A., and Knowles, G. 1985. Surface descriptions from stereo and shading.Image Vision Computation, 3(4):183–191.

    Google Scholar 

  • Choe, Y. and Kashyap, R.L. 1991. 3-d shape from a shaded and textural surface image.IEEE Transactions on Pattern Analysis and Machine Intelligence, 13:907–919.

    Google Scholar 

  • Cohen, I., Cohen, L.D., and Ayache, N. 1991. Introducing new deformable surfaces to segment 3D images. InConference on Computer Vision and Pattern Recognition, pp. 738–739.

  • Cryer, J.E., Tsai P.-S., and Shah, M. 1992.Combining shape from shading and stereo using human vision model. Technical Report CS-TR-92-25, U. Central Florida.

  • Delingette, H., Hebert, M., and Ikeuchi, K. 1991. Shape representation and image segmentation using deformable surfaces. InConference on Computer Vision and Pattern Recognition, pp. 467-472.

  • Diehl, H. and Heipke, C. 1992. Surface reconstruction from data of digital line cameras by means of object based image matching. InInternational Society for Photogrammetry and Remote Sensing, Washington, DC, pp. 287–294.

  • Faugeras, O. and Toscani, G. 1986. The calibration problem for stereo. InConference on Computer Vision and Pattern Recognition, Miami Beach, Florida, pp. 15–20.

  • Ferrie, F.P., Lagarde, J., and Whaite, P. 1992. Recovery of volumetric object descriptions from laser rangefinder images. InEuropean Conference on Computer Vision, Genoa, Italy.

  • Fua, P. 1993. A parallel stereo algorithm that produces dense depth maps and preserves image features.Machine Vision and Applications, 6(1). Available as INRIA research report 1369.

  • Fua, P. and Leclerc, Y.G. 1990. Model driven edge detection. Machine Vision and Applications, 3:45–56.

    Google Scholar 

  • Fua, P. and Sander, P. 1992. Segmenting unstructured 3d points into surfaces. InEuropean Conference on Computer Vision, Genoa, Italy.

  • Grimson, W.E.L. and Huttenlocher, D.P. 1992. Introduction to the special issue on interpretation of 3-d scenes.IEEE Transactions on Pattern Analysis and Machine Intelligence, 14(2):97–98.

    Google Scholar 

  • Hannah, M.J. 1989. A system for digital stereo image matching.Photogrammetric Engineering and Remote Sensing, 55(12):1765–1770.

    Google Scholar 

  • Hartt, K. and Carlotto, M. 1989. A method for shape-from-shading using multiple images acquired under different viewing and lighting conditions. InConference on Computer Vision and Pattern Recognition, pp. 53–60.

  • Heipke, C. 1992. Integration of digital image matching and multi-image shape from shading. InInternational Society for Photogrammetry and Remote Sensing, Washington, DC, pp. 832–841.

  • Horn, B.K.P. 1990. Height and gradient from shading.International Journal of Computer Vision, 5(1):37–75.

    Google Scholar 

  • Hung, Y., Cooper, D.B., and Cernuschi-Frias, B. 1991. Asymptotic bayesian surface estimation using an image sequence.International Journal of Computer Vision, 6(2):105–132.

    Google Scholar 

  • Kaiser, B., Schmolla, M., and Wrobel, B.P. 1992. Application of image pyramid for surface reconstruction with fast vision. InInternational Society for Photogrammetry and Remote Sensing, Washington, DC, p. 1.

  • Kanade, T. and Okutomi, M. 1990. A stereo matching algorithm with an adaptative window: Theory and experiment. InARPA Image Understanding Workshop.

  • Kass, M., Witkin, A., and Terzopoulos, D. 1988. Snakes: Active contour models.International Journal of Computer Vision, 1(4):321–331.

    Google Scholar 

  • Leclerc, Y.G. 1989a. Constructing simple stable descriptions for image partitioning.International Journal of Computer Vision, 3(1):73–102.

    Google Scholar 

  • Leclerc, Y.G. 1989b.The Local Structure of Image Intensity Discontinuities. Ph.D. thesis, McGill University, Montréal, Québec, Canada

    Google Scholar 

  • Leclerc, Y.G. and Bobick, A.F. 1991. The direct computation of height from shading. InConference on Computer Vision and Pattern Recognition, Lahaina, Maui, Hawaii.

  • Liedtke, C.E., Busch, H., and Koch, R. 1991. Shape adaptation for modelling of 3D objects in natural scenes. InConference on Computer Vision and Pattern Recognition, pp. 704–705.

  • Lowe, D.G. 1991. Fitting parameterized three-dimensional models to images.IEEE Transactions on Pattern Analysis and Machine Intelligence, 13:441–450.

    Google Scholar 

  • Luenberger, D.G. 1984.Linear and Nonlinear Programming. Addison-Wesley: Menlo Park, California, second edition.

    Google Scholar 

  • Marr, D. 1982.Vision. W.H. Freeman: San Francisco, California.

    Google Scholar 

  • Nishihara, H. 1984. Practical real-time imaging stereo matcher.Optical Engineering, 23(5).

  • Okutomi, M. and Kanade, T. 1991. A multiple-baseline stereo. InComputer Vision and Pattern Recognition 91, Maui, Hawaii, pp. 63–69.

  • Oren, M. and Nayar, S. 1993. Generalization of the lambertian model. InARPA Image Understanding Workshop, pp. 1037–1048.

  • Panton, D.J. 1978. A flexible approach to digital stereo mapping.Photogramm. Eng. Remote Sensing, 44(12):1499–1512.

    Google Scholar 

  • Pentland, A. 1990. Automatic extraction of deformable part models. International Journal of Computer Vision, 4(2):107–126.

    Google Scholar 

  • Pentland, A. and Sclaroff, S. 1991. Closed-form solutions for physically based shape modeling and recognition.IEEE Transactions on Pattern Analysis and Machine Intelligence, 13:715–729.

    Google Scholar 

  • Poggio, T., Torre, V., and Koch, C. 1985. Computational vision and regularization theory.Nature, Vol. 317.

  • Press, W., Flannery, B., Teukolsky, S., and Vetterling, W. 1986.Numerical Recipes, the Art of Scientific Computing. Cambridge U. Press: Cambridge, MA.

    Google Scholar 

  • Quam, L. 1984. Hierarchical warp stereo. InARPA Image Understanding Workshop, pp. 149–155.

  • Stokely, E.M. and Wu, S.Y. 1992. Surface parameterization and curvature measurement of arbitrary 3-d objects: five practical methods. IEEE Transactions on Pattern Analysis and Machine Intelligence, 14(8):833–839.

    Google Scholar 

  • Szeliski, R. 1991. Shape from rotation. InConference on Computer Vision and Pattern Recognition, pp. 625–630.

  • Szeliski, R. and Tonnesen, D. 1992. Surface modeling with oriented particle systems. InComputer Graphics (SIGGRAPH'92), pp. 185–194.

  • Terzopoulos, D. 1986. Regularization of inverse visual problems involving discontinuities.IEEE Transactions on Pattern Analysis and Machine Intelligence, 8:413–424.

    Google Scholar 

  • Terzopoulos, D. 1988. The computation of visible-surface representations. IEEE Transactions on Pattern Analysis and Machine Intelligence, pp. 417–438.

  • Terzopoulos, D. and Metaxas, D. 1991. Dynamic 3D models with local and global deformations: Deformable superquadrics.IEEE Transactions on Pattern Analysis and Machine Intelligence, 13:703–714.

    Google Scholar 

  • Terzopoulos, D. and Vasilescu, M. 1991. Sampling and reconstruction with adaptive meshes. InConference on Computer Vision and Pattern Recognition, pp. 70–75.

  • Terzopoulos, D., Witkin, A., and Kass, M. 1987. Symmetry-seeking models and 3D object reconstruction.International Journal of Computer Vision, 1:211–221.

    Google Scholar 

  • Tomasi, C. and Kanade, T. 1992. The factorization method for the recovery of shape and motion from image streams. InARPA Image Understanding Workshop, pp. 459–472.

  • Vemuri, B.C. and Malladi, R. 1991. Deformable models: Canonical parameters for surface representation and multiple view integration. InConference on Computer Vision and Pattern Recognition, pp. 724–725.

  • Wang, Y.F. and Wang, J.F. 1992. Surface reconstruction using deformable models with interior and boundary constraints.IEEE Transactions on Pattern Analysis and Machine Intelligence, 14(5):572–579.

    Google Scholar 

  • Whaite, P. and Ferrie, F.P. 1991. From uncertainty to visual exploration. IEEE Transactions on Pattern Analysis and Machine Intelligence, 13:1038–1049.

    Google Scholar 

  • Witkin, A.W., Terzopoulos, D., and Kass, M. 1987. Signal matching through scale space.International Journal of Computer Vision, 1:133–144.

    Google Scholar 

  • Wrobel, B.P. 1991. The evolution of digital photogrammetry from analytical photogrammetry. Photogrammetric Record, 13(77):765–776.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. SRI International, 333 Ravenswood Avenue, 94025, Menlo Park, CA

    P. Fua & Y. G. Leclerc

Authors
  1. P. Fua
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Y. G. Leclerc
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

About this article

Cite this article

Fua, P., Leclerc, Y.G. Object-centered surface reconstruction: Combining multi-image stereo and shading. Int J Comput Vision 16, 35–56 (1995). https://doi.org/10.1007/BF01428192

Download citation

  • Received:

  • Revised:

  • Issue Date:

  • DOI: https://doi.org/10.1007/BF01428192

Keywords

Search

Navigation