Skip to main content
SpringerLink
Log in

Plenoptic Image Editing

  • Published:
International Journal of Computer Vision Aims and scope Submit manuscript

Abstract

This paper presents a new class of interactive image editing operations designed to maintain consistency between multiple images of a physical 3D scene. The distinguishing feature of these operations is that edits to any one image propagate automatically to all other images as if the (unknown) 3D scene had itself been modified. The modified scene can then be viewed interactively from any other camera viewpoint and under different scene illuminations. The approach is useful first as a power-assist that enables a user to quickly modify many images by editing just a few, and second as a means for constructing and editing image-based scene representations by manipulating a set of photographs. The approach works by extending operations like image painting, scissoring, and morphing so that they alter a scene's plenoptic function in a physically-consistent way, thereby affecting scene appearance from all viewpoints simultaneously. A key element inrealizing these operations is a new volumetric decomposition technique for reconstructing an scene's plenoptic function from an incomplete set of camera viewpoints.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Adelson, E.H. and Bergen, J.R. 1991. The plenoptic function and the elements of early vision. In Computation Models of Visual Processing, M. Landy and J.A. Movshon (Eds.). MIT Press: Cambridge, MA.

    Google Scholar 

  • Adobe Systems, Inc., 1998. Photoshop TM 5.0, Computer Program.

  • Avidan, S. and Shashua, A. 1998. Novel view synthesis by cascading trilinear tensor. Transactions on Visualization and Computer Graphics, 4(4):1034–1040.

    Google Scholar 

  • Basri, R. and Jacobs, R.B.D.W. 2001. Lambertian reflectance and linear subspaces. In Proc. 8th Int. Conf. on Computer Vision, pp. 383–390.

  • Beier, T. and Neely, S. 1992. Feature-based image metamorphosis. In Proc. SIGGRAPH 92, pp. 35–42.

  • Belhumeur, P.N. and Kriegman, D.J. 1998. What is the set of images of an object under all possible lighting conditions? Int. J. Computer Vision, 28(3):245–260.

    Google Scholar 

  • Benton, S. 1983. Survey of holographic stereograms. In Processing and Display of Three-Dimensional Data, Proc. SPIE.

  • Bolles, R.C., Baker, H.H., and Marimont, D.H. 1987. Epipolarplane image analysis: An approach to determining structure from motion. Int. J. Computer Vision, 1:7–55.

    Google Scholar 

  • Cipolla, R. and Blake, A. 1992. Surface shape from the deformation of apparent contours. Int. J. Computer Vision, 9(2):83–112.

    Google Scholar 

  • Collins, R.T. 1996. A space-sweep approach to true multi-image matching. In Proc. Computer Vision and Pattern Recognition Conf., pp. 358–363.

  • Debevec, P.E., Taylor, C.J., and Malik, J. 1996. Modeling and ren-dering architecture from photographs: A hybrid geometry-and image-based approach. In Proc. SIGGRAPH'96, pp. 11–20.

  • Epstein, R., Yuille, A.L., and Belhumeur, P.N. 1996. Learning object representations from lighting variations. In Object Representation in Computer Vision II, J. Ponce, A. Zisserman, and M. Hebert (Eds.). Springer-Verlag: Berlin, pp. 179–199.

    Google Scholar 

  • Faugeras, O.D. and Keriven, R. 1998. Variational principles, surface evolution, pde', level set methods and the stereo problem. IEEE Trans. Image Processing, 7(3):336–344.

    Google Scholar 

  • Fitzgibbon, A.W. and Zisserman, A. 1998. Automatic camera recovery for closed or open image sequences. In Proc. 5th European Conf. on Computer Vision, pp. 311–326.

  • Foley, J.D., van Dam, A., Feiner, S.K., and Hughes, J.F. 1990. Computer Graphics Principles and Practice. Addison-Wesley: Reading, MA.

    Google Scholar 

  • Gleicher, M. 1995. Image snapping. In Proc. SIGGRAPH'95, pp. 183–190.

  • Gortler, S.J., Grzeszczuk, R., Szeliski, R., and Cohen, M.F. 1996. The lumigraph. In Proc. SIGGRAPH'96, pp. 43–54.

  • Hanrahan, P. and Haeberli, P. 1990. Direct WYSIWYG painting and texturing on 3D shapes. In Proc. SIGGRAPH'90, pp. 215–223.

  • Katayama, A., Tanaka, K., Oshino, T., and Tamura, H. 1995. A view-point dependent stereoscopic display using interpolation of multi-viewpoint images. In Proc. SPIE, vol. 2409A, pp. 21–30.

    Google Scholar 

  • Kutulakos, K.N. 1997. Shape from the light field boundary. In Proc. Computer Vision and Pattern Recognition, pp. 53–59.

  • Kutulakos, K.N. 2000. Approximate N-view stereo. In Proc. European Conf. on Computer Vision, pp. 67–83.

  • Kutulakos, K.N. and Dyer, C.R. 1994. Recovering shape by purpo-sive viewpoint adjustment. Int. J. Computer Vision, 12(2):113–136.

    Google Scholar 

  • Kutulakos, K.N. and Seitz, S.M. 2000. A theory of shape by space carving. Int. J. Computer Vision (Marr Prize Special Issue), 38(3):199–218.

    Google Scholar 

  • Laveau, S. and Faugeras, O. 1994. 3-D scene representation as a collection of images. In Proc. Int. Conf. on Pattern Recognition, pp. 689–691.

  • Levoy, M. and Hanrahan, P. 1996. Light field rendering. In Proc. SIGGRAPH'96, pp. 31–42.

  • Lichtenbelt, B., Crane, R., and Naqvi, S. 1998. Introduction to Volume Rendering. Prentice-Hall: Englewood Cliffs, NJ.

    Google Scholar 

  • McMillan, L. and Bishop, G. 1995. Plenoptic modeling: An image-based rendering system. In Proc. SIGGRAPH'95, pp. 39–46.

  • Moezzi, S., Katkere, A., Kuramura, D.Y., and Jain, R. 1996. Reality modeling and visualization from multiple video sequences. IEEE Computer Graphics and Applications, 16(6):58–63.

    Google Scholar 

  • Mortensen, E.N. and Barrett, W.A. 1995. Intelligent scissors for image composition. In Proc. SIGGRAPH'95, pp. 191–208.

  • Narayanan, P.J., Rander, P.W., and Kanade, T. 1998. Constructing virtual worlds using dense stereo. In Proc. Int. Conf. on Computer Vision, pp. 3–10.

  • Nielson, G. 1993. Scattered data modeling. IEEE Computer Graphics and Applications, pp. 60–70.

  • Preparata, F.P. and Shamos, M.I. 1985. Computational Geometry. Springer-Verlag: Berlin.

    Google Scholar 

  • Ramamoorthi, R. and Hanrahan, P. 2001. Asignal-processing frame-work for inverse rendering. In Proc. SIGGRAPH'01, pp. 117–128.

  • Sato, Y., Wheeler, M.D., and Ikeuchi, K. 1997. Object shape and reflectance modeling from observation. In Proc. SIGGRAPH'97, pp. 379–387.

  • Schroder, P. and Sweldens, W. 1995. Spherical wavelets: Efficiently representing functions on the sphere. In Proc. SIGGRAPH'95, pp. 161–172.

  • Seitz, S.M. and Dyer, C.R. 1995. Physically-valid view synthesis by image interpolation. In Proc. IEEE Workshop on Representations of Visual Scenes, pp. 18–25.

  • Seitz, S.M. and Dyer, C.R. 1996. View morphing. In Proc. SIGGRAPH 96, pp. 21–30.

  • Seitz, S.M. and Dyer, C.R. 1999. Photorealistic scene reconstruction by voxel coloring. Int. J. Computer Vision, 35(2):151–173.

    Google Scholar 

  • Shade, J.W., Gortler, S.J., He, W.-L., and Szeliski, R. 1998. Layered depth images. In Proc. SIGGRAPH 98, pp. 231–242.

  • Shashua, A. 1992. Geometry and photometry in 3D visual recognition. PhD thesis, MIT.

  • Sillion, F.X., Arvo, J.R., Westin, S.H., and Greenberg, D.P. 1991. A global illumination solution for general reflectance distribution functions. In Proc. SIGGRAPH'91, pp. 187–196.

  • Stollnitz, E.J., DeRose, T.D., and Salesin, D.H. 1996. Wavelets for Computer Graphics. Morgan Kaufmann: San Mateo, CA.

    Google Scholar 

  • Szeliski, R. 1993. Rapid octree construction from image sequences. CVGIP: Image Understanding, 58(1):23–32.

    Google Scholar 

  • Wilkes, D. and Tsotsos, J.K. 1992. Active object recognition. In Proc. Computer Vision and Pattern Recognition Conf., pp. 136–141.

  • Wood, D.N., Azuma, D.I., Aldinger, K., Curless, B., Duchamp, T., Salesin, D.H., and Stuetzle, W. 2000. Surface light fields for 3d photography. In Proc. SIGGRAPH'00, pp. 287–296.

  • Woodham, R.J., Iwahori, Y., and Barman, R.A. 1991. Photometric stereo: Lambertian reflectance and light sources with unknown direction and strength. Technical Report 91-18, University of British Columbia, Laboratory for Computational Intelligence.

  • Zeleznik, R.C., Herndon, K.P., and Hughes, J.F. 1996. SKETCH: An interface for sketching 3D scenes. In Proc. SIGGRAPH'96, pp. 163–170.

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science and Engineering, University of Washington, Seattle, WA, 98195, USA

    Steven M. Seitz

  2. Department of Computer Science, University of Toronto, Toronto, Ontario, M5S 3H5, Canada

    Kiriakos N. Kutulakos

Authors
  1. Steven M. Seitz
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kiriakos N. Kutulakos
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

About this article

Cite this article

Seitz, S.M., Kutulakos, K.N. Plenoptic Image Editing. International Journal of Computer Vision 48, 115–129 (2002). https://doi.org/10.1023/A:1016046923611

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1023/A:1016046923611

Search

Navigation