Skip to main content
SpringerLink
Log in

Closed-form solutions to image flow equations for 3D structure and motion

  • Published:
International Journal of Computer Vision Aims and scope Submit manuscript

Abstract

A major source of three-dimensional (3D) information about objects in the world is available to the observer in the form of time-varying imagery. Relative motion between textured objects and the observer generates a time-varying optic array at the image, from which image motion of contours, edge fragments, and feature points can be extracted. These dynamic features serve to sample the underlying “image flow” field. New, closed-form solutions are given for the structure and motion of planar and curved surface patches from monocular image flow and its derivatives through second order. Both planar and curved surface solutions require at most, the solution of a cubic equation. The analytic solution for curved surface patches combines the transformation of Longuet-Higgins and Prazdny [25] with the planar surface solution of Subbarao and Waxman [43]. New insights regarding uniqueness of solutions also emerge. Thus, the “structure-motion coincidence” of Waxman and Ullman [54] is interpreted as the “duality of tangent plane solutions.” The multiplicity of transformation angles (up to three) is related to the sign of the Gaussian curvature of the surface patch. Ovoid patches (i.e., bowls) are shown to possess a unique transform angle, though they are subject to the local structure-motion coincidence. Thus, ovoid patches almost always yield a unique 3D interpretation. In general, ambiguous solutions can be resolved by requiring continuity of the solution over time.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. E.H. Adelson and J.R. Bergen, “Spatiotemporal energy models for the perception of motion,” J. Opt. Soc. Amer. vol. A2, pp. 284–299, 1985.

    Google Scholar 

  2. G. Adiv, “Determining 3-D motion and structure from optical flow generated by several moving objects,” PROC. DARPA IMAGE UNDERSTANDING WORKSHOP, New Orleans: SAIC October 1984, pp. 113–129.

  3. C.H. Anderson, P.J. Burt, and G.S. van der Wal, “Change detection and tracking using pyramid transform techniques,” in PROC. SPIE CONF. ON INTELLIGENT ROBOTS AND COMPUTER VISION, Boston, September, 1985.

  4. R. Aris, Vectors, Tensors, and the Basic Equations of Fluid Mechanics. Prentice-Hall: Englewood Cliffs, 1962.

    Google Scholar 

  5. A. Bandyopadhyay, “Constraints on the computation of rigid motion parameters from retinal displacements,” University of Rochester, Dept. of Computer Science Tech. Report 168, October, 1985.

  6. M.L. Braunstein, Depth Perception Through Motion. Academic Press: New York, 1976.

    Google Scholar 

  7. M.L. Braunstein, “Perception of rotation in depth: The psychophysical evidence,” in Proc. Workshop on Motion: Representation and Perception, Toronto: ACM, April, 1983, pp. 119–124.

    Google Scholar 

  8. B.F. Buxton, H. Buxton, D.W. Murray, and N.S. Williams, “3-D solutions to the aperture problem,” PROC. 6TH EUROPEAN CONF. ARTIF. INTEL., Pisa, 1984, pp. 631–640.

  9. J. Doner, J.S. Lappin, and G. Perfetto, “Detection of three-dimensional structure in moving optical patterns,” J. Exp. Psychology: Human Perception and Performance, vol. 10, 1984, pp. 1–11.

    Google Scholar 

  10. D.J. Fleet and A.D. Jepson, “Velocity extraction without form perception,” in Proc. Workshop on Computer Vision: Representation and Control, Bellaire: IEEE, October, 1985, pp. 179–185.

    Google Scholar 

  11. J.J. Gibson. The Senses Considered as Perceptual Systems. Houghton Mifflin: Boston, 1966.

    Google Scholar 

  12. J.J. Gibson. The Ecological Approach to Visual Perception. Houghton Mifflin: Boston, 1979.

    Google Scholar 

  13. J.C. Hay, “Optical motions and space perception: An extension of Gibson's analysis,” Psychological Review vol. 73, pp. 550–565, 1966.

    Google Scholar 

  14. E.C. Hildreth, “The measurement of visual motion,” Ph.D. thesis, Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science; also “Computing the velocity field along contours,” in PROC. WORKSHOP ON MOTION: REPRESENTATION AND PERCEPTION, Toronto: ACM, April, 1983, pp. 26–32.

  15. D.D. Hoffman and B.M. Bennett, “Inferring the relative three-dimensional positions of two moving points,” J. Opt. Soc. Amer. vol. A2, pp. 350–353, 1985.

    Google Scholar 

  16. B.K.P. Horn and B.G. Schunck, “Determining optical flow,” Artificial Intelligence vol. 17, pp. 185–203, 1981.

    Google Scholar 

  17. T.S. Huang, Image Sequence Processing and Dynamic Scene Analysis. Springer-Verlag: Berlin, 1983.

    Google Scholar 

  18. M.R.M. Jenkin, “The stereopsis of time-varying images,” Univ. of Toronto, Dept. of Computer Science Tech. Report RBCV-TR-83-3, September, 1984.

  19. G. Johansson, “Visual motion perception.” Sci. Amer. vol. 232, pp. 76–88, 1975.

    Google Scholar 

  20. K. Kanatani, “Structure from motion without correspondence: General principle,” in Proc. Darpa Image Understanding Workshop, Miami: SAIC, December, 1985, pp. 107–116.

    Google Scholar 

  21. J.J. Koenderink and A.J. van Doorn, “Invariant properties of the motion parallax field due to the movement of rigid bodies relative to an observer,” Optica Acta vol. 22, pp. 773–791, 1975.

    Google Scholar 

  22. J.J. Koenderink and A.J. van Doorn, “Local structure of movement parallax of the plane,” J. Opt. Soc. Amer. vol. 66, pp. 717–723, 1976.

    Google Scholar 

  23. H.C. Longuet-Higgins, “A computer algorithm for reconstructing a scene from two projections,” Nature vol. 293, pp. 133–135, 1981.

    Google Scholar 

  24. H.C. Longuet-Higgins, “The visual ambiguity of a moving plane,” in PROC. ROY. SOC. LONDON B223, pp. 165–175, 1984.

  25. H.C. Longuet-Higgins and K. Prazdny, “The interpretation of a moving retinal image,” in PROC. ROY. SOC. LONDON B208, pp. 385–397, 1980.

  26. D. Marr, Vision. Freeman: San Francisco: 1982.

    Google Scholar 

  27. D. Marr and S. Ullman, “Directional selectivity and its use in early visual processing,” in PROC. ROY. SOC. LONDON B211, pp. 151–180, 1981.

  28. S.J. Maybank, “The angular velocity associated with the optical flowfied arising from motion through a rigid environment,” in PROC. ROY. SOC. LONDON A401, pp. 317–326, 1985.

  29. A.J. McConnell, Applications of Tensor Analysis. Dover Press: New York, 1957.

    Google Scholar 

  30. A. Mitiche, “On combining stereopsis and kineopsis for space perception; First Conf. Art. Intell. Applications. Denver: IEEE, December, 1984, pp. 156–160.

    Google Scholar 

  31. D.W. Murray and B.F. Buxton, “Scene segmentation from visual motion using global optimization,” IEEE Trans. Pat. Anal. Mach. Intel. vol. 9, pp. 220–228, 1987.

    Google Scholar 

  32. H.-H. Nagel, “On the estimation of dense displacement vector fields from image sequences,” in Proc. Workshop on Motion: Representation and Perception, Toronto: ACM, April, 1983, pp. 59–65.

    Google Scholar 

  33. H.-H. Nagel, “Dynamic stereo vision in a robot feedback loop based on the evaluation of multiple interconnected displacement vector fields,” in Proc. 3rd Int. Symp. Robotics Res., Gouvieux, MIT Press: Cambridge; October, 1985.

    Google Scholar 

  34. K. Prazdny, “Egomotion and relative depth map from optical flow,” Biological Cybernetics vol. 36, pp. 87–102, 1980.

    Google Scholar 

  35. D. Regan and K.I. Beverley, “Binocular and monocular stimuli for motion in depth: Changing-disparity and changing-size feed the same motion-in-depth stage,” Vision Research vol. 19, pp. 1331–1342, 1979.

    Google Scholar 

  36. D. Regan and K.I. Beverley, “Visually gudied locomotion: Psychophysical evidence for a neural mechanism sensitive to flow patterns,” Science vol. 205, pp. 311–313, 1979.

    Google Scholar 

  37. D. Regan and K.I. Beverley, “Visual responses to vorticity and the neural analysis of optic flow,” J. Opt. Soc. Amer. A2, pp. 280–283, 1985.

    Google Scholar 

  38. W. Richards, “Structure from stereo and motion,” M.I.T. Artif. Intell. Laboratory Memo 731, 1983; also J. OPT. SOC. AMER. A2, pp. 343–349, 1985.

  39. B.G. Schunck, “Motion segmentation and estimation by constraint line clustering,” in Proc. Workshop on Computer Vision: Representation and Control, Annapolis: IEEE, April, 1984, pp. 58–62.

    Google Scholar 

  40. B.G. Schunck, “Image flow continuity equations for motion and density,” in Proc. of Workshop on Motion: Representation and Analysis, Charleston: ACM, May, 1986, pp. 89–94.

    Google Scholar 

  41. M. Subbarao, “Interpretation of visual motion: A computational study,” Ph.D. thesis, Univ. of Maryland, Dept. of Computer Science, 1986.

  42. M. Subbarao, “Interpretation of image motion fields: Rigid curved surfaces in motion,” Univ. of Maryland, Cent. Autom. Res. Tech. Report; also submitted to INT. J. COMPUTER VISION, 1986.

  43. M. Subbarao and A.M. Waxman, “On the uniqueness of image flow solutions for planar surfaces in motion,” in Proc. Workshop on Computer Vision: Representation and Control, Bellaire: IEEE, october, 1985, pp. 129–140; also in Computer Vision, Graphics and Image Processing vol. 36, pp. 208–228, 1986.

    Google Scholar 

  44. W.B. Thompson, K.M. Mutch and V.A. Berzins, “Dynamic occlusion analysis in optical flow fields,” IEEE Trans. Pami vol. 4, pp. 374–383, 1985.

    Google Scholar 

  45. J.T. Tood, “Perception of structure from motion: Is projective correspondence of moving elements a necessary condition?” J. Exp. Psychology: Human Perception and Performance vol. 11, pp. 689–710, 1985.

    Google Scholar 

  46. R.Y. Tsai and T.S. Huang, “Uniqueness and estimation of 3-D motion parameters of rigid objects with curved surfaces,” Univ. of Illinois, Coord. Sci. Lab. Report R-921, 1981.

  47. R.Y. Tsai and T.S. Huang, “Estimating 3-D motion parameters of a rigid planar patch,” Univ. of Illinois, Coord. Sci. Lab. Report R-922, 1981.

  48. S. Ullman, The Interpretation of Visual Motion. MIT Press: Cambridge; 1979.

    Google Scholar 

  49. S. Ullman, “Maximizing rigidity: the incremental recovery of structure from rigid and rubbery motion,” M.I.T. Artif. Intell. Lab. Memo 721, 1982.

  50. H. Wallach and D.N. O'Connell, “The kinetic depth effect,” J. Exp. Psychology vol. 45, pp. 205–217, 1953.

    Google Scholar 

  51. A.M. Waxman, “An image flow paradigm,” in Proc. Workshop on Computer Vision: Representation and Control, Annapolis: IEEE, April, 1984, pp. 49–57.

    Google Scholar 

  52. A.M. Waxman and J.H. Duncan, ”Binocular image flows: Steps toward stereo-motion fusion,” Univ. of Maryland, Cent. Autom. Res. Tech. Report 119, October, 1985; also IEEE TRANS PAMI vol. 8, pp. 715–729, 1986.

  53. A.M. Waxman and S. Sinha, “Dynamic Stereo: Passive ranging to moving objects from relative image flows,” in Proc. Darpa Image Understanding Workshop, New Orleans: SAIC, October, 1984, pp. 130–136.

    Google Scholar 

  54. A.M. Waxman, A.M. and S. Ullman, “Surface structure and 3-D motion from image flow: A kinematic analysis,” Univ. of Maryland, Cent. Autom. Res. Tech. Report 24, October, 1983; also INT. J. ROBOTICS RES. vol. 4(3), pp. 72–94, 1985.

  55. A.M. Waxman and K. Wohn, “Contour evolution, neighborhood deformation and global image flow: Planar surfaces in motion,” Univ. of Maryland, Cent. Autom. Res. Tech. Report 58, April, 1984; also INT. J. ROBOTICS RES. vol. 4(3), pp. 95–108, 1985.

  56. A.M. Waxman and K. Wohn, “Image flow theory: A framework for 3-D inference from time-varuing imagery,” in ADVANCES IN COMPUTER VISION (ed. C. Brown), Erlbaum Publ. (in press).

  57. R.M. Winger, Projective Geometry. Dover: New York, 1962.

    Google Scholar 

  58. K. Wohn, “A contour-based approach to image flow,” Ph.D. thesis, Univ. of Maryland, Depart. of Computer Science, September, 1984.

  59. K. Wohn and A.M. Waxman, “Contour evolution, neighborhood deformation and local image flow: Curved surfaces in motion,” Univ. of Maryland, Cent. Autom. Res. Tech. Report 134.

  60. K. Wohn and A.M. Waxman, “The analytic structure of image flows: Deformation and segmentation.” Univ. of Maryland, Cent. Autom. Res. Tech. Report, January, 1987; submitted to COMPUTER VISION, GRAPHICS AND IMAGE PROCESSING, January, 1987.

Download references

Author information

Author notes

  1. Allen M. Waxman

    Present address: Laboratory for Sensory Robotics, College of Engineering, Boston University, 02215, Boston, MA

  2. Muralidhara Subbarao

    Present address: Department of Electrical Engineering, State University of New York, 11794, Stony Brook, N.Y.

Authors and Affiliations

  1. Computer Vision Laboratory, Center for Automation Research, University of Maryland, 20742, College Park, Md.

    Allen M. Waxman, Behrooz Kamgar-Parsi & Muralidhara Subbarao

Authors
  1. Allen M. Waxman
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Behrooz Kamgar-Parsi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Muralidhara Subbarao
    View author publications

    You can also search for this author in PubMed Google Scholar

Additional information

The support of the Defense Advanced Research Projects Agency and the U.S. Army Night Vision Laboratory under Contract DAAK70-83-K-0018 (DARPA Order 3206) is gratefully acknowledged.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Waxman, A.M., Kamgar-Parsi, B. & Subbarao, M. Closed-form solutions to image flow equations for 3D structure and motion. Int J Comput Vision 1, 239–258 (1988). https://doi.org/10.1007/BF00127823

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1007/BF00127823

Keywords

Search

Navigation