Skip to main content
SpringerLink
Log in

Stereo vision for planetary rovers: Stochastic modeling to near real-time implementation

  • Published:
International Journal of Computer Vision Aims and scope Submit manuscript

Abstract

NASA scenarios for lunar and planetary missions include robotic vehicles that function in both teleoperated and semi-autonomous modes. Under teleoperation, on-board stereo cameras may provide 3-D scene information to human operators via stereographic displays; likewise, under semi-autonomy, machine stereo vision may provide 3-D information for obstacle avoidance. In the past, the slow speed of machine stereo vision systems has posed a hurdle to the semi-autonomous scenario; however, recent work at JPL and other laboratories has produced stereo systems with high reliability and near real-time performance for low-resolution image pairs. In particular, JPL has taken a significant step by achieving the first autonomous, cross-country robotic traverses (of up to 100 meters) to use stereo vision, with all computing on-board the vehicle. Here, we describe the stereo vision system, including the underlying statistical model and the details of the implementation. The statistical and algorithmic aspects employ random field models of the disparity map, Bayesian formulations of single-scale matching, and area-based image comparisons. The implementation builds bandpass image pyramids and produces disparity maps from the 60×64 level of the pyramids at rates of up to two seconds per image pair. All vision processing is done in one 68020 augmented with Datacube image processing boards. We argue that the overall approach provides a unifying paradigm for practical, domain-independent stereo ranging. We close with a discussion of practical and theoretical issues involved in evaluating and extending the performance of the stereo system.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. P. Anandan, Computing dense displacement fields with confidence measures in scenes containing occlusion, Proc. DARPA Image Understanding Workshop, pp. 236–246. SAIC, 1984.

  2. N.Ayache, and O.D.Faugeras, Building, registrating, and fusing noisy visual maps, Intern. J. Robot. Res. 7(6):45–65, December 1988.

    Google Scholar 

  3. N. Ayache and F. Lustman, Fast and reliable passive stereovision using three cameras, Proc. Intern. Workshop on Industrial Applications of Machine Vision and Machine Intelligence, pp. 186–191, February 1987.

  4. H.H. Baker, Depth from edge and intensity based stereo. Ph.D. thesis, Stanford University, 1982.

  5. H.H. Baker and T.O. Binford, Depth from edge and intensity based stereo, Proc. 7th Intern. Joint conf. Artif. Intell., pp. 631–636, August 1981.

  6. S.T.Barnard, Stochastic stereo matching over scale, Intern. J. Comput. Vision 3(1):17–32, May 1989.

    Google Scholar 

  7. P.J. Burt, C.H. Anderson, J.O. Sinniger, and G.S. van der Wal. A pipelined pyramid machine. In Pyramidal Systems for Computer Vision. Vol. NATO ASI Series, vol. F25, pp. 133–152. Springer-Verlag, 1986.

  8. P.J.Burt, Fast filter transforms for image processing, Comput. Graph. Image Process. 16:20–51, 1981.

    Google Scholar 

  9. L. Chen and T. Boult, An integrated approach to stereo matching, surface reconstruction and depth segmentation using consistent smoothness assumptions, Proc. DARPA Image Understanding Workship, Washington, pp. 166–176, Morgan Kaufmann Publishers, April 1988.

  10. J.L.Crowley and R.M.Stern, Fast computation of the difference of low-pass transform, IEEE Trans. Patt. Anal. Mach. Intell. 6(2):212–222, March 1984.

    Google Scholar 

  11. A.Elfes, Sonar-based real-world mapping and navigation, IEEE J. Robot. Autom. 3(3):249–265, June 1987.

    Google Scholar 

  12. W. Forstner, Personal communication, 1988.

  13. W.Forstner and A.Pertl. Photogrammetric standard methods and digital image matching techniques for high precision surface measurements. In E.S.Gelsema and L.N.Kanal, ed. Pattern Recognition in Practice II, Elsevier Science Publishers: New York, pp. 57–72, 1986.

    Google Scholar 

  14. E. Gat, M.G. Slack, D.P. Miller, and R.J. Firby, Path planning and execution monitoring for a planetary rover, Proc. IEEE Intern. Conf. Robot. Autom. IEEEE Computer Society Press, pp. 20–25, May 1990.

  15. D. Gennery. Personal communication, 1990.

  16. D.B. Gennery. Modeling the environment of an exploring vehicle by means of stereo vision. Ph.D. thesis, Stanford University, June 1980.

  17. D.B.Gennery, Visual terrain matching for a Mars rover, Proc. Conf. Comput. Vision Patt. Recog., San Diego, IEEE Computer Society Press, pp. 483–491, June 1989.

    Google Scholar 

  18. F.A.Graybill, Matrices with Applications in Statistics. Wadsworth International Group: Belmont, CA, 1983.

    Google Scholar 

  19. B.K.P.Horn, Robot Vision. MIT Press: Cambridge, MA 1986.

    Google Scholar 

  20. A.K.Jain. Fundamentals of Digital Image Processing. Prentice Hall: Englewood Cliffs, NJ, 1989.

    Google Scholar 

  21. J.R. Jordan and A.C. Bovik, Computational stereo vision using color, IEEE Cont. Syst. Mag. pp. 31–36, June 1988.

  22. A.E. Kayaalp, High speed machine perception using passive sensing technology, Proc. SPIE Conf. 1195, Mobile Robots IV, pp. 62–74, November 1989.

  23. D.Marr and T.Poggio, Cooperative computation of stereo disparity, Science, 194:283–287, 1976.

    Google Scholar 

  24. J.Marroquin, S.Mitter, and T.Poggio. Probabilistic solution of ill-posed problems in computational vision, J. Amer. Stat. Assoc., 82(397):76–89, March 1987.

    Google Scholar 

  25. J.L. Marroquin. Probabilistic Solution of Inverse Problems. Ph.D. thesis, MIT, September 1985.

  26. L.H. Matthies. Dynamic Stereo Vision. Ph.D. thesis, Carnegie Mellon University, October 1989.

  27. L.H. Matthies, Toward stochastic modeling of obstacle detectability in passive stereo range imagery, Proc. Conf. Comput. Vision Patt. Recog., IEEE Computer Society, June 1992.

  28. L.H. Matthies and M. Okutomi. A Bayesian foundation for active stereo vision. In P. Schenker Proc. SPIE Conf. 1198, Sensor Fusion II: Human and Machine Strategies, pp. 62–74, November 1989.

  29. L.H.Matthies and S.A.Shafer, Error modeling in stereo navigation, IEEE J. Robot. Autom., RA-3(3):239–248, June 1987.

    Google Scholar 

  30. L.H.Matthies, R.Szeliski, and T.Kanade, Kalman filter-based algorithms for estimating depth from image sequences. Intern. J. Comput. Vision, 3:209–236, 1989.

    Google Scholar 

  31. P.S.Maybeck, Stochastic Models, Estimation, and Control, vol.. 1, Academic Press: New York, 1979.

    Google Scholar 

  32. V.J. Milenkovic and T. Kanade, Trinocular vision using photometric and edge orientation constraints, Proc. DARPA Image Understanding Workship, December 1985.

  33. H.P. Moravec. Obstacle avoidance and navigation in the real world by a seeing robot rover. Ph.D. thesis, Stanford University, September 1980.

  34. H.K.Nishihara, Practical real-time imaging stereo matcher, Optical Engineering 23(5):536–545, October 1984.

    Google Scholar 

  35. H.K. Nishihara, RTVS-3: Real-time binocular stereo and optical flow measurement system. System description manuscript, July 1990.

  36. Y.Ohta and T.Kanade, Stereo by intra- and inter-scanline search using dynamic programming, IEEE Trans. Patt. Anal. Mach. Intell., 7(2):139–154, March 1985.

    Google Scholar 

  37. T.Poggio, V.Torre, and C.Koch, Computational vision and regularization theory, Nature 317(n):314–319, September 1985.

    Google Scholar 

  38. K.Prazdny, Detection of binocular disparities, Biological Cybernetics 52:93–99, 1985.

    Google Scholar 

  39. L.H. Quam, Hierarchical warp stereo, Proc. Image Understanding Workshop. Science Applications International, 1984.

  40. T.W.Ryan, R.T.Gray, and B.R.Hunt, Prediction of correlation errors in stereo-pair images, Optical Engineering, 19(3):312–322, May/June 1980.

    Google Scholar 

  41. D.Sankoff and J.B.Kruskal, Time Warps, String Edits, and Macromolecules: the Theory and Practice of Sequence Comparison. Addison-Wesley: Reading, MA, 1983.

    Google Scholar 

  42. R.Sedgewick, Algorithms. Addison-Wesley: Reading, MA, 1983.

    Google Scholar 

  43. M.G. Slack. Situationally driven local navigation for mobile robots. Technical report, Jet Propulsion Laboratory, April 1990.

  44. C.C.Slama, ed. Manual of Photogrammetry, 4th ed. American Society of Photogrammetry: Falls Church, A, 1980.

    Google Scholar 

  45. C.V. Stewart and C.R. Dyer, The trinocular general support algorithm: a three-camera stereo algorithm for overcoming binocular matching errors, Proc. 2nd Itern. Conf. Comput. Vision, Tarpon Springs, FL, pp. 134–138, December 1988.

  46. R.Szeliski. Cooperative algorithms for solving random-dot stereograms. Technical Report CMU-CS-8–133, Computer Science Department, Carnegie Mellon University: Pittsburgh, PA, June 1986.

    Google Scholar 

  47. R. Szeliski, Regularization uses fractal priors, Proc. AAAI-87: 6th Natl. Conf. Artif. Intell., Morgan Kaufmann Publishers, pp. 749–754, 1987.

  48. R. Szeliski, bayesian Modeling of Uncertainty in Low Level Vision. Ph.D. thesis, Carnegie Mellon University, August 1988.

  49. R. Szeliski and G. Hinton, Solving random-dot stereograms using the heat equation, Proc. Conf. Comput. Vision Patt. Recog., San Francisco, pp. 284–288, 1985.

  50. D.Terzopoulos, Multilevel computational processes for visual surface reconstruction, Comput. Vision, Graph, Image Process. 24:52–96, 1983.

    Google Scholar 

  51. D.Terzopoulos, Regularization of inverse visual problems involving discontinuities, Trans. Patt. Anal. Mach. Intell. 8(4):413–424, July 1986.

    Google Scholar 

  52. H.L.VanTrees, Detection, Estimation, and Modulation Theory, Part I. Wiley: New York, 1968.

    Google Scholar 

  53. A.Witkin, D.Terzopoulos, and M.Kass, Signal matching through scale space, Intern. J. Comput. Vision 1(2):133–144, 1987.

    Google Scholar 

  54. G. Xu, S. Tsuji, and M. Asada, Coarse-to-fine control strategy for matching motion stereo pairs, Proc. 9th Intern. Joint Conf. Artif. Intell., Los Angeles, pp. 892–894, 1985.

Download references

Author information

Authors and Affiliations

  1. Robotic Systems and Advanced Computer Technology Section, Jet Propulsion Laboratory, California Institute of Technology, 91109, Pasadena, CA

    Larry Matthies

Authors
  1. Larry Matthies
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

About this article

Cite this article

Matthies, L. Stereo vision for planetary rovers: Stochastic modeling to near real-time implementation. Int J Comput Vision 8, 71–91 (1992). https://doi.org/10.1007/BF00126401

Download citation

  • Received:

  • Revised:

  • Issue Date:

  • DOI: https://doi.org/10.1007/BF00126401

Keywords

Search

Navigation