Abstract
We present a method to automatically extract spatio-temporal descriptions of moving objects from synchronized and calibrated multi-view sequences. The object is modeled by a time-varying multi-resolution subdivision surface that is fitted to the image data using spatio-temporal multi-view stereo information, as well as contour constraints. The stereo data is utilized by computing the normalized correlation between corresponding spatio-temporal image trajectories of surface patches, while the contour information is determined using incremental segmentation of the viewing volume into object and background. We globally optimize the shape of the spatio-temporal surface in a coarse-to-fine manner using the multi-resolution structure of the subdivision mesh. The method presented incorporates the available image information in a unified framework and automatically reconstructs accurate spatio-temporal representations of complex non-rigidly moving objects.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Bonet, J.S.D. and Viola, P. 1999. Roxels: Responsibility weighted 3d volume reconstruction. In Proceedings of ICCV, Sept. 1990.
Burt, P. and Adelson, E.H. 1983. The laplacian pyramid as a compact image code. IEEE Transactions on Communication, 31:532–540.
Carceroni, R.L. and Kutulakos, K. 2001. Multi-view scene capture by surfel sampling: From video streams to non-rigid 3d motion, shape, and reflectance. In Proc. Int. Conf. Computer Vision, June 2001.
Essa, I. and Pentland, A. 1997. Coding, analysis, interpretation, and recognition of facial expressions. IEEE Trans. PAMI, 19:757–763.
Faugeras, O. and Keriven, R. 1998. Complete dense stereovision using level set methods. In Proc. Europ. Conf. Computer Vision, Freiburg, Germany, 1998, pp. 379–393.
Fua, P. 2000. Regularized bundle-adjustment to model heads from image sequences without calibration data. Int. Journal of Computer Vision, 38:153–171.
Hoppe, H., DeRose, T., Duchamp, T., Halstead, M., Jin, H., McDonald, J., Schweitzer, J., and Stuetzle, W. 1994. Piecewise smooth surface reconstruction. In Proc. of ACM SIGGRAPH, 1994, pp. 295–302.
Horn, B.K.P. 1986. Robot Vision. McGraw Hill: New York.
Hubeli, A. and Gross, M. 2000. A survey of surface representations for geometric modeling. Technical Report 335, ETH Zürich, Institute of Scientific Computing.
Kakadiaris, I. and Metaxas, D. 1998. Three-dimensional human body model acquisition from multiple views. Int. Journal of Computer Vision, 30:191–218.
Kobbelt, L., Bareuther, T., and Seidel, H.-P. 2000. Multiresolution shape deformations for meshes with dynamic vertex connectivity. In Eurographics 2000 proceedings.
Kutulakos, K.N. and Seitz, S.M. 2000. A theory of shape by space carving. Int. Journal of Computer Vision, 38(3):199–218.
Laurentini, A. 1994. The visual hull concept for silhouette-based image understanding. IEEE Trans. PAMI, 16:150–162.
Loop, C. 1987. Smooth subdivision surfaces based on triangles. Master's thesis, University of Utah, USA.
Malassiotis, S. and Strintzis, M. 1997. Model-based joint motion and structure estimation from stereo images. Computer Vision and Image Understanding, 65:79–94.
Mandal, C., Qin, H., and Vemuri, B. 1999. Physics-based shape modeling and shape recovery using multiresolution subdivision surfaces. In Proc. of ACM SIGGRAPH.
Matusik, W., Buehler, C., Gortler, S.J., Raskar, R., and McMillan, L. 2000. Image based visual hulls. In Proc. of ACM SIGGRAPH.
Plänkers, R. and Fua, P. 2001. Tracking and modeling people in video sequences. Int. Journal of Computer Vision, 81(4):285–302.
Schröder, P. and Zorin, D. 2000. Subdivision for modeling and animation. Siggraph 2000 Course Notes.
Seitz, S. and Dyer, C. 1999. Photorealistic scene reconstruction by voxel coloring. Int. Journal of Computer Vision, 25.
Spies, H., Jähne, B., and Barron, J. 2000. Regularised range flow. In Proc. Europ. Conf. Computer Vision, Dublin, Ireland, June 2000.
Stam, J. 1999. Evaluation of loop subdivision surfaces. SIGGRAPH'99 Course Notes.
Taubin, G. 1995. A signal processing approach to fair surface design. In Proc. of ACM SIGGRAPH.
Vedula, S., Baker, S., Rander, P., Collins, R., and Kanade, T. 1999. Three-dimensional scene flow. In Proc. Int. Conf. Computer Vision, Corfu,Greece, Sept. 1999.
Vedula, S., Baker, S., Seitz, S., and Kanade, T. 2000. Shape and motion carving in 6d. In Proc. IEEE Conf. Computer Vision and Pattern Recognition, Head Island, South Carolina, USA, June 2000.
Vedula, S., Rander, P., Saito, H., and Kanade, T. 1998. Modeling, combining, and rendering dynamic real-world events from image sequences. In Proc. of Int. Conf. on Virtual Systems and Multimedia, Gifu, Japan, Nov. 1998.
Zhang, Y. and Kambhamettu, C. 2000. Integrated 3d scene flow and structure recovery from multiviewimage sequences. In Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. II:674–681, Hilton Head.
Zorin, D., Schrder, P., and Sweldens, W. 1997. Interactive multi-resolution mesh editing. In Proc. of ACM SIGGRAPH.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Neumann, J., Aloimonos, Y. Spatio-Temporal Stereo Using Multi-Resolution Subdivision Surfaces. International Journal of Computer Vision 47, 181–193 (2002). https://doi.org/10.1023/A:1014597925429
Issue Date:
DOI: https://doi.org/10.1023/A:1014597925429