Skip to main content
SpringerLink
Log in

An Iterative Multiresolution Scheme for SFM with Missing Data

  • Published:
Journal of Mathematical Imaging and Vision Aims and scope Submit manuscript

Abstract

Several techniques have been proposed for tackling the Structure from Motion problem through factorization in the case of missing data. However, when the percentage of unknown data is high, most of them may not perform as well as expected. Focussing on this problem, an iterative multiresolution scheme, which aims at recovering missing entries in the originally given input matrix, is proposed. Information recovered following a coarse-to-fine strategy is used for filling in the missing entries. The objective is to recover, as much as possible, missing data in the given matrix. Thus, when a factorization technique is applied to the partially or totally filled in matrix, instead of to the originally given input one, better results will be obtained. An evaluation study about the robustness to missing and noisy data is reported. Experimental results obtained with synthetic and real video sequences are presented to show the viability of the proposed approach.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Aanaes, H., Fisker, R.: Robust factorization. IEEE Trans. Pattern Anal. Mach. Intell. 24, 1215–1225 (2002)

    Article  Google Scholar 

  2. Buchanan, A., Fitzgibbon, A.W.: Damped Newton algorithms for matrix factorization with missing data. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), vol. 2, pp. 316–322. University of Oxford, Oxford (2005)

    Chapter  Google Scholar 

  3. Chen, P., Suter, D.: Recovering the missing components in a large noisy low-rank matrix: Application to SFM. IEEE Trans. Pattern Anal. Mach. Intell. 26, 1051–1063 (2004)

    Article  Google Scholar 

  4. Costeira, J.P., Kanade, T.: A multibody factorization method for independently moving objects. Int. J. Comput. Vis. 3, 159–179 (1998)

    Article  Google Scholar 

  5. Fischler, M., Bolles, R.: Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography. Graph. Image Process. 24, 381–395 (1981)

    MathSciNet  Google Scholar 

  6. Golub, G.H., Van Loan, C.F. (eds.): Matrix Computations. Johns Hopkins Univ. Press, Baltimore (1989)

    MATH  Google Scholar 

  7. Guerreiro, R.F.C., Aguiar, P.M.Q.: Estimation of rank deficient matrices from partial observations: two-step iterative algorithms. In: EMMCVPR, pp. 450–466 (2003)

  8. Guilbert, N., Bartoli, A., Heyden, A.: Affine approximation for direct batch recovery of Euclidean structure and motion from sparse data. Int. J. Comput. Vis. 69, 317–333 (2006)

    Article  Google Scholar 

  9. Han, M., Kanade, T.: Reconstruction of a scene with multiple linearly moving objects. Int. J. Comput. Vis. 53, 285–300 (2000)

    Google Scholar 

  10. Harris, C., Stephens, M.: A combined corner and edge detector. In: Proceedings of the Fourth Alvey Vision Conference, pp. 147–151 (1988)

  11. Hartley, R., Schaffalitzky, F.: Powerfactorization: 3D reconstruction with missing or uncertain data. In: Australian-Japan Advanced Workshop on Computer Vision (2003)

  12. Hartley, R., Zisserman, A. (eds.): Multiple View Geometry in Computer Vision. Cambridge University Press. Book web page at: http://www.robots.ox.ac.uk/~vgg/hzbook/index.html, accessed February 2007 (2000)

  13. Huynh, D.Q., Hartley, R., Heyden, A.: Outlier correction in image sequences for the affine camera. In: IEEE ICCV, vol. 1, pp. 585–590 (2003)

  14. Huynh, D.Q., Heyden, A.: Outlier detection in video sequences under affine projection. In: IEEE CVPR, vol. 1, pp. 695–701 (2001)

  15. Jacobs, D.W.: Linear fitting with missing data for structure-from-motion. Comput. Vis. Image Underst. 1, 57–81 (2001)

    Article  MathSciNet  Google Scholar 

  16. Jia, H., Fortuna, J., Martinez, A.: Perturbation estimation of the subspaces for structure from motion with noisy and missing data. In: Third International Symposium on 3D Data Processing, Visualization, and Transmission, pp. 1101–1007 (2006)

  17. Julià, C., Sappa, A., Lumbreras, F., Serrat, J., López, A.: Factorization with missing and noisy data. In: Computational Science ICCS 2006: 6th International Conference. LNCS, vol. 1, pp. 555–562 (2006)

  18. Julià, C., Sappa, A., Lumbreras, F., Serrat, J., López, A.: An iterative multiresolution scheme for SFM. In: ICIAR 2006. LNCS, vol. 4141, pp. 804–815 (2006)

  19. Ma, Y., Soatto, J., Kosecká, J., Sastry, S.: An Invitation to 3D Vision: From Images to Geometric Models. Springer, New York (2004)

    Google Scholar 

  20. Martinec, D., Pajdla, T.: 3D reconstruction by fitting low-rank matrices with missing data. In: IEEE CVPR, vol. 1, pp. 198–205 (2005)

  21. Morita, T., Kanade, T.: A sequential factorization method for recovering shape and motion from image streams. IEEE Trans. Pattern Anal. Mach. Intell. 19(8), 858–867 (1997)

    Article  Google Scholar 

  22. Morris, D., Kanade, T.: A unified factorization algorithm for points, line segments and planes with uncertainty models. In: International Conference on Computer Vision, pp. 696–702 (1998)

  23. Okatani, T., Deguchi, K.: On the Wiberg algorithm for matrix factorization in the presence of missing components. Int. J. Comput. Vis. 72, 329–337 (2007)

    Article  Google Scholar 

  24. Poelman, C.J., Kanade, T.: A paraperspective factorization method for shape and motion recovery. IEEE Trans. Pattern Anal. Mach. Intell. 19, 206–218 (1997)

    Article  Google Scholar 

  25. Quan, L., Kanade, T.: A factorization method for affine structure from line correspondences. In: CVPR (1996)

  26. Tomasi, C., Kanade, T.: Shape and motion from image streams under orthography: a factorization method. Int. J. Comput. Vis. 9(2), 137–154 (1992)

    Article  Google Scholar 

  27. Triggs, B.: Linear projective reconstruction from matching tensors. Image Vis. Comput. 15, 617–625 (1997)

    Article  Google Scholar 

  28. Wiberg, T.: Computation of principal components when data is missing. In: Second Symposium of Computational Statistics, pp. 229–326 (1976)

Download references

Author information

Authors and Affiliations

  1. Intelligent Robotics and Computer Vision Group, Dept. of Computer Science and Mathematics, ETSE-URV, Av.Països Catalans 26, 43007, Tarragona, Spain

    Carme Julià

  2. Computer Vision Center, UAB Edifici O Campus UAB, 08193, Bellaterra, Barcelona, Spain

    Angel D. Sappa

  3. Computer Vision Center, Computer Science Department, UAB Edifici O Campus UAB, 08193, Bellaterra, Barcelona, Spain

    Felipe Lumbreras, Joan Serrat & Antonio López

Authors
  1. Carme Julià
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Angel D. Sappa
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Felipe Lumbreras
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Joan Serrat
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Antonio López
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Carme Julià.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Julià, C., Sappa, A.D., Lumbreras, F. et al. An Iterative Multiresolution Scheme for SFM with Missing Data. J Math Imaging Vis 34, 240–258 (2009). https://doi.org/10.1007/s10851-009-0144-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10851-009-0144-3

Keywords

Search

Navigation