Skip to main content
SpringerLink
Log in

Structure from Motion with Line Segments Under Relaxed Endpoint Constraints

  • Published:
International Journal of Computer Vision Aims and scope Submit manuscript

Abstract

We present a novel structure from motion pipeline, which estimates motion and wiry 3D structure from imaged line segments across multiple views. Although the position and orientation of line segments can be determined more accurately than point features, the instability of their endpoints and the fact that lines are not constrained by epipolar geometry diverted most research focus away to point-based methods. In our approach, we tackle the problem of instable endpoints by utilizing relaxed constraints on their positions, both during matching and as well in the following bundle adjustment stage. Furthermore, we gain efficiency in estimating trifocal image relations by decoupling rotation and translation. To this end, a novel linear solver for relative translation estimation given rotations from five line correspondences in three views is introduced. Extensive experiments on long image sequences show that our line-based structure from motion pipeline advantageously complements point-based methods, giving more meaningful 3D representation for indoor scenarios.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

Notes

  1. Recall that fisheye images possessing field-of-view larger than 180\(^\circ \) cannot be undistorted and warped into a perspective image with straight lines.

References

  • Agarwal, S., Mierle, K. et al. (2010). Ceres Solver. http://www.ceres-solver.org.

  • Agarwal, S., Furukawa, Y., Snavely, N., Simon, I., Curless, B., Seitz, M. S., et al. (2011). Building Rome in a day. Commununications of ACM, 54(10), 105–112.

    Article  Google Scholar 

  • Ansar, A., & Daniilidis, K. (2002). Linear pose estimation from points or lines. In Proceedings of the European Conference on Computer Vision (ECCV).

  • Antone, M. E., & Teller, S. J. (2000). Automatic recovery of relative camera rotations for urban scenes. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 2282–2289).

  • Barrow, H. G., Tenenbaum, J. M., Bolles, R. C., & Wolf, H. C. (1977). Parametric correspondence and chamfer matching: Two new techniques for image matching. In Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI) (pp. 659–663).

  • Bartoli, A., & Sturm, P. (2005). Structure-from-motion using lines: Representation, triangulation, and bundle adjustment. Computer Vision and Image Understanding (CVIU), 100(3), 416–441.

    Article  Google Scholar 

  • Bay, H., Ferrari, V., & Van Gool, L. (2005). Wide-baseline stereo matching with line segments. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 329–336).

  • Bazin, J. C., & Pollefeys, M. (2012). 3-line ransac for orthogonal vanishing point detection. In Proceedings of the International Conference on Intelligent Robots and Systems (IROS) (pp. 4282–4287).

  • Bazin, J. C., Demonceaux, C., Vasseur, P., & Kweon, I. (2010). Motion estimation by decoupling rotation and translation in catadioptric vision. Computer Vision and Image Understanding (CVIU), 114(2), 254–273.

    Article  Google Scholar 

  • Burns, J. B., Hanson, A. R., & Riseman, E. M. (1986). Extracting straight lines. IEEE Transactions on Pattern Analysis and Machine Intelligence (T-PAMI), 8(4), 425–455.

    Article  Google Scholar 

  • Chen, H. (1991). Pose determination from line-to-plane correspondences: Existence condition and closed-form solutions. IEEE Transactions on Pattern Analysis and Machine Intelligence (T-PAMI), 13(6), 530–541.

    Article  Google Scholar 

  • Davison, A. J., Reid, I. D., Molton, N., & Stasse, O. (2007). Monoslam: Real-time single camera SLAM. IEEE Trans Pattern Anal Mach Intell, 29(6), 1052–1067.

    Article  Google Scholar 

  • Deriche, R., & Faugeras, O. (1990). Tracking line segments. In Proceedings of the European Conference on Computer Vision (ECCV).

  • Elqursh, A., & Elgammal, A. M. (2011). Line-based relative pose estimation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

  • Fan, B., Wu, F., & Hu, Z. (2010). Line matching leveraged by point correspondences. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 390–397).

  • Faugeras, O., Lustman, F., & Toscani, G. (1987). Motion and structure from motion from point and line matches. In Proceedings of the International Conference on Computer Vision (ICCV).

  • Geyer, C., & Daniilidis, K. (2001). Catadioptric projective geometry. International Journal of Computer Vision (IJCV), 45(3), 223–243.

    Article  MATH  Google Scholar 

  • Hartley, R. (1997). Lines and points in three views and the trifocal tensor. International Journal of Computer Vision (IJCV), 22(2), 125–140.

    Article  Google Scholar 

  • Hartley, R., & Zisserman, A. (2000). Multiple view geometry in computer vision. Cambridge: Cambridge University Press.

    MATH  Google Scholar 

  • Hartley, R. I., Trumpf, J., Dai, Y., & Li, H. (2013). Rotation averaging. International Journal of Computer Vision (IJCV), 103(3), 267–305.

    Article  MathSciNet  MATH  Google Scholar 

  • Hirose, K., & Saito, H. (2012). Fast line description for line-based SLAM. In Proceedings of the Britisch Machine Vision Conference (BMVC).

  • Hofer, M., Wendel, A., & Bischof, H. (2013). Incremental line-based 3D reconstruction using geometric constraints. In Proceedings of the Britisch Machine Vision Conference (BMVC).

  • Huynh, D. Q. (2009). Metrics for 3d rotations: Comparison and analysis. Journal of Mathematical Imaging and Vision, 35(2), 155–164.

    Article  MathSciNet  Google Scholar 

  • Jain, A., Kurz, C., Thormählen, T., & Seidel, H.P. (2010). Exploiting global connectivity constraints for reconstruction of 3d line segments from images. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

  • Jiang, N., Cui, Z., & Tan, P. (2013). A global linear method for camera pose registration. In Proceedings of the International Conference on Computer Vision (ICCV).

  • Liu, Y., & Huang, T. S. (1988a). Estimation of rigid body motion using straight line correspondences. Computer Vision, Graphics, and Image Processing (CVGIP), 43(1), 35–52.

    Article  Google Scholar 

  • Liu, Y., & Huang, T. S. (1988b). A linear algorithm for motion estimation using straight line correspondences. Computer Vision, Graphics, and Image Processing (CVGIP), 44(1), 35–57.

    Article  Google Scholar 

  • Lowe, D. G. (2004). Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision (IJCV), 60(2), 91–110.

    Article  Google Scholar 

  • Martinec, D., & Pajdla, T. (2003). Line reconstruction from many perspective images by factorization. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

  • Micusik, B., & Pajdla, T. (2006). Structure from motion with wide circular field of view cameras. IEEE Transactions on Pattern Analysis and Machine Intelligence (T-PAMI), 28(7), 1135–1149.

    Article  Google Scholar 

  • Micusik, B., & Wildenauer, H. (2015). Descriptor free visual indoor localization with line segments. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

  • Morris, D. D., & Kanade, T. (1998). A unified factorization algorithm for points, line segments and planes with uncertainty models. In Proceedings of the International Conference on Computer Vision (ICCV) (pp. 696–702).

  • Rother, C. (2003). Multi-view reconstruction and camera recovery using a real or virtual reference plane. PhD thesis, KTH.

  • Schindler, G., Krishnamurthy, P., & Dellaert, F. (2006). Line-based structure from motion for urban environments. In Proceedings of the International Symposium on 3D Data Processing, Visualization and Transmission (3DPVT).

  • Sinha, S. N., Steedly, D., & Szeliski, R. (2009). Piecewise planar stereo for image-based rendering. In Proceedings of the International Conference on Computer Vision (ICCV) (pp .1881–1888).

  • Sinha, S. N., Steedly, D., & Szeliski, R. (2010). A multi-stage linear approach to structure from motion. In: Proceedings of the ECCV Workshop—Trends and Topics in Computer Vision (pp. 267–281).

  • Smith, P., Reid, I., & Davison, A. (2006). Real-time monocular SLAM with straight lines. In Proceedings of the Britisch Machine Vision Conference (BMVC).

  • Spetsakis, M. E. (1992). A linear algorithm for point and line-based structure from motion. Computer Vision, Graphics, and Image Processing: Image Understanding (CVGIPIU), 56(2), 230–241.

    MATH  Google Scholar 

  • Strecha, C., von Hansen, W., Van Gool, L., Fua, P., & Thoennessen, U. (2008). On benchmarking camera calibration and multi-view stereo for high resolution imagery. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

  • Subbarao, R., & Meer, P. (2009). Nonlinear mean shift over Riemannian manifolds. International Journal of Computer Vision (IJCV), 84(1), 1–20.

    Article  Google Scholar 

  • Taylor, J. C., & Kriegman, J. D. (1995). Structure and motion from line segments in multiple images. IEEE Transactions on Pattern Analysis and Machine Intelligence (T-PAMI), 17(11), 1021–1032.

    Article  Google Scholar 

  • Verhagen, B., Timofte, R., & Gool, L. J. V. (2014). Scale-invariant line descriptors for wide baseline matching. In: IEEE Winter Conference on Applications of Computer Vision, Steamboat Springs, CO, USA, March 24–26, (pp. 493–500).

  • VISCODA (2012). Voodoo camera tracker.

  • Wang, L., Neumann, U., & You, S. (2009). Wide-baseline image matching using line signatures. In Proceedings of the International Conference on Computer Vision (ICCV).

  • Weng, J., Huang, T., & Ahuja, N. (1992). Motion and structure from line correspondences; closed-form solution, uniqueness, and optimization. IEEE Transactions on Pattern Analysis and Machine Intelligence (T-PAMI), 14(3), 318–336.

    Article  Google Scholar 

  • Weng, J., Huang, T., & Ahuja, N. (1993). Motion and structure from image sequences. Springer series in information sciences. Berlin: Springer.

    Book  Google Scholar 

  • Werner, T., & Zisserman, A. (2002). New techniques for automated architectural reconstruction from photographs. In Proceedings of the European Conference on Computer Vision (ECCV).

  • Wildenauer, H., & Hanbury, A. (2012). Robust camera self-calibration from monocular images of manhattan worlds. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 2831–2838).

  • Wu, C. (2011). VisualSFM: A visual structure from motion system.

  • Zhang, L., & Koch, R. (2014). Structure and motion from line correspondences: Representation, projection, initialization and sparse bundle adjustment. Elsevier Journal of Visual Communication and Image Representation (JVCIR), 25(5), 904–915.

    Article  Google Scholar 

  • Zhang, Z. (1995). Estimating motion and structure from correspondences of line segments between two perspective images. IEEE Transactions on Pattern Analysis and Machine Intelligence (T-PAMI), 17(12), 1129–1139.

    Article  Google Scholar 

  • Zhang, Z. (1997). Parameter estimation techniques: A tutorial with application to conic fitting. Image and Vision Computing (IVC), 15(1), 59–76.

    Article  Google Scholar 

Download references

Acknowledgements

This research received funding from the Austrian Research Promotion Agencys (FFG) Projects LOLOG 840168, LARAH 4586620 and PAMON 835916.

Author information

Authors and Affiliations

  1. Digital Safety and Security Department, AIT Austrian Institute of Technology, Donau-City-Straße 1, 1220, Vienna, Austria

    Branislav Micusik

  2. Zeno Track GmbH, Barichgasse 40-42, 1030, Vienna, Austria

    Horst Wildenauer

Authors
  1. Branislav Micusik
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Horst Wildenauer
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Branislav Micusik.

Additional information

Communicated by K. Ikeuchi.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Micusik, B., Wildenauer, H. Structure from Motion with Line Segments Under Relaxed Endpoint Constraints. Int J Comput Vis 124, 65–79 (2017). https://doi.org/10.1007/s11263-016-0971-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11263-016-0971-9

Keywords

Search

Navigation