Skip to main content
SpringerLink
Log in

Quantum Motion Segmentation

  • Conference paper
  • First Online:
Computer Vision – ECCV 2022 (ECCV 2022)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13689))

Included in the following conference series:

Abstract

Motion segmentation is a challenging problem that seeks to identify independent motions in two or several input images. This paper introduces the first algorithm for motion segmentation that relies on adiabatic quantum optimization of the objective function. The proposed method achieves on-par performance with the state of the art on problem instances which can be mapped to modern quantum annealers.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 89.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 119.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    See the project page https://4dqv.mpi-inf.mpg.de/QuMoSeg/.

  2. 2.

    For any matrices A, B of proper dimensions we have: \({{\,\textrm{trace}\,}}(A^{\textsf{T}}B) = {{\,\textrm{vec}\,}}(A)^{\textsf{T}}{{\,\textrm{vec}\,}}(B) \).

  3. 3.

    For any matrices A, B, Y of proper dimensions, the Kronecker product [38] satisfies: \( {{\,\textrm{vec}\,}}(AYB) = (B^{\textsf{T}}\otimes A) {{\,\textrm{vec}\,}}(Y) \).

  4. 4.

    Other measures can be considered with similar results, such as the misclassification error, which is widely adopted in motion segmentation.

  5. 5.

    At least four points are needed to estimate a homography, whereas at least seven points are required for the fundamental matrix [28].

  6. 6.

    Note that this reflects the current situation in the field: indeed, other quantum methods [10, 55] do not outperform classical methods in all scenarios too.

References

  1. https://alex-xun-xu.github.io/ProjectPage/CVPR_18/

  2. https://github.com/federica-arrigoni/ICCV_19

  3. Arrigoni, F., Fusiello, A.: Synchronization problems in computer vision with closed-form solutions. Int. J. Comput. Vision 128, 26–52 (2020)

    Article  MathSciNet  Google Scholar 

  4. Arrigoni, F., Pajdla, T.: Motion segmentation via synchronization. In: IEEE International Conference on Computer Vision Workshops (ICCVW) (2019)

    Google Scholar 

  5. Arrigoni, F., Pajdla, T.: Robust motion segmentation from pairwise matches. In: Proceedings of the International Conference on Computer Vision (2019)

    Google Scholar 

  6. Barath, D., Matas, J.: Multi-class model fitting by energy minimization and mode-seeking. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11220, pp. 229–245. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01270-0_14

    Chapter  Google Scholar 

  7. Bernard, F., Thunberg, J., Gemmar, P., Hertel, F., Husch, A., Goncalves, J.: A solution for multi-alignment by transformation synchronisation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2015)

    Google Scholar 

  8. Bernard, F., Thunberg, J., Swoboda, P., Theobalt, C.: HiPPI: higher-order projected power iterations for scalable multi-matching. In: Proceedings of the International Conference on Computer Vision (2019)

    Google Scholar 

  9. Birdal, T., Arbel, M., Simsekli, U., Guibas, L.J.: Synchronizing probability measures on rotations via optimal transport. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1566–1576 (2020)

    Google Scholar 

  10. Birdal, T., Golyanik, V., Theobalt, C., Guibas, L.: Quantum permutation synchronization. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2021)

    Google Scholar 

  11. Boothby, K., Bunyk, P., Raymond, J., Roy, A.: Next-Generation Topology of D-Wave Quantum Processors. arXiv e-prints 2003.00133 (2020)

  12. Born, M., Fock, V.: Beweis des adiabatensatzes. Z. Phys. 51(3), 165–180 (1928)

    Article  Google Scholar 

  13. Cai, J., Macready, W.G., Roy, A.: A practical heuristic for finding graph minors. arXiv e-prints 1406.2741 (2014)

  14. Carlone, L., Tron, R., Daniilidis, K., Dellaert, F.: Initialization techniques for 3D SLAM: a survey on rotation estimation and its use in pose graph optimization. In: Proceedings of the IEEE International Conference on Robotics and Automation (2015)

    Google Scholar 

  15. Cavallaro, G., Willsch, D., Willsch, M., Michielsen, K., Riedel, M.: Approaching remote sensing image classification with ensembles of support vector machines on the d-wave quantum annealer. In: IEEE International Geoscience and Remote Sensing Symposium (IGARSS) (2020)

    Google Scholar 

  16. Chatterjee, A., Govindu, V.M.: Efficient and robust large-scale rotation averaging. In: Proceedings of the International Conference on Computer Vision (2013)

    Google Scholar 

  17. Chen, Y., Guibas, L., Huang, Q.: Near-optimal joint object matching via convex relaxation. In: Proceedings of the International Conference on Machine Learning, pp. 100–108 (2014)

    Google Scholar 

  18. D-Wave Systems Inc: D-wave ocean software documentation (2021). https://docs.ocean.dwavesys.com/en/stable/. Accessed 05 Mar 2022

  19. D-Wave Systems Inc: dwave-neal documentation (2021). https://docs.ocean.dwavesys.com/_/downloads/neal/en/latest/pdf/. Accessed 6 Mar 2022

  20. Dattani, N., Szalay, S., Chancellor, N.: Pegasus: the second connectivity graph for large-scale quantum annealing hardware. arXiv e-prints (2019)

    Google Scholar 

  21. Denchev, V.S., Boixo, S., Isakov, S.V., Ding, N., Babbush, R., Smelyanskiy, V., Martinis, J., Neven, H.: What is the computational value of finite-range tunneling? Phys. Rev. X 6, 031015 (2016)

    Google Scholar 

  22. D-wave: What is quantum annealing? (2021). https://docs.dwavesys.com/docs/latest/c_gs_2.html. Accessed 05 Mar 2022

  23. Eriksson, A., Olsson, C., Kahl, F., Chin, T.J.: Rotation averaging and strong duality. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 127–135 (2018)

    Google Scholar 

  24. Farhi, E., Goldstone, J., Gutmann, S., Lapan, J., Lundgren, A., Preda, D.: A quantum adiabatic evolution algorithm applied to random instances of an np-complete problem. Science 292(5516), 472–475 (2001)

    Article  MathSciNet  Google Scholar 

  25. Golyanik, V., Theobalt, C.: A quantum computational approach to correspondence problems on point sets. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2020)

    Google Scholar 

  26. Govindu, V.M., Pooja, A.: On averaging multiview relations for 3D scan registration. IEEE Trans. Image Process. 23(3), 1289–1302 (2014)

    Article  MathSciNet  Google Scholar 

  27. Grant, E.K., Humble, T.S.: Adiabatic quantum computing and quantum annealing. In: Oxford Research Encyclopedia of Physics (2020)

    Google Scholar 

  28. Hartley, R., Zisserman, A.: Multiple View Geometry in Computer Vision, 2nd edn. Cambridge University Press, Cambridge (2004)

    Book  Google Scholar 

  29. Huang, J., et al.: MultiBodySync: multi-body segmentation and motion estimation via 3d scan synchronization. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7108–7118 (2021)

    Google Scholar 

  30. Ji, P., Li, H., Salzmann, M., Dai, Y.: Robust motion segmentation with unknown correspondences. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8694, pp. 204–219. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10599-4_14

    Chapter  Google Scholar 

  31. Ji, P., Salzmann, M., Li, H.: Shape interaction matrix revisited and robustified: efficient subspace clustering with corrupted and incomplete data. In: Proceedings of the International Conference on Computer Vision, pp. 4687–4695 (2015)

    Google Scholar 

  32. Kirkpatrick, S., Gelatt, C.D., Vecchi, M.P.: Optimization by simulated annealing. Science 220(4598), 671–680 (1983)

    Article  MathSciNet  Google Scholar 

  33. Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems, vol. 25 (2012)

    Google Scholar 

  34. Lai, T., Wang, H., Yan, Y., Chin, T.J., Zhao, W.L.: Motion segmentation via a sparsity constraint. IEEE Trans. Intell. Transp. Syst. 18(4), 973–983 (2017)

    Article  Google Scholar 

  35. Li, J., Ghosh, S.: Quantum-soft QUBO Suppression for accurate object detection. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12374, pp. 158–173. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58526-6_10

    Chapter  Google Scholar 

  36. Li, X., Ling, H.: PoGO-Net: pose graph optimization with graph neural networks. In: Proceedings of the International Conference on Computer Vision, pp. 5895–5905 (2021)

    Google Scholar 

  37. Li, Z., Guo, J., Cheong, L.F., Zhou, S.Z.: Perspective motion segmentation via collaborative clustering. In: Proceedings of the International Conference on Computer Vision, pp. 1369–1376 (2013)

    Google Scholar 

  38. Liu, S., Trenkler, G.: Hadamard, Khatri-Rao, Kronecker and other matrix products. Int. J. Inf. Syst. Sci. 4(1), 160–177 (2008)

    MathSciNet  MATH  Google Scholar 

  39. Magri, L., Fusiello, A.: T-Linkage: A continuous relaxation of J-Linkage for multi-model fitting. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp. 3954–3961 (June 2014)

    Google Scholar 

  40. Magri, L., Fusiello, A.: Multiple structure recovery via robust preference analysis. Image Vis. Comput. 67, 1–15 (2017)

    Article  Google Scholar 

  41. Maset, E., Arrigoni, F., Fusiello, A.: Practical and efficient multi-view matching. In: Proceedings of IEEE International Conference on Computer Vision, pp. 4568–4576 (2017)

    Google Scholar 

  42. McGeoch, C.C.: Adiabatic Quantum Computation and Quantum Annealing: Theory and Practice. Morgan & Claypool, Burlington (2014)

    Book  Google Scholar 

  43. Noormandipour, M., Wang, H.: Matching Point Sets with Quantum Circuit Learning. arXiv e-prints 2102.06697 (2021)

  44. O’Malley, D., Vesselinov, V.V., Alexandrov, B.S., Alexandrov, L.B.: Nonnegative/binary matrix factorization with a d-wave quantum anneale. PLoS ONE 13(12), e0206653 (2018)

    Article  Google Scholar 

  45. Ozden, K.E., Schindler, K., Van Gool, L.: Multibody structure-from-motion in practice. IEEE Trans. Pattern Anal. Mach. Intell. 32(6), 1134–1141 (2010)

    Article  Google Scholar 

  46. Ozyesil, O., Voroninski, V., Basri, R., Singer, A.: A survey of structure from motion. Acta Numer 26, 305–364 (2017)

    Article  MathSciNet  Google Scholar 

  47. Pachauri, D., Kondor, R., Singh, V.: Solving the multi-way matching problem by permutation synchronization. In: Advances in Neural Information Processing Systems, vol. 26, pp. 1860–1868 (2013)

    Google Scholar 

  48. Rao, S., Tron, R., Vidal, R., Ma, Y.: Motion segmentation in the presence of outlying, incomplete, or corrupted trajectories. Pattern Anal. Mach. Intell. 32(10), 1832–1845 (2010)

    Article  Google Scholar 

  49. Rosen, D.M., Carlone, L., Bandeira, A.S., Leonard, J.J.: SE-Sync: a certifiably correct algorithm for synchronization over the special Euclidean group. Int. J. Robot. Res. 38(2–3), 95–125 (2019)

    Article  Google Scholar 

  50. Sabzevari, R., Scaramuzza, D.: Multi-body motion estimation from monocular vehicle-mounted cameras. IEEE Trans. Rob. 32(3), 638–651 (2016)

    Article  Google Scholar 

  51. Santellani, E., Maset, E., Fusiello, A.: Seamless image mosaicking via synchronization. In: ISPRS Annals of Photogrammetry, Remote Sensing and Spatial Information Sciences, IV-2, pp. 247–254 (2018)

    Google Scholar 

  52. Saputra, M.R.U., Markham, A., Trigoni, N.: Visual SLAM and structure from motion in dynamic environments: a survey. ACM Comput. Surv. 51(2), 37:1–37:36 (2018)

    Google Scholar 

  53. Schroeder, P., Bartoli, A., Georgel, P., Navab, N.: Closed-form solutions to multiple-view homography estimation. In: IEEE Workshop on Applications of Computer Vision (WACV), pp. 650–657 (2011)

    Google Scholar 

  54. Seelbach Benkner, M., Golyanik, V., Theobalt, C., Moeller, M.: Adiabatic quantum graph matching with permutation matrix constraints. In: International Conference of 3D Vision (3DV) (2020)

    Google Scholar 

  55. Seelbach Benkner, M., Lähner, Z., Golyanik, V., Wunderlich, C., Theobalt, C., Moeller, M.: Q-match: iterative shape matching via quantum annealing. In: International Conference on Computer Vision (ICCV) (2021)

    Google Scholar 

  56. Singer, A.: Angular synchronization by eigenvectors and semidefinite programming. Appl. Comput. Harmon. Anal. 30(1), 20–36 (2011)

    Article  MathSciNet  Google Scholar 

  57. Thunberg, J., Bernard, F., Goncalves, J.: Distributed methods for synchronization of orthogonal matrices over graphs. Automatica 80, 243–252 (2017)

    Article  MathSciNet  Google Scholar 

  58. Tron, R., Daniilidis, K.: Statistical pose averaging with non-isotropic and incomplete relative measurements. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8693, pp. 804–819. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10602-1_52

    Chapter  Google Scholar 

  59. Tron, R., Vidal, R.: A benchmark for the comparison of 3-d motion segmentation algorithms. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–8. IEEE (2007)

    Google Scholar 

  60. Wang, Q., Zhou, X., Daniilidis, K.: Multi-image semantic matching by mining consistent features. In: Computer Vision and Pattern Recognition (CVPR) (2018)

    Google Scholar 

  61. Wang, Y., Liu, Y., Blasch, E., Ling, H.: Simultaneous trajectory association and clustering for motion segmentation. IEEE Signal Process. Lett. 25(1), 145–149 (2018)

    Article  Google Scholar 

  62. Xu, X., Cheong, L.F., Li, Z.: 3d rigid motion segmentation with mixed and unknown number of models. IEEE Trans. Pattern Anal. Mach. Intell. 43, 1–6 (2019)

    Article  Google Scholar 

  63. Yan, J., Pollefeys, M.: A general framework for motion segmentation: independent, articulated, rigid, non-rigid, degenerate and non-degenerate. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006. LNCS, vol. 3954, pp. 94–106. Springer, Heidelberg (2006). https://doi.org/10.1007/11744085_8

    Chapter  Google Scholar 

  64. Zhou, X., Zhu, M., Daniilidis, K.: Multi-image matching via fast alternating minimization. In: Proceedings of the International Conference on Computer Vision, pp. 4032–4040 (2015)

    Google Scholar 

Download references

Acknowledgements

This work was partially supported by the PRIN project LEGO-AI (Prot. 2020TA3K9N).

Author information

Authors and Affiliations

  1. Politecnico di Milano, Milan, Italy

    Federica Arrigoni

  2. University of Trento, Trento, Italy

    Willi Menapace & Elisa Ricci

  3. University of Siegen, Siegen, Germany

    Marcel Seelbach Benkner

  4. Bruno Kessler Foundation, Trento, Italy

    Elisa Ricci

  5. MPI for Informatics, Saarbrücken, Germany

    Vladislav Golyanik

Authors
  1. Federica Arrigoni
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Willi Menapace
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Marcel Seelbach Benkner
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Elisa Ricci
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Vladislav Golyanik
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Federica Arrigoni .

Editor information

Editors and Affiliations

  1. Tel Aviv University, Tel Aviv, Israel

    Shai Avidan

  2. University College London, London, UK

    Gabriel Brostow

  3. Google AI, Accra, Ghana

    Moustapha Cissé

  4. University of Catania, Catania, Italy

    Giovanni Maria Farinella

  5. Facebook (United States), Menlo Park, CA, USA

    Tal Hassner

1 Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (pdf 6669 KB)

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Arrigoni, F., Menapace, W., Benkner, M.S., Ricci, E., Golyanik, V. (2022). Quantum Motion Segmentation. In: Avidan, S., Brostow, G., Cissé, M., Farinella, G.M., Hassner, T. (eds) Computer Vision – ECCV 2022. ECCV 2022. Lecture Notes in Computer Science, vol 13689. Springer, Cham. https://doi.org/10.1007/978-3-031-19818-2_29

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-19818-2_29

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-19817-5

  • Online ISBN: 978-3-031-19818-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation