Skip to main content
Log in

Perspective-1-Ellipsoid: Formulation, Analysis and Solutions of the Camera Pose Estimation Problem from One Ellipse-Ellipsoid Correspondence

  • Published:
International Journal of Computer Vision Aims and scope Submit manuscript

Abstract

In computer vision, camera pose estimation from correspondences between 3D geometric entities and their projections into the image has been a widely investigated problem. Although most state-of-the-art methods exploit low-level primitives such as points or lines, the emergence of very effective CNN-based object detectors in the recent years has paved the way to the use of higher-level features carrying semantically meaningful information. Pioneering works in that direction have shown that modelling 3D objects by ellipsoids and 2D detections by ellipses offers a convenient manner to link 2D and 3D data. However, the mathematical formalism most often used in the related litterature does not enable to easily distinguish ellipsoids and ellipses from other quadrics and conics, leading to a loss of specificity potentially detrimental in some developments. Moreover, the linearization process of the projection equation creates an over-representation of the camera parameters, also possibly causing an efficiency loss. In this paper, we therefore introduce an ellipsoid-specific theoretical framework and demonstrate its beneficial properties in the context of pose estimation. More precisely, we first show that the proposed formalism enables to reduce the pose estimation problem to a position or orientation-only estimation problem in which the remaining unknowns can be derived in closed-form. Then, we demonstrate that it can be further reduced to a 1 Degree-of-Freedom (1DoF) problem and provide the analytical derivations of the pose as a function of that unique scalar unknown. We illustrate our theoretical considerations by visual examples and include a discussion on the practical aspects. Finally, we release this paper along with the corresponding source code in order to contribute towards more efficient resolutions of ellipsoid-related pose estimation problems. The source code is available here: https://gitlab.inria.fr/vgaudill/p1e.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

Notes

  1. This experiment can be reproduced using our code (file main.m, experiment Xp=4)

  2. https://members.loria.fr/moberger/Documents/P1E.mp4

References

  • Avron, H., Ng, E., & Toledo, S. (2008). A generalized courant-fischer minimax theorem. https://escholarship.org/uc/item/4gb4t762.

  • Bonin-Font, F., Ortiz, A., & Oliver, G. (2008). Visual navigation for mobile robots: A survey. Journal of Intelligent and Robotic Systems, 53(3), 263–296.

    Article  Google Scholar 

  • Crocco, M., Rubino, C., & Del Bue, A. (2016). Structure from motion with objects. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

  • Dong, W., & Isler, V. (2021). Ellipse regression with predicted uncertainties for accurate multi-view 3d object estimation. CoRR arXiv:2101.05212.

  • Dong, W., Roy, P., Peng, C., & Isler, V. (2021). Ellipse R-CNN: Learning to infer elliptical object from clustering and occlusion. IEEE Transactions on Image Processing, 30, 2193–2206.

    Article  Google Scholar 

  • Eberly, D. (2007). Reconstructing an ellipsoid from its perspective projection onto a plane. https://www.geometrictools.com/. Updated version: March 1, 2008.

  • Fischler, M. A., & Bolles, R. C. (1981). Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography. Communications of the ACM, 24(6), 381–395.

    Article  MathSciNet  Google Scholar 

  • Garg, S., Sünderhauf, N., Dayoub, F., Morrison, D., Cosgun, A., Carneiro, G., Wu, Q., Chin, T. J., Reid, I., Gould, S., Corke, P., & Milford, M. (2020). Semantics for robotic mapping, perception and interaction: A survey. Foundations and Trends in Robotics, 8(1–2), 1–224.

    Article  Google Scholar 

  • Gaudillière, V., Simon, G., & Berger, M. O. (2019a). November. Camera Pose Estimation with Semantic 3D Model. In IROS 2019 - 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems, Macau, Macau SAR China.

  • Gaudillière, V., Simon, G., & Berger, M. O. (2019b). Camera Relocalization with Ellipsoidal Abstraction of Objects. In ISMAR 2019 - 18th IEEE International Symposium on Mixed and Augmented Reality, Beijing, China.

  • Gaudillière, V., Simon, G., & Berger, M. O. (2019c). Perspective-12-Quadric: An analytical solution to the camera pose estimation problem from conic - quadric correspondences.

  • Gaudillière, V., Simon, G. & Berger, M. O. (2020a). Perspective-2-Ellipsoid: Bridging the Gap Between Object Detections and 6-DoF Camera Pose. In IROS 2020 – 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems, Las Vegas, United States.

  • Gaudillière, V., Simon, G., & Berger, M. O. (2020). Perspective-2-Ellipsoid: Bridging the gap between object detections and 6-DoF camera pose. IEEE Robotics and Automation Letters, 5189–5196.

  • Gay, P., Rubino, C., Bansal, V., & Del Bue, A. (2017). Probabilistic structure from motion with objects (psfmo). In Proceedings of the IEEE international conference on computer vision (ICCV).

  • Girshick, R. (2015). Fast r-cnn. In Proceedings of the IEEE international conference on computer vision (ICCV).

  • Girshick, R., Donahue, J., Darrell, T., & Malik, J. (2014). Rich feature hierarchies for accurate object detection and semantic segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR).

  • Golub, G. H., & Van Loan, C. F. (1996). Matrix computations (3rd ed.). The Johns Hopkins University Press.

    MATH  Google Scholar 

  • Hartley, R. I., & Zisserman, A. (2004). Multiple view geometry in computer vision (2nd ed.). Cambridge University Press.

    Book  MATH  Google Scholar 

  • Hoque, S., Arafat, M. Y., Xu, S., Maiti, A., & Wei, Y. (2021). A comprehensive review on 3d object detection and 6d pose estimation with deep learning. IEEE Access, 9, 143746–143770.

    Article  Google Scholar 

  • Kisantal, M., Sharma, S., Park, T. H., Izzo, D., Märtens, M., & D’Amico, S. (2020). Satellite pose estimation challenge: Dataset, competition design, and results. IEEE Transactions on Aerospace and Electronic Systems, 56(5), 4083–4098.

    Article  Google Scholar 

  • Lang, S. (2002). Algebra. Graduate Texts in Mathematics, Vol. 211 (Revised third ed.).

  • Lepetit, V., Moreno-Noguer, F., & Fua, P. (2009). Epnp: An accurate O(n) solution to the pnp problem. International Journal of Computer Vision, 81(2), 155–166.

    Article  Google Scholar 

  • Li, J., Meger, D., & Dudek, G. (2017). Context-coherent scenes of objects for camera pose estimation. In 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2017, IEEE, Vancouver, BC, Canada, September 24–28, 2017 (pp. 655–660).

  • Li, J., Meger, D., & Dudek, G. (2019). Semantic mapping for view-invariant relocalization. In International Conference on Robotics and Automation, ICRA 2019, IEEE, Montreal, QC, Canada, May 20–24, 2019 (pp. 7108–7115).

  • Li, J., Xu, Z., Meger, D., & Dudek, G. (2018). Semantic scene models for visual localization under large viewpoint changes. In 15th Conference on Computer and Robot Vision (CRV), Toronto, Canada.

  • Li, Y. (2019). Detecting lesion bounding ellipses with gaussian proposal networks. In Machine Learning in Medical Imaging - 10th International Workshop, MLMI 2019, Held in Conjunction with MICCAI 2019, Shenzhen, China, October 13, 2019, Proceedings, Volume 11861 of Lecture Notes in Computer Science (pp. 337–344). Springer.

  • Liao, Z., Wang, W., Qi, X., Zhang, X., Xue, L., Jiao, J. & Wei, R. (2020). Object-oriented SLAM using quadrics and symmetry properties for indoor environments. Corr arxiv:2004.05303

  • Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S. E., Fu, C., & Berg, A. C. (2016). SSD: single shot multibox detector. In Computer Vision - ECCV 2016 - 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part I, Volume 9905 of Lecture Notes in Computer Science (pp. 21–37). Springer.

  • Marchand, É., Uchiyama, H., & Spindler, F. (2016). Pose estimation for augmented reality: A hands-on survey. IEEE Transactions on Visualization and Computer Graphics, 22(12), 2633–2651.

    Article  Google Scholar 

  • Nicholson, L., Milford, M., & Sünderhauf, N. (2019). Quadricslam: Dual quadrics from object detections as landmarks in object-oriented slam. IEEE Robotics and Automation Letters, 4(1), 1–8.

    Article  Google Scholar 

  • Pan, S., Fan, S., Wong, S. W. K., Zidek, J. V. & Rhodin, H. (2021). Ellipse detection and localization with applications to knots in sawn lumber images. In IEEE Winter Conference on Applications of Computer Vision, IEEE, WACV 2021, Waikoloa, HI, USA, January 3–8, 2021 (pp. 3891–3900).

  • Park, T. H., Märtens, M., Jawaid, M., Wang, Z., Chen, B., Chin, T. J., Izzo, D., & D’Amico, S. (2023). Satellite pose estimation competition 2021: Results and analyses. Acta Astronautica. https://doi.org/10.1016/j.actaastro.2023.01.002

    Article  Google Scholar 

  • Rathinam, A., Gaudillière, V., Pauly, L., & Aouada, D. (2022). Pose estimation of a known texture-less space target using convolutional neural networks. In 73rd International Astronautical Congress (IAC), Paris, France, 18-22 September 2022.

  • Redmon, J., Divvala, S., Girshick, R., & Farhadi, A. (2016). You only look once: Unified, real-time object detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

  • Redmon, J., & Farhadi, A. (2017). Yolo9000: Better, faster, stronger. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

  • Redmon, J., & Farhadi, A. (2018). Yolov3: An incremental improvement. arXiv:1804.02767.

  • Rubino, C., Crocco, M., & Bue, A. D. (2018). 3d object localisation from multi-view image detections. IEEE Transactions on Pattern Analysis and Machine Intelligence, 40(6), 1281–1294.

    Google Scholar 

  • Sturm, J., Engelhard, N., Endres, F., Burgard, W., & Cremers, D. (2012). A benchmark for the evaluation of RGB-D SLAM systems. In 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2012, IEEE, Vilamoura, Algarve, Portugal, October 7–12, 2012 (pp. 573–580).

  • Tian, R., Zhang, Y., Feng, Y., Yang, L., Cao, Z., Coleman, S., & Kerr, D. (2021). Accurate and robust object-oriented SLAM with 3d quadric landmark construction in outdoor environment. CoRR arXiv:2110.08977.

  • Wokes, D. S., & Palmer, P. L. (2010). Perspective reconstruction of a spheroid from an image plane ellipse. International Journal of Computer Vision, 90(3), 369–379.

    Article  MathSciNet  MATH  Google Scholar 

  • Wylie, C. R. (2008). Introduction to projective geometry. Dover Publications.

    MATH  Google Scholar 

  • Xu, C., Zhang, L., Cheng, L., & Koch, R. (2017). Pose estimation from line correspondences: A complete analysis and a series of solutions. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(6), 1209–1222.

    Article  Google Scholar 

  • Zhao, M., Jia, X., Fan, L., Liang, Y., & Yan, D. (2021). Robust ellipse fitting using hierarchical gaussian mixture models. IEEE Transactions on Image Processing, 30, 3828–3843.

    Article  Google Scholar 

  • Zins, M., Simon, G., & Berger, M. O. (2020). 3d-aware ellipse prediction for object-based camera pose estimation. In 8th International Conference on 3D Vision, 3DV 2020, IEEE, Virtual Event, Japan, November 25–28, 2020 (pp. 281–290).

Download references

Acknowledgements

The work presented in this paper was carried out at Université de Lorraine, CNRS, Inria, LORIA. The writing effort was partly funded by the Luxembourg National Research Fund (FNR) under the project reference BRIDGES2020/IS/14755859/MEET-A/Aouada.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Vincent Gaudillière.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Communicated by Tomas Pajdla.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendices

Appendix A: Equivalent Problem Formulations

To prove that Eqs. (1) and (1’) are equivalent, we demonstrate below that (1) implies (1’) then (1’) implies (1).

\(\boxed {(1) \implies (1')}\) Multiply (1) on the right by \({\varvec{\Delta }}\) to obtain \(A{\varvec{\Delta }}=\sigma B'{\varvec{\Delta }}\) (see Appendix B). Whence

$$\begin{aligned} A{\varvec{\Delta }}{\varvec{\Delta }}^\top A+\mu A = \sigma B' \implies \sigma B'-\sigma B'{\varvec{\Delta }}{\varvec{\Delta }}^\top A = \mu A \end{aligned}$$

Left-multiplying by \(B'^{-1}\)

$$\begin{aligned} \sigma I-\sigma {\varvec{\Delta }}{\varvec{\Delta }}^\top A = \mu B'^{-1}A \end{aligned}$$

Then right-multiplying by \(A^{-1}\)

$$\begin{aligned} \sigma A^{-1}-\sigma {\varvec{\Delta }}{\varvec{\Delta }}^\top = \mu B'^{-1} \end{aligned}$$

Finally (\(\sigma \ne 0\))

$$\begin{aligned} A^{-1}-{\varvec{\Delta }}{\varvec{\Delta }}^\top = \frac{\mu }{\sigma } B'^{-1} \end{aligned}$$

\(\boxed {(1') \implies (1)}\) Multiply (1’) on the right by A then on the left by \(B'\) to obtain

$$\begin{aligned} B'-B'{\varvec{\Delta }}{\varvec{\Delta }}^\top A = \frac{\mu }{\sigma } A \end{aligned}$$

Then right-multiply by \({\varvec{\Delta }}\) to obtain \(\mu B'{\varvec{\Delta }}=\frac{\mu }{\sigma } A{\varvec{\Delta }}\), whence (\(\mu \ne 0\)) \(A{\varvec{\Delta }}=\sigma B'{\varvec{\Delta }}\).

Injecting that result into he previous equation leads to

$$\begin{aligned} \sigma B'-A{\varvec{\Delta }}{\varvec{\Delta }}^\top A = \mu A \end{aligned}$$

Appendix B: Proof of Result 2

Let us first prove that the couple {A,B’} has exactly two distinct generalized values, that are non zero and of opposite signs.

Proof

The generalized eigenvalues of {A,B’} are non-zero because \(B'^{-1}A\) is not singular.

We can then observe that

$$\begin{aligned} Q(x)=\mu x^2-(\mu +1)\sigma x+\sigma ^2 \end{aligned}$$

is an annihilator polynomial of \(B'^{-1}A\) (see proof in Appendix C):

$$\begin{aligned} \mu (B'^{-1}A)^2-(\mu +1)\sigma B'^{-1}A+\sigma ^2 I={\textit{0}}. \end{aligned}$$
(B30)

In linear algebra, the minimal polynomial \(\pi (.)\) is defined as the monic annihilator polynomial having the lowest possible degree. It can be shown (Lang, 2002) that (i) \(\pi (.)\) divides any annihilator polynomial and (ii) the roots of \(\pi (.)\) are identical to the roots of the characteristic polynomial. Since Q is an annihilator polynomial of degree 2, we can thus infer that \(B'^{-1}A\), and thus \(\{A,B'\}\), has at most two distinct eigenvalues.

We are now going to prove by contradiction that {A,B’} has exactly two distinct eigenvalues. Let’s thus assume that the couple has only one eigenvalue with multiplicity 3 denoted \(\sigma _0\).

Since A is positive definite and \(B'\) is symmetric, the couple {A,B’} has the following properties (Golub and Van Loan, 1996) (Corollary 8.7.2, p. 462):

  1. 1.

    their generalized eigenvalues are real,

  2. 2.

    their reducing subspaces are of the same dimension as the multiplicity of the associated eigenvalues,

  3. 3.

    their generalized eigenvectors form a basis of \({\mathbb {R}}^3\), and those with distinct eigenvalues are A-orthogonal.

According to property 2. above, we have

$$\begin{aligned} dim(Ker(A-\sigma _0B'))=3\text{, } \end{aligned}$$

i.e.

$$\begin{aligned} A=\sigma _0B'\text{, } \end{aligned}$$

which is impossible because A represents an ellipsoid whereas \(B'\) represents a cone. So \(\varvec{\{A,B'\}}\) has exactly two distinct generalized eigenvalues.

Let’s then denote \(\sigma _1\) (multiplicity 1) and \(\sigma _2\) (multiplicity 2) these two eigenvalues. Observing that \(\frac{1}{\sigma _1}\) and \(\frac{1}{\sigma _2}\) are the generalized eigenvalues of \(\{B',A\}\), we can write, according to Avron et al. (2008) (Theorem 3)

$$\begin{aligned} \begin{aligned} \forall \textbf{X}&\in {\mathbb {R}}^{3}\backslash \mathbf{\{0\}},\\&min\left( \frac{1}{\sigma _1},\frac{1}{\sigma _2}\right) \le \frac{\textbf{X}^\top B'\textbf{X}}{\textbf{X}^\top A\textbf{X}}\le max\left( \frac{1}{\sigma _1},\frac{1}{\sigma _2}\right) \end{aligned} \end{aligned}$$

If \(\sigma _1\) and \(\sigma _2\) were of the same sign, then \(\forall \textbf{X}\in {\mathbb {R}}^3\backslash \mathbf{\{0\}}\), \(\textbf{X}^\top B'\textbf{X}\) would be of that sign (since \(\textbf{X}^\top A\textbf{X}>0\)). Yet, it is impossible since \(B'\) is neither positive nor negative definite (cone). We thus conclude that the two distinct eigenvalues are of opposite signs. \(\square \)

Let us now prove that \(\sigma \) is the generalized eigenvalue of {A,B’} with multiplicity 1

Proof

Let’s consider \((\sigma _1,{\varvec{\delta }}_1)\) and \((\sigma _2,{\varvec{\delta }}_2)\) the generalized eigenvalues and eigenvectors of \(\{A,B'\}\), such that \(\Vert {\varvec{\delta }}_i\Vert =1\).

We are going to prove (3) by contradiction.

Let’s suppose that there is \(k\in {\mathbb {R}}^*\) such that \((A,\sigma _2,k{\varvec{\delta }}_2)\) is solution of Equation (1).

By injecting these values into (1), we therefore have

$$\begin{aligned} B'-\frac{1}{\sigma _2}A=MA\text{, } \end{aligned}$$

where

$$\begin{aligned} M=\frac{k^2}{\sigma _2}(A{\varvec{\delta }}_2{\varvec{\delta }}_2^\top -{\varvec{\delta }}_2^\top A{\varvec{\delta }}_2I)\text{. } \end{aligned}$$

According to property 2 of the proof of Result 2,

$$\begin{aligned} dim\left( Ker\left( B'-\frac{1}{\sigma _2}A\right) \right) =2\text{, } \end{aligned}$$

whence, since A is invertible,

$$\begin{aligned} dim(Ker(MA))=dim(Ker(M))=2\text{. } \end{aligned}$$

However, defining

$$\begin{aligned} {\varvec{\delta }}_2^{\perp }=\{\textbf{X}\in {\mathbb {R}}^3 / \textbf{X}\perp {\varvec{\delta }}_2\} \end{aligned}$$

the subspace of dimension 2 orthogonal to \({\varvec{\delta }}_2\), we observe that, \(\forall \textbf{X}\in {\varvec{\delta }}_2^{\perp }\),

$$\begin{aligned} M\textbf{X}&= \frac{k^2}{\sigma _2}A{\varvec{\delta }}_2{\varvec{\delta }}_2^\top \textbf{X}-\frac{k^2}{\sigma _2}{\varvec{\delta }}_2^\top A{\varvec{\delta }}_2\textbf{X}\\&= \frac{k^2}{\sigma _2}A{\varvec{\delta }}_2({\varvec{\delta }}_2\cdot \textbf{X})-\frac{k^2}{\sigma _2}{\varvec{\delta }}_2^\top A{\varvec{\delta }}_2\textbf{X}\\&= -\frac{k^2}{\sigma _2}{\varvec{\delta }}_2^\top A{\varvec{\delta }}_2 \textbf{X}\text{. } \end{aligned}$$

Since A is positive definite, \({\varvec{\delta }}_2^\top A{\varvec{\delta }}_2>0\), whence it comes

$$\begin{aligned} \forall \textbf{X}\in {\varvec{\delta }}_2^{\perp }\backslash {{\mathbf{\{0\}}}}\text{, } M\textbf{X}\ne {\textbf{0}}\text{. } \end{aligned}$$

It means that

$$\begin{aligned} {\varvec{\delta }}_2^{\perp }\cap Ker(M)=\{{\textbf{0}}\}\text{, } \end{aligned}$$

whence the direct sum

$$\begin{aligned} {\varvec{\delta }}_2^{\perp }\bigoplus Ker(M) \end{aligned}$$

is a subspace of \({\mathbb {R}}^3\) with dimension

$$\begin{aligned} dim({\varvec{\delta }}_2^{\perp }) + dim(Ker(M)) = 2+2=4\text{. } \end{aligned}$$

We end up with a contradiction since \(4>dim({\mathbb {R}}^3)=3\).

As a result, triplets \((A,\sigma _2,k{\varvec{\delta }}_2)\) cannot be solutions of (1), thus solutions are necessarily in the form \((A,\sigma _1,k{\varvec{\delta }}_1)\), where \(k\in {\mathbb {R}}^*\). \(\square \)

Appendix C: Proving that \(Q(B'^{-1}A)=0\)

Replacing (2) into (1), we obtain:

$$\begin{aligned} \sigma ^2B'{\varvec{\Delta }}{\varvec{\Delta }}^\top B'-(\sigma {\varvec{\Delta }}^\top B'{\varvec{\Delta }}-1)A=\sigma B' \end{aligned}$$

We can then deduce the following expression for A:

$$\begin{aligned} A=\frac{\sigma }{1-\sigma {\varvec{\Delta }}^\top B'{\varvec{\Delta }}}(B'-\sigma B'{\varvec{\Delta }}{\varvec{\Delta }}^\top B') \end{aligned}$$

Whence, denoting I the identity matrix and defining \(f=\frac{\sigma }{1-\sigma {\varvec{\Delta }}^\top B'{\varvec{\Delta }}}\), then left-multiplying by \(B'^{-1}\), we obtain

$$\begin{aligned} B'^{-1}A&= f(I-\sigma {\varvec{\Delta }}{\varvec{\Delta }}^\top B') \end{aligned}$$

Squaring that expression leads to

$$\begin{aligned} (B'^{-1}A)^2&= f^2(I-\sigma {\varvec{\Delta }}{\varvec{\Delta }}^\top B')^2\\&= f^2(I-2\sigma {\varvec{\Delta }}{\varvec{\Delta }}^\top B'+\sigma ^2{\varvec{\Delta }}({\varvec{\Delta }}^\top B'{\varvec{\Delta }}){\varvec{\Delta }}^\top B')\\&= f^2(I-2\sigma {\varvec{\Delta }}{\varvec{\Delta }}^\top B'+\sigma ^2({\varvec{\Delta }}^\top B'{\varvec{\Delta }}){\varvec{\Delta }}{\varvec{\Delta }}^\top B')\\&= f^2(I-\sigma (2-\sigma {\varvec{\Delta }}^\top B'{\varvec{\Delta }}){\varvec{\Delta }}{\varvec{\Delta }}^\top B') \end{aligned}$$

Defining \(\mu =1-\sigma {\varvec{\Delta }}^\top B'{\varvec{\Delta }}=1-{\varvec{\Delta }}^\top A{\varvec{\Delta }}\):

$$\begin{aligned} (B'^{-1}A)^2&= f^2(I-\sigma (\mu +1){\varvec{\Delta }}{\varvec{\Delta }}^\top B')\\&= f^2((\mu +1)(I-\sigma {\varvec{\Delta }}{\varvec{\Delta }}^\top B')-\mu I)\\&= f(\mu +1)B'^{-1}A-f^2\mu I\\&= \frac{\sigma }{\mu }(\mu +1)B'^{-1}A-\frac{\sigma ^2}{\mu }I \end{aligned}$$

Finally, we have

$$\begin{aligned} \mu (B'^{-1}A)^2 = \sigma (\mu +1)B'^{-1}A-\sigma ^2I \end{aligned}$$

Whence, denoting \(Q(x)=\mu x^2-(\mu +1)\sigma x+\sigma ^2\),

$$\begin{aligned} Q(B'^{-1}A)=0 \end{aligned}$$

Appendix D: Characterizations of \(\mu \)

Proof of result 3

The trace of \(B'^{-1}A\) is given by its eigenvalues:

$$\begin{aligned} tr(B'^{-1}A)=\sigma _1+2\sigma _2\text{. } \end{aligned}$$

Whence, by squaring the matrix,

$$\begin{aligned} tr((B'^{-1}A)^2)=\sigma _1^2+2\sigma _2^2\text{. } \end{aligned}$$

Therefore, since \(tr(I)=3\) and \(\sigma =\sigma _1\), applying the operator tr() to Eq. (B30) leads to

$$\begin{aligned} \mu (\sigma _1^2+2\sigma _2^2)-\sigma _1(\mu +1)(\sigma _1+2\sigma _2)+3\sigma _1^2 =0\text{, } \end{aligned}$$

which is equivalent to

$$\begin{aligned} \mu \sigma _2(\sigma _2-\sigma _1)=\sigma _1(\sigma _2-\sigma _1)\text{, } \end{aligned}$$

i.e.

$$\begin{aligned} \mu =\frac{\sigma _1}{\sigma _2}\text{. } \end{aligned}$$

\(\square \)

Proof of result 4

Determinant of \(B'^{-1}A\) is given by its eigenvalues:

$$\begin{aligned} det(B'^{-1}A)=\sigma _1\sigma _2^2\text{, } \end{aligned}$$

i.e.

$$\begin{aligned} \frac{det(A)}{det(B')}=\sigma _1\sigma _2^2\text{. } \end{aligned}$$
(D31)

We obtain (5) by injecting (3) (4) into (D31) and using \(\mu <0\).

\(\square \)

Appendix E: Proof of Co-occurences

Obviously, only circular cones can be tangent to a sphere. Furthermore, we are going to prove that only a non-circular elliptic cone can be tangent to a triaxial ellipsoid.

Let’s prove it by contradiction, and assume that the projection cone has a revolution axis (circular cone).

Let’s also assume that the ellipsoid center \(\textbf{C}\) does not belong to that axis. Since the ellipsoid is tangent to the cone, any new ellipsoid obtained by rotating the original one around the cone revolution axis shall still be tangent to the cone, thus be solution of (1). Yet, in this case, the locus of ellipsoid centers would be a circle located in a plane orthogonal to that axis and whose center would belong to it. Every center would thus be at a fixed distance to the cone vertex \(\textbf{E}\), whence there would be an infinite number of \({\varvec{\Delta }}\) solutions for the same \(\sigma \), given (6). However, this contradicts Eq. (18). Therefore, the ellipsoid center must belong to the revolution axis of the cone.

If the ellipsoid center belonged to the cone revolution axis, then \({\varvec{\Delta }}\) would be parallel to that axis, i.e. would be an eigenvector of \(B'\), whence an eigenvector of A given (1’). However, in such a case, the symmetries of the cone-ellipsoid pair would impose that the tangent ellipse (intersection between the ellipsoid and the polar plane derived from \(\textbf{E}\) (Wylie, 2008)) belongs to a plane orthogonal to the cone revolution axis, that is also a principal axis of the ellipsoid. Therefore, that tangent ellipse should be both a circle (orthogonal section of a circular cone) and a non-circular ellipse (section of an ellipsoid by a plane parallel to one of its principal planes), which is impossible. Therefore, the cone cannot have a revolution axis.

Appendix F: Vandermonde Matrix

A Vandermonde matrix is a matrix with the terms of a geometric progression in each row or column:

$$\begin{aligned} V=\begin{pmatrix} 1 &{} 1 &{} 1 &{} \cdots &{} 1 \\ x_1 &{} x_2 &{} x_3 &{} \cdots &{} x_n \\ x_1^2 &{} x_2^2 &{} x_3^2 &{} \cdots &{} x_n^2 \\ \vdots &{} \vdots &{} \vdots &{} \ddots &{} \vdots \\ x_1^{m-1} &{} x_2^{m-1} &{} x_3^{m-1} &{} \cdots &{} x_n^{m-1} \end{pmatrix}\text{. } \end{aligned}$$

The determinant of a square Vandermonde matrix (when \(m=n\)) is given by

$$\begin{aligned} det(V)=\prod _{1\le i<j\le n}(x_j-x_i)\text{. } \end{aligned}$$

Therefore, V is not singular (i.e. \(det(V)\ne 0\)) if and only if all \(x_i\) are distinct.

Appendix G: Demonstration of Result  10

Proof

Given that A has two distinct eigenvalues, its minimal polynomial is

$$\begin{aligned} \pi _A(x)=(x-\lambda _{A,single})(x-\lambda _{A,double})\text{. } \end{aligned}$$

\(\pi _A\) being an annihilator polynomial of A, evaluating \(\pi _A(A)\) gives

$$\begin{aligned} A^2=\alpha A+\beta I\text{, } \end{aligned}$$

where

$$\begin{aligned} \alpha&=\lambda _{A,single}+\lambda _{A,double}\text{, }\\ \beta&=-\lambda _{A,single}\lambda _{A,double}\text{. } \end{aligned}$$

Left-multiplying by \({\varvec{\Delta }}^\top \) and right-multiplying by \({\varvec{\Delta }}\) gives

$$\begin{aligned} {\varvec{\Delta }}^\top A^2{\varvec{\Delta }}=\alpha {\varvec{\Delta }}^\top A{\varvec{\Delta }}+\beta {\varvec{\Delta }}^\top {\varvec{\Delta }}\text{. } \end{aligned}$$
(G32)

Injecting the expressions of \({\varvec{\Delta }}^\top A^2{\varvec{\Delta }}\), \({\varvec{\Delta }}^\top A{\varvec{\Delta }}\) and \({\varvec{\Delta }}^\top {\varvec{\Delta }}\) as functions of m from (17) into (G32) and observing that

$$\begin{aligned} tr(A)=\lambda _{A,single}+2\lambda _{A,double} \end{aligned}$$

and that

$$\begin{aligned} tr(A^{-1})=\frac{1}{\lambda _{A,single}} +\frac{2}{\lambda _{A,double}}\text{, } \end{aligned}$$

we obtain, after simplification

$$\begin{aligned} m^3=\frac{tr(B')d}{\lambda _{A,double}}m^2+\frac{\lambda _{A,single} tr(B'^{-1})}{d}m-\frac{\lambda _{A,single}}{\lambda _{A,double}}\text{. } \end{aligned}$$

Let’s call \(P_{spheroid}(x)\) the above polynomial whose m is root:

$$\begin{aligned}&P_{spheroid}(x)\\&\quad =x^3-\frac{tr(B')d}{\lambda _{A,double}}x^2 -\frac{\lambda _{A,single}tr(B'^{-1})}{d}x+\frac{\lambda _{A,single}}{\lambda _{A,double}}\text{. } \end{aligned}$$

Developing \(tr(B')\) and \(tr(B'^{-1})\), one can observe that it can be rewritten

$$\begin{aligned} P_{spheroid}(x) = \left( x-\frac{\lambda _{B',single}}{\lambda _{A,double}}d\right) \left( x-\frac{\lambda _{B',double}}{\lambda _{A,double}}d\right) ^2\text{. } \end{aligned}$$

At this stage, one can note that the sign of d is the sign of \(\lambda _{B',simple}\):

$$\begin{aligned} d=\root 3 \of {\frac{det(A)}{det(B')}}=\root 3 \of {\frac{\lambda _{A,single} \lambda _{A,double}^2}{\lambda _{B',single}\lambda _{B',double}^2}}\text{. } \end{aligned}$$

Thus the signs of the roots of \(P_{spheroid}(x)\) are:

$$\begin{aligned} \frac{\lambda _{B',single}}{\lambda _{A,double}}d>0 \text{ and } \frac{\lambda _{B',double}}{\lambda _{A,double}}d<0\text{. } \end{aligned}$$

Since \(m<0\), the only possible value for it is the second one. Therefore, \(\sigma \) is given by:

$$\begin{aligned} \sigma&=d\left( \frac{\lambda _{B',double}}{\lambda _{A,double}}d\right) ^2\\&=\frac{\lambda _{A,single}}{\lambda _{B',single}}\text{. } \end{aligned}$$

Let’s now focus on \({\varvec{\Delta }}\), and consider the first two equations of System 17:

$$\begin{aligned} \left\{ \begin{array}{l} {\varvec{\Delta }}^\top {\varvec{\Delta }}= tr(A^{-1})-\frac{tr(B'^{-1})}{d}m\\ {\varvec{\Delta }}^\top A{\varvec{\Delta }}= 1-m^3 \end{array} \right. \end{aligned}$$

Considering the ellipsoid frame, its left hand side can be rewritten in a matrix form:

$$\begin{aligned} \begin{pmatrix} 1 &{} 1 \\ \lambda _{A,single} &{} \lambda _{A,double} \end{pmatrix} \begin{pmatrix} {\varvec{\Delta }}_{ell,x}^2 \\ {\varvec{\Delta }}_{ell,y}^2+{\varvec{\Delta }}_{ell,z}^2 \end{pmatrix} \end{aligned}$$

That Vandermonde matrix is not singular given that \( \lambda _{A,single}\ne \lambda _{A,double}\), thus the system can be inverted.

After developing the right hand side, we finally obtain

$$\begin{aligned} {\varvec{\Delta }}_{ell,x}^2 = \frac{1}{\lambda _{A,single}}\left( 1-\frac{\lambda _{A,single} \lambda _{B',double}}{\lambda _{A,double}\lambda _{B',single}}\right) \text{, } \end{aligned}$$

and

$$\begin{aligned} {\varvec{\Delta }}_{ell,y}^2+{\varvec{\Delta }}_{ell,z}^2 = 0\text{. } \end{aligned}$$

Therefore,

$$\begin{aligned} {\varvec{\Delta }}_{ell}= \begin{pmatrix} \pm \sqrt{\frac{1}{\lambda _{A,single}}\left( 1-\frac{\lambda _{A,single} \lambda _{B',double}}{\lambda _{A,double}\lambda _{B',simple}}\right) } \\ 0 \\ 0 \end{pmatrix}\text{. } \end{aligned}$$
(G33)

\({\varvec{\Delta }}\) is thus an eigenvector of A corresponding to eigenvalue \(\lambda _{A,single}\), i.e. coincides with the revolution axis of the spheroid.

Table 3 The different types of ellipsoids
Table 4 The different types of elliptic cones

Equation (2) then requires that \({\varvec{\Delta }}\) is also an eigenvector of \(B'\). It must be the one corresponding to the revolution axis of the cone since, if not, the ellipsoid center would be located outside of the cone. In that respect, both axes of revolutions (cone and spheroid) coincide, and \(\Vert {\varvec{\Delta }}\Vert \) is given by:

$$\begin{aligned} \Vert {\varvec{\Delta }}\Vert =\sqrt{\frac{1}{\lambda _{A,single}}\left( 1-\frac{\lambda _{A,simple} \lambda _{B',double}}{\lambda _{A,double}\lambda _{B',single}}\right) }\text{. } \end{aligned}$$

\(\square \)

Appendix H: Ellipsoid and Cone Types

See Tables 3 and 4.

Appendix I: Theorem 1: Maple code

figure n

Appendix J: Theorem 8: Maple code

figure o

Appendix K: Solving the Polynomial Equation in Theorem 3

In case #1, the signs of \(B'\) eigenvalues ensure that

$$\begin{aligned} d=\root 3 \of {\frac{\lambda _{A,single}\lambda _{A,double}^2}{\lambda _{B',1} \lambda _{B',2}\lambda _{B',3}}}>0\text{, } \end{aligned}$$

then that

$$\begin{aligned} k_1<0 \text{, } k_2>0 \text{ and } k_3<0\text{. } \end{aligned}$$

Let’s denote \(S_i\) the root of \(P_i(x)\) with multiplicity 1, and \(D_i\) the root with multiplicity 2, such that

$$\begin{aligned} S_i=\frac{\lambda _{B',i}}{\lambda _{A,single}}d \quad \text{ and } \quad D_i=\frac{\lambda _{B',i}}{\lambda _{A,double}}d. \end{aligned}$$

Considering the signs of \(\lambda _{B',i}\) and the fact that \(\lambda _{A,simple}<\lambda _{A,double}\), it comes

$$\begin{aligned} S_1&<D_1<0\text{, }\\ S_2&<D_2<0\text{, }\\ 0&<D_3<S_3\text{. } \end{aligned}$$

One can therefore note that the roots of \(P_1(x)\) and \(P_2(x)\) are negative, while those of \(P_3(x)\) are positive. Since, in addition, \(k_3<0\), we have

$$\begin{aligned} \forall x\le 0\text{, } P_3(x)>0. \end{aligned}$$

Let’s now focus on the signs of \(P_1(x)\) and \(P_2(x)\) to determine the locus of possible m values. Since \(\lambda _{B',1}<\lambda _{B',2}\), their roots verify

$$\begin{aligned} S_1&<S_2\text{, }\\ D_1&<D_2\text{. } \end{aligned}$$

Therefore, one can distinguish between two configurations regaring the roots order:

$$\begin{aligned} S_1<D_1<S_2<D_2\text{, } \end{aligned}$$

or

$$\begin{aligned} S_1<S_2<D_1<D_2\text{. } \end{aligned}$$

For the first case, the variations of the three polynomials are presented in Table 5.

One can observe that the three polynomials are never all non-negative, thus this configuration is impossible. For the second case, however, there is one value for which all three polynomials are non-negative: \(D_1\) (see Table 6).

Appendix L: Proof of Result 12

Proof

Left-multiplying \(A=\lambda _{A,triple}I\) by \({\varvec{\Delta }}^\top \) right-multiplying it by \({\varvec{\Delta }}\), we obtain

$$\begin{aligned} {\varvec{\Delta }}^\top A{\varvec{\Delta }}=\lambda _{A,triple}{\varvec{\Delta }}^\top {\varvec{\Delta }}\text{. } \end{aligned}$$
Table 5 Signs of \(P_i(x)\) when \(S_1<D_1<S_2<D_2\)
Table 6 Signs of \(P_i(x)\) when \(S_1<S_2<D_1<D_2\)

Injecting the first two equations of System (17), we then have

$$\begin{aligned} 1-m^3=\lambda _{A,triple}\left( tr(A^{-1})-\frac{tr(B'^{-1})}{d}m\right) \text{. } \end{aligned}$$

Developing \(tr(A^{-1})\) and \(tr(B'^{-1})\) leads to

$$\begin{aligned} 1-m^3=3-\frac{\lambda _{A,triple}}{d}\frac{\lambda _{B',double} +2\lambda _{B',simple}}{\lambda _{B',simple}\lambda _{B',double}}m \end{aligned}$$

Furthermore, one can observe that

$$\begin{aligned} d=\root 3 \of {\frac{det(A)}{det(B')}}=\frac{\lambda _{A,triple}}{\lambda _{B',simple}^{1/3}\lambda _{B',double}^{2/3}}\text{. } \end{aligned}$$

Whence, injecting this into the former equation:

$$\begin{aligned} 0&=m^3-\frac{\lambda _{B',double}+2\lambda _{B',simple}}{\lambda _{B',simple }^{2/3}\lambda _{B',double}^{1/3}}m+2\\&=m^3-\left( \frac{\lambda _{B',double}^{2/3}}{\lambda _{B',simple}^{2/3}} +2\frac{\lambda _{B',simple}^{1/3}}{\lambda _{B',double}^{1/3}}\right) m+2 \end{aligned}$$

Denoting

$$\begin{aligned} R=\root 3 \of {\frac{\lambda _{B',double}}{\lambda _{B',simple}}}\text{, } \end{aligned}$$

the last equation means that m is root of the polynomial

$$\begin{aligned} P_{sphere}(x)=x^3-\left( R^2+\frac{2}{R}\right) x+2\text{. } \end{aligned}$$

Yet, R is an obvious root of this polynomial:

$$\begin{aligned} P_{sphere}(x)=\left( x-R\right) \left( x^2+Rx-\frac{2}{R}\right) \end{aligned}$$

Even if obtaining a formal expression of the two other roots is not straighforward, Vieta’s formulas provide the following constraints:

$$\begin{aligned} \left\{ \begin{array}{l} x_1+x_2+R=0\text{, }\\ x_1x_2R=-2\text{. } \end{array} \right. \end{aligned}$$

If roots \(x_1\) are \(x_2\) complex, then R is the only possible value for m. If they are real, then, since \(R<0\) (\(B'\) eigenvalues are of opposite signs), the second formula requires that \(x_1\) and \(x_2\) are of the same sign, and the first formula requires that they are positive. Finally,

$$\begin{aligned} m=R\text{. } \end{aligned}$$

Corresponding \(\sigma \) value is:

$$\begin{aligned} \sigma&=dR^2\\&=\frac{\lambda _{A,triple}}{\lambda _{B',single}^{1/3}\lambda _{B',double}^{2/3}}\frac{\lambda _{B',double}^{2/3}}{\lambda _{B',single}^{2/3}}\\&=\frac{\lambda _{A,triple}}{\lambda _{B',single}}\text{. } \end{aligned}$$

Applying tr() to Eq. (1’) gives the value of \(\Vert {\varvec{\Delta }}\Vert ^2\):

$$\begin{aligned} \Vert {\varvec{\Delta }}\Vert ^2=\frac{3}{\lambda _{A,triple}}-\frac{\mu }{\sigma } \left( \frac{1}{\lambda _{B',single}}+\frac{2}{\lambda _{B',double}}\right) \text{. } \end{aligned}$$

Given

$$\begin{aligned} \sigma =\frac{\lambda _{A,triple}}{\lambda _{B',simple}}\text{, } \end{aligned}$$

and

$$\begin{aligned} \mu =m^3=\frac{\lambda _{B',double}}{\lambda _{B',simple}}\text{, } \end{aligned}$$

the right hand side can be rewritten

$$\begin{aligned} \frac{3}{\lambda _{A,triple}}-\frac{\lambda _{B',double}}{\lambda _{A,triple}}\left( \frac{1}{\lambda _{B',simple}} +\frac{2}{\lambda _{B',double}}\right) \text{, } \end{aligned}$$

i.e.

$$\begin{aligned} \Vert {\varvec{\Delta }}\Vert ^2&=\frac{3}{\lambda _{A,triple}}-\frac{\lambda _{B',double}}{\lambda _{B',simple}}\frac{1}{\lambda _{A,triple}}-\frac{2}{\lambda _{A,triple}}\\&=\left( 1-\frac{\lambda _{B',double}}{\lambda _{B',simple}}\right) \frac{1}{\lambda _{A,triple}}\text{. } \end{aligned}$$

\(\square \)

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Gaudillière, V., Simon, G. & Berger, MO. Perspective-1-Ellipsoid: Formulation, Analysis and Solutions of the Camera Pose Estimation Problem from One Ellipse-Ellipsoid Correspondence. Int J Comput Vis 131, 2446–2470 (2023). https://doi.org/10.1007/s11263-023-01794-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11263-023-01794-x

Keywords

Navigation