Critical configurations for two projective views, a new approach

This article develops new techniques to classify critical configurations for 3D scene reconstruction from images taken by unknown cameras. Generally, all information can be uniquely recovered if enough images and image points are provided, but there are certain cases where unique recovery is impossible; these are called critical configurations . In this paper, we use an algebraic approach to study the critical configurations for two projective cameras. We show that all critical configurations lie on quadric surfaces, and classify exactly which quadrics constitute a critical configuration. The paper also describes the relation between the different reconstructions when unique reconstruction is impossible


Introduction
In computer vision, one of the main problems is that of structure from motion, where given a set of 2-dimensional images the task is to reconstruct a scene in 3-space and find the camera positions in this scene. Over time, many algorithms have been developed for solving these problems for varying camera models and under different assumptions on the space and image points. In general, with enough images and enough points in each image, one can uniquely recover all information about the original scene. However, there are also some configuration of points and images where a unique recovery is never possible. These were described as far back as 1941 by Krames [5], where he showed that for two projective cameras, any configuration for which unique reconstruction is impossible, needs to lie on a ruled quadric surface. Krames referred to these configurations "Gefährliche Örter", nowadays called critical configurations.
The result by Krames comes with a partial converse, listing some (but not all) of the critical configurations for two cameras. Hartley and Kahl give a classification of all critical configurations for two calibrated cameras in [4] and for two and three projective cameras in [2]. The latter paper claims to give a classification for any number of projective cameras, but for four or more views several of the critical configurations described turn out to not be critical at all. We plan to resolve this, and correct these important results by Hartley and Kahl in future work, which will build heavily on the methods introduced in this paper.
In this paper, we will be studying the critical configurations for up to two projective cameras from the point-of-view of algebraic geometry. The key results of this paper are the same as what was found in [2]. This paper does, however, contain some new results on the one-view critical configurations and on relations between critical configurations and their conjugates (Sections 3 and 5) which do not appear in [2]. It also provides proofs to many of the assertions that are stated in [2], but not proven. Furthermore, even though the main results are the same, the techniques for obtaining them are new. These techniques provide insight as to how tools from algebraic geometry can be used to study critical configurations, as it relies less on computations and ad hoc examples, and more on the general properties of the varieties in question. We also plan to use these new techniques to classify critical configurations for any number of views, as well as using them in other, more complicated scenarios in future work.
In Section 2, we introduce the main concepts, such as cameras and critical configurations, as well as some general results on critical configurations. Section 3 gives a brief summary of the one-view case before we get to the the two-view case in Section 4. Here, we describe the multi-view variety for two views. We also introduce the fundamental form and use this to give a complete classification of the critical configurations for two views. Finally, Section 5 describes the relation between a critical configuration and the alternative reconstructions of the images it produces.

Background
Let C denote the complex numbers, and let P n denote the projective space over the vector space C n+1 . The projection from a point p ∈ P 3 is a linear map P : P 3 P 2 .
We will refer to such a projection and its projection center p as a camera and its camera center. Following this theme, we will sometimes refer to points in P 3 as space points and points in P 2 as image points. Similarly, P 2 will sometimes be referred to as an image.
Once a frame of coordinates is chosen in P 3 and P 2 , a camera can be represented by a 3 × 4 matrix of full rank, called the camera matrix. The camera center is then given as the kernel of the matrix. For the most part, we will make no distinction between a camera and its camera matrix, referring to both simply as cameras. We will be using the real projective pinhole camera model, meaning that we require the camera matrix to be of full rank and to have only real entries.
Since the map P is not defined at the camera center p, it is not a morphism. This problem can be easily mended by taking the blow-up. Let P 3 be the blow-up of P 3 in the camera center of P . We then get the following diagram: where π denotes the blow-down of P 3 . This gives us a morphism from P 3 to P 2 . For ease of notation, we will retain the symbol P and the names camera and camera center, although one should note that in P 3 the camera center is no longer a point, but an exceptional divisor.

Remark:
We use the blow-up for the sake of mathematical rigor, but a reader unfamiliar with the concept is free to think of P 3 simply as P 3 . Indeed, since many of our arguments will be geometric in nature, it might be wise even for those familiar with blow-ups to think of P 3 for the sake of intuition.
Definition 2.1. Given an n-tuple of cameras P = (P 1 , . . . , P n ), with camera centers p 1 , . . . , p n , let P 3 denote the blow-up of P 3 in the n camera centers. We define the joint camera map to be the map This again gives us a commutative diagram The reason we use the blow up P 3 rather P 3 is to turn the cameras (and hence the joint camera map) into morphisms rather than rational maps. This ensures that the image of the joint camera map is Zariski closed, turning it into a projective variety.

Definition 2.2.
We denote the image of the joint camera map φ P as the multi view variety of the cameras P 1 , . . . , P n . The ideal of this variety will be referred to as the multi-view ideal.
Notation: While the multi-view variety is always irreducible, we will be using the term variety to also include reducible algebraic sets.

Definition 2.3.
Given a set of points S ⊂ (P 2 ) n , a reconstruction of S is an n-tuple of cameras P = (P 1 , . . . , P n ) and a set of points X ⊂ P 3 such that S = φ P (X) where φ P is the joint camera map of the cameras P 1 , . . . , P n . Definition 2.4. Given a configuration of cameras and points (P 1 , . . . , P n , X), we will refer to φ P (X) ⊂ (P 2 ) n as the images of (P 1 , . . . , P n , X).
Now, assume we are given a set of image points S ⊂ (P 2 ) n as well as a reconstruction (P 1 , . . . , P n , X) of S. Note that any scaling, rotation, translation, or more generally, any projective transformation of (P 1 , . . . , P n , X) will not change the images, hence providing us with a large family of reconstructions of S. However, we are not interested in differentiating between these reconstructions. Definition 2.5. Given a set of points S ⊂ (P 2 ) n , let (P, X) and (Q, Y ) be two reconstructions of S, and let X and Y denote the blow-downs of X and Y respectively. The two reconstructions are considered equivalent if there exists an element A ∈ PGL(4), such that where p i and q i denotes the camera centers.
From now on, whenever we talk about a configuration of cameras and points, it is to be understood as unique up to such an isomorphism/action of PGL(4), and two configurations will be considered different only if they are not isomorphic/do not lie in the same orbit under this group action. As such, we will consider a reconstruction to be unique if it is unique up to action by PGL(4).
Definition 2.6. Given a configuration of cameras and points (P 1 , . . . , P n , X), a conjugate configuration is a configuration (Q 1 , . . . , Q n , Y ), nonequivalent to the first, such that φ P (X) = φ Q (Y ). Given a conjugate configuration, each point Definition 2.7. A configuration of cameras (P 1 , . . . , P n ) and points X ⊂ P 3 is said to be a critical configuration if it has at least one conjugate configuration. A critical configuration (P 1 , . . . , P n , X) is said to be maximal if there exists no critical configuration (P 1 , . . . , P n , X ) such that X X .
Hence, a configuration is critical if and only if the images it produces do not have a unique reconstruction.
Definition 2.8. Let P and Q be two n-tuples of cameras, let P 3 P and P 3 Q denote the blow up of P 3 in the camera centers of P and Q respectively. Define The projection of I to each coordinate gives us two varieties, X and Y , which we denote as the set of critical points of P (resp. Q).
This definition is motivated by the following fact: Proposition 2.9. Let P and Q be two (different) n-tuples of cameras, and let X and Y be their respective sets of critical points. Then (P, X) is a maximal critical configuration, with (Q, Y ) as its conjugate.
Proof. It is clear by Definition 2.8 that for each point x ∈ X, we will have a conjugate point y ∈ Y . It follows that the two configurations have the same images, so they are both critical configurations, conjugate to one another.
The maximality follows from the fact that if we add a point x 0 to X that does not lie in the set of critical points, there will (by Definition 2.8) not be any point y 0 ∈ P 3 Q such that φ P (x 0 ) = φ Q (y 0 ). Hence (P, X) will no longer be critical.
Our goal in this paper is to classify all maximal critical configurations for two cameras. The reason we focus primarily on the maximal ones is that every critical configuration is contained in a maximal one and in general the converse is true as well, that any subconfiguration of a critical configuration is also critical. The latter is not always true though, since removing some points from a configuration might put the two (formerly different) conjugates into the same orbit under PGL(4).
We wrap this section of with a final, useful property of critical configurations, namely that the only property of the cameras we need to consider when exploring critical configurations is the position of their camera centers.
Proof. If seen in a coordinate free setting without camera matrices, this statement is immediate since a projection from a point is unique up to a choice of coordinates in the domain and in the image. For a proof in the setting with camera matrices where coordinates are chosen, see the proof of Proposition 3.7 in [2].

The one-view case
Reconstruction of a 3D-scene from the image of one projective camera is generally considered impossible, so most papers start with the two-view case. Still, for the sake of completeness, we will be giving a brief summary of the critical configurations for one camera.
Let P be a camera, and let p be its camera center, we then have the joint camera map: For any point x ∈ P 2 , the fiber over x is a line through p, so no point can be uniquely recovered. From this, one might assume that every configuration with one camera is critical. However, this is only the case if our configuration consists of sufficiently many points.

Theorem 3.1. A configuration of one point and one camera is never critical. A configuration of one camera and n > 1 points is critical if and only if the camera center along with the n points span a space of dimension less than n.
Proof. For the first part, note that up to a projective transformation, there exists only one configuration of one point and one camera. In other words, any configuration of one point and one camera can be taken to any other such configuration by simply changing coordinates. By Definition 2.5 this makes them equivalent, which means that only one reconstruction exists.
The same turns out to be the case if the configuration is such that the camera center along with the n > 1 points span a space of dimension n. In P 3 , one can never span a space of dimension greater than 3, so this implies that n ≤ 3. Furthermore, if the n points along with the camera center span a space of dimension n, the points and camera center must lie in general position. However, for n ≤ 4 points (fewer than 3 points + one camera center) there exists only one configuration (up to action with PGL (4)) where all points are in general position. This means (by Definition 2.5) that all reconstructions are equivalent.
Now it only remains to show that a configuration is critical if the camera center along with the n > 1 points span a space of dimension less than n. Indeed, if the points along with the camera center span a space of dimension less than n, the image points will span a space of dimension m < n − 1. Then there will always be at least two nonequivalent reconstructions; one reconstruction where the points span a space of dimension m not containing the camera center, and one where they span a space of dimension m + 1 which contains the camera center.

The two-view case 4.1 The multi-view ideal
We start the study of the case of two cameras P 1 and P 2 by understanding their multi-view variety. We assume, here and throughout the rest of the paper, that all cameras have distinct centers. The two cameras define the joint camera map: Proposition 4.1. For two cameras (P 1 , P 2 ), the joint camera map φ P contracts the line spanned by the two camera centers, and is an embedding everywhere else.
Proof. For x = (x 1 , x 2 ) ∈ P 2 × P 2 , the preimage of x is given by where l i is the line P −1 i (x 1 ). The line l i passes through the camera center p i , so the intersection of the two lines is a single point unless they are both equal to the line spanned by the camera centers.
This means that the multi-view variety Im( φ P ) is an irreducible singular 3-fold in P 2 × P 2 . It is described by a single bilinear polynomial F P , which we call the fundamental form of P 1 , P 2 .

The fundamental form
The fundamental form is well studied in the literature, and is often represented by a 3 × 3 matrix of rank 2 called the fundamental matrix. See [3, section 9.2] for a geometric construction of the fundamental matrix. We will instead be using the construction in [1] where the fundamental form is given as the determinant of a 6 × 6 matrix: The fundamental form of P 1 , P 2 is given by 3 and y 1 , y 2 , y 3 are the variables in the first and second image respectively.
Proof. The fact that this polynomial describes the multi-view ideal for two views follows directly from Theorem 3.7 in [1].

Remark:
Recall that the entries in the camera matrix are real, this means that the fundamental form will always have real coefficients.

Definition 4.3. The epipoles e j
Pi are the image points we get by mapping the j-th camera center to the i-th image Note that the fundamental form satisfies in other words, it vanishes in either epipole. This means that the fundamental form is of rank 2 (also follows from the fact that Im( φ P ) is singular), so for each pair of cameras we get a bilinear form of rank 2. The following result states that the converse is also true, i.e. that any bilinear form of rank 2 is the fundamental form for some pair of cameras.

Theorem 4.4.
There is a 1 : 1 correspondence between bilinear forms of rank two, and pairs of cameras P 1 , P 2 (up to action by PGL (4)) Proof. Follows from [3, Theorem 9.10].
With these results, we can move on to classifying all the critical configurations for two views. We start with a special type of critical configuration:

Trivial critical configurations
Definition 4.5. A configuration (P 1 , P 2 , X) is said to be a non-trivial critical configuration if it has a conjugate configuration (Q 1 , Q 2 , Y ) satisfying Critical configurations not satisfying this property exist, they are called trivial. If (P 1 , P 2 , X) is a trivial critical configuration, then all its conjugates (Q 1 , Q 2 , Y ) have the same fundamental form as the cameras P 1 , P 2 . By Theorem 4.4 this means that, after a change of coordinates, Q 1 = P 1 and Q 2 = P 2 . Since the cameras are the same, Proposition 4.1 tells us that the sets X and Y must also be equal, with the exception of any point lying on the line spanned by the two camera centers. It is a well known fact that no number of cameras can differentiate between points lying on a line containing all the camera centers, hence the name "trivial".
For the remainder of the paper, we will focus on non-trivial critical configurations, so any critical configuration is understood to satisfy the criterion in Definition 4.5.
A classification of the trivial critical configurations for any number of views can be found in [2] Section 4, or in [3] Chapter 22.

Critical configurations for two views
Let us consider a critical configuration (P 1 , P 2 , X). Since it is critical, there must exist a conjugate configuration (Q 1 , Q 2 , Y ) giving the same images in P 2 × P 2 .
The two sets of cameras define two joint-camera maps φ P and φ Q .
Now, since the configuration is critical, we have that φ P (X) = φ Q (Y ). As such, the two sets X and Y must lie in such a way that they both map (with their respective maps) into the intersection of the two multi-view varieties Im( φ P ) ∩ Im( φ Q ).
From Proposition 4.1, we know that the two multi-view varieties are irreducible 3-folds, so their intersection is a surface in P 2 × P 2 . Taking the preimage under φ P , we get a surface S in P 3 which needs to contain all the points in X. Moreover, if (P 1 , P 2 , X) is maximal, we must have X = S. As such, classifying all maximal critical configurations can be done by classifying all possible intersections between two multi-view varieties, and then examining what these intersections pull back to in P 3 . This is made even simpler by the fact that the pullback of Im( φ P ) ∩ Im( φ P ) is just the variety we get by pulling back the fundamental form F Q (the fundamental form F P pulls back to the zero polynomial).
The pullback of a bilinear form is a quadratic polynomial, which describes a quadric surface. It follows that the blow-downs π P (X) and π Q (Y ) must lie on quadric surfaces which we denote by S P and S Q respectively. These quadrics are given by the following equations: The two quadrics have the following properties:  1. The quadric S P contains the camera centers p 1 , p 2 .

The quadric S P is ruled (contains real lines).
Proof. The first item follows from the nature of the pullback, whereas the second is due to the fact that we are working with forms of rank 2. A detailed proof is given in [2].
There are only four quadrics containing lines (up to choice of coordinates), these are illustrated in Figure 1.
The discussion so far can be summarized as follows: Theorem 4.7 (Lemma 5.10 in [2]). Let (P 1 , P 2 , X) be a critical configuration, and let π P (X) denote the blow-down of X. Then there exists a ruled quadric S P passing through the camera centers p 1 , p 2 , and containing the set of points π P (X).
Theorem 4.7 tells us that all critical configurations need to lie on ruled quadrics. The converse however, is not always true. Just because a configuration lies on a ruled quadric, it does not need to be critical. Let us give a partial converse: Lemma 4.8. Let (P 1 , P 2 , X) be a configuration of cameras and points such that X is contained in the strict transform S P of a ruled quadric S P that passes through both the camera centers. Then for each bilinear form F Q of rank 2 such that: there exists a conjugate configuration to (P 1 , P 2 , X).
Proof. Assume such a bilinear form F Q exists. By Theorem 4.4 we know that there exists a pair of cameras (Q 1 , Q 2 ) such that F Q is their fundamental form. Since X lies on S P , we have Then for every point x ∈ X we can find a point y such that Let Y be the set of these points y. Then (Q 1 , Q 2 , Y ) is a conjugate to (P 1 , P 2 , X). This can be repeated for each bilinear form of rank 2, giving unique conjugate configurations.
The problem of determining which configurations are critical is now reduced to finding out which quadrics are the pullback a real bilinear form of rank 2.
Let F denote the space of bilinear forms on P 2 × P 2 . Since all such forms can be represented by a 3 × 3 matrix (up to scaling) we have that F is isomorphic to P 8 . The fundamental form F P of the two cameras (P 1 , P 2 ) is an element in F.

Lemma 4.9.
There is a 1 : 1 correspondence between the set of real quadrics in P 3 passing through p 1 , p 2 , and the set of real lines in F passing through F P .
Proof. Let L ⊂ F be a line through F P , and let F 0 = F P be some point on L.
Every point F ∈ L can be written as αF 0 + βF P for some [α : β] ∈ P 1 . But then we have for F = F P Hence, the equation F (P 1 (x), P 2 (x)) describes the same quadric for all points F = F P on L.
Next, a quadric S passing through p 1 and p 2 is fixed by a set of 7 points {x 1 , . . . , x 7 } on S in generic position. Demanding that a bilinear form pulls back to a quadric passing through a specific point is one linear constraint in F. So with seven generic points, there is exactly one line L through F P such that the forms on this line pull back to S.
Using this 1 : 1 correspondence and Lemma 4.8, the problem has been reduced to determining which quadrics correspond to lines in F containing at least one viable form of rank 2. Let F 2 denote the Zariski closure of the rank 2 locus. Since F 2 is a hypersurface of degree 3, a generic line L will contain two forms of rank 2 in addition to F P . There are also other possibilities, listed in the tables below (the underlying computations for these tables can be found in Appendix A).
We start with the cases where the line L corresponding to S P has a finite number of intersections with the rank 2 locus F 2 .

Intersection points
S P All three intersection points are distinct real points A smooth quadric, cameras not on a line Two intersection points F P and F Q , L ∩ F 2 has multiplicity 2 at F P A cone, two cameras not on a line, neither camera on a vertex Two intersection points F P and F Q , L ∩ F 2 has multiplicity 2 at F Q A smooth quadric, cameras lie on a line These are the cases where we have at least one real rank 2 form F ∈ L different from F P . There are, however, some cases where there are no viable forms:

Intersection points
S P The two other intersection points are complex conjugates A smooth non-ruled quadric The two other intersection points are of rank 1 [1] Union of two planes, cameras in different planes The two intersection points are equal to F P A cone, both cameras on a line, neither camera at the vertex In the case where L is contained in F 2 , all the forms on L are of rank 2 (with the possible exception of at most 2 that can be of rank 1). As such, rather than looking at where the intersections are, we look at the epipoles of the forms in L:

Epipoles
S P All forms have different epipoles Two planes, cameras lying in same plane All forms share the same right epipole, the left epipoles trace a line in P 2 [2] Two planes, one camera on the intersection of the planes All forms share the same right epipole, the left epipoles trace a conic in P 2 [2] Cone, one camera at the vertex All forms share the same right and left epipole Two planes, both cameras lying on the intersection of the planes All forms share the same right and left epipole AND the two rank one forms on L coincide Double plane (as a set it is equal to a plane, but every point has multiplicity 2) With this, we have a classification of all (maximal) critical configurations for two views: Theorem 4.10. A configuration (P 1 , P 2 , X) is critical if and only if the camera centers p 1 , p 2 and all the points π P (X) are contained in a quadric S P , where S P is one of the quadrics in Table 1. (illustrated in Figure 2) Recall that by Definition 2.5, we required two conjugate configurations to not be projectively equivalent. Yet by Theorem 4.10, we note that many of the critical configurations have conjugates that are of the same type. Now, while there is indeed some A ∈ PGL(4) taking any smooth quadric S P to any other [1] If they are of rank 1, they must be equal. [2] The statement also holds true if we swap "left" and "right".  smooth quadric S Q , we will soon see that the map taking a point in S P to its conjugate on S Q certainly does not lie in PGL(4). In the final section we will give a description of the map taking a point to its conjugate, to make it clear that it is not a projective transformation.

Epipolar lines
Before we can describe the map taking a point to its conjugate, we need to point out a certain pair of lines on S P . Given two pairs of cameras (P 1 , P 2 ) and (Q 1 , Q 2 ), let S P be the pullback of F Q using φ P , and define the two lines The lines are the pullback of the epipoles from the other set of cameras, so we will call them epipolar lines.
Epipolar lines are key in understanding the relation between points on S P and points on its conjugate S Q . They also play an important role in the study of critical configurations for more than 2 cameras, so let us give a brief analysis of these lines.
1. The line g j Pi lies on S P and passes through p i .

2.
Any point lying on both g j Pi and g i Pj must be a singular point on S P .

Any point in the singular locus of S P that lies on one of the lines must also lie on the other.
4. If S P is the union of two planes, g j Pi and g i Pj lie in the same plane.
The first two properties are taken from Lemma 5.10 in [2], the last two are neither stated nor proven in the paper. Nevertheless, the authors seem to have been aware of all four properties.
3. For ease of reading, we will be using matrix notation. As such, S P , F Q and P i are represented by matrices of dimensions 4 × 4, 3 × 3 and 3 × 4 respectively.
If x 0 ∈ g 2 P1 lies in the singular locus of S P , then we have However, since x 0 lies in the singular locus of S P , this should be zero. But since both the camera matrices are of full rank, the only way we can get zero is if x 0 = p 2 or if P 2 x 0 = e 1 Q2 . In either case, it follows that x 0 must lie on g 1 P2 as well.
4. Assume that there exists cameras P 1 , P 2 , Q 1 , Q 2 such that S P is the union of two planes and the epipolar lines g j Pi lie in different planes. When the quadric S P consists of two planes, one of the planes, which we denote by Π, will (by Theorem 4.10) contain both camera centers . As such, the only way that the epipolar lines can lie in different planes is if one of the camera centers, say p 2 , lies on the intersection of the two planes and the other does not (if both lie on the intersection, then by 3. the epipolar lines will both be equal to the intersection of the two planes).
Recall that the quadric S P and its conjugate S Q (also two planes) are both pullbacks of the surface Im( φ P ) ∩ Im( φ Q ) ⊂ P 2 × P 2 . The map φ P takes the plane Π to the product of two lines in P 2 × P 2 . The line in the first image will pass through the epipole e 2 Q1 , whereas the line in the second image will not pass through the epipole e 1 Q2 (this is due to the fact that Π contains one of the epipolar lines but not the other). The problem is now that neither of the planes on S Q can map to the product of these two lines, since any such plane would have to be both (a) a plane passing through q 2 but not through q 1 (due to the fact that in the second image, the line does not pass through e 1 Q2 ) and (b) a plane passing through both q 1 and q 2 (due to the fact that the plane maps to a line in both images), which gives us a contradiction. It follows that there are no P 1 , P 2 , Q 1 , Q 2 such that S P is the union of two planes and the epipolar lines g j Pi lie in different planes, so whenever S P is the union of two planes, the epipolar lines must lie in the same plane.
Proposition 5.2. Let P 1 , P 2 be two cameras, and let S P be a quadric passing through their camera centers. The configuration (P 1 , P 2 , S P ) is critical if and only if S P contains a pair of lines satisfying the four conditions in Lemma 5.1. Furthermore, there is a 1:1 correspondence between pairs of permissible lines and configurations conjugate to (P 1 , P 2 , S P ) unless p 1 , p 2 both lie in the singular locus of S P .
Proof. The first part can be proven by comparing the quadrics in Table 1 to the set of quadrics containing the required lines, and noting that they are the same. We leave this to the reader.
The second part is immediate for the three cases where there is a finite number of conjugates, since the number of conjugates is equal to the number of pairs of epipolar lines. For the remaining quadrics, (cone and two planes) we recall that there is a 1-dimensional family of fundamental forms F Q satisfying S P (x) = F Q (P 1 (x), P 2 (x)).
Each of these will give rise to a pair of epipolar lines (which, by Lemma 5.1, always satisfy the four conditions). Conversely, we observe that each of these quadrics has a one dimensional family of pairs of permissible lines corresponding exactly to the epipolar lines we get by choosing different fundamental forms F Q .

Maps between quadrics
In this section we examine the relations between points on one quadric and points on its conjugates. We want to differentiate between the regular, blown-up case, and the simplified case where we consider everything as objects in P 3 . Hence, we use S P to denote the quadric in P 3 , and S P to denote its strict transform in P 3 . We let φ P denote the joint camera map from P 3 to P 2 × P 2 , and let φ P denote the joint camera map restricted to points that are not camera centers. The camera centers in P 3 will be denoted by p i as usual, whereas the exceptional divisors we get when we blow them up will be denoted by E pi .
We fix two pairs of cameras (P 1 , P 2 ) and (Q 1 , Q 2 ), and let S P and S Q be defined as in Equation (1) on page 8. The two configurations (P 1 , P 2 , S P ) and (Q 1 , Q 2 , S Q ) are then conjugate to one another. Define the incidence variety: If we can understand I, we will know the exact relation between points on one quadric and the other. We have the following the commutative diagram: We will study I, by studying the fibers of the projection map π 1 . Note that for any point x ∈ S P , we have where l i is the line Q −1 i (P i (x)). We will not refer to this formula explicitly, but it is the foundation for the analysis of the fibers. For x ∈ S P , note that the fiber π −1 1 (x) consists of a single point, as long as x does not lie on the intersection of the two epipolar lines g 2 P1 , g 1 P2 on S P . If x lies on both the epipolar lines, the fiber is a line l ⊂ I, such that π 2 (l) is the line spanned by the two camera centers q 1 , q 2 . It follows that: 1. If S P is smooth, the map π 1 is an isomorphism.

The quadric S P is singular if and only if the quadric S Q contains the line
spanned by the camera centers q 1 , q 2 . Proof.
1. The fiber π −1 1 (x) consists of a single point, as long as x does not lie on the intersection of the two epipolar lines on S P . By Lemma 5.1, any point lying on the intersection of the epipolar lines must be a singular point on S P , so if S P is smooth, the epipolar lines do not intersect. Then every fiber is a singleton, meaning that π 1 is injective. Furthermore, for each point x ∈ S P there is at least one point y ∈ S Q such that φ P (x) = φ Q (y), so π 1 is surjective as well.
2. If S P is singular, the epipolar line g 2 P1 must pass through some singular point x ∈ S P (since every line on S P does, see Figure 1). By Lemma 5.1, the other epipolar line g 1 P2 must pass through the same point. Since the two epipolar lines intersect, the quadric S Q must contain the line spanned by the camera centers.
Conversely, if S Q contains the line spanned by the camera centers, the conjugate to any point on this line is the intersection of the epipolar lines g 2 P1 , g 1 P2 on S P . By Lemma 5.1, these points are always singular points on S P .
Let us now consider the general case, where both surfaces S P and S Q are smooth. In this case, since π 1 and π 2 are isomorphisms, both surfaces are isomorphic to I, and hence isomorphic to one another. In fact, they are isomorphic to a Del Pezzo surface of degree 6. This surface has 6 (−1)-curves, and the only difference between S P and S Q is which two lines we choose to be the blow-up of the camera centers. Figure 3 shows the configuration of lines on the degree 6 Del Pezzo surface. Figure 4 shows the same surface embedded in P 3 . In one embedding we chose the two green lines to be the exceptional divisors, in the other we chose the red ones. Whichever pair of lines we chose to be the exceptional divisors on S P , will be the epipolar lines on S Q and vice versa.
We now know that the conjugates of the exceptional divisors (or camera centers) on S P are the epipolar lines g Q on S Q and vise versa. Furthermore, we know that the conjugates of two non-epipolar lines passing through the camera centers (the ones in the family without epipolar lines), are two non-epipolar lines passing through the camera centers on the other side. With this, we can determine the conjugate of any curve on S P . For this final part, we will switch to the blown-down case for the sake of intuition, where the exceptional divisors are blown down to camera centers. In this case, the epipolar lines on one quadric, collapse to camera centers on the other. Figure 4: Illustration of the isomorphism between S P and S Q . It maps red lines to red lines, blue to blue and green to green. The lines on S that are blow-ups of the camera centers are represented as lines "sticking out" of the quadric.
Let S P be a smooth quadric, and let C be a curve on S P . The two epipolar lines g j Pi lie in the same family of lines. Let a be the number of times C intersects a generic line in this family, and b the number of times it intersects a generic line in the other family. Furthermore, let c i be the intersection multiplicity with the camera center p i . Then we say that C is of type (a, b, c 1 , c 2 ) (recall that (a, b) is called the bidegree of the curve). Proof. Let C Q be of type (a , b , c 1 , c 2 ). We know that all points x = p 1 , p 2 on the epipolar lines g P map to camera centers on the other side, so we get Furthermore, due to the obvious symmetry of the problem, the argument can be reversed to go from S Q to S P , giving us: It follows that a = a.
Lastly, we need to determine how many times C Q intersects the lines in the other family (the one without the epipolar lines). Any line is as good as any other, so we choose the line l passing through q 1 . The line l is the image of the line through p 1 that is not an epipolar line, so any point on both C P and this line (except the camera center p 1 ) is mapped to the line l, giving us b − c 1 points. Furthermore, the point q 1 lies on l, so we need to add the a − c 2 points on C P that map to q 1 . This gives us b = a + b − c 1 − c 2 Figure 5: Illustration of the map taking a point to its conjugate. The smooth quadric has two conjugates (original in center), the left is the one we get if we take the lines in the blue family to be the epipolar lines, whereas the one on the right is the one we get if we choose the red. intersection points with l.
Proposition 5.4 does in fact hold true even if S P or S Q is a cone, as long as no camera center lies on the vertex. In this case, a needs to be interpreted as the number of times a curve intersects each line outside of the vertex, and b is the number of times it intersects each line. From this we can see that on the smooth quadric, one family of lines will map to lines on the conjugate configuration, whereas lines in the other family will map to conics passing through the two camera centers. This is illustrated in Figure 5 and 6.

A Appendix
Proof behind tables in Section 4.4 Let P 1 , P 2 be two cameras with camera centers p 1 , p 2 and fundamental form F P . Let F denote space of bilinear forms on P 2 × P 2 . We know (by Lemma 4.9) that there is an isomorphism between the set of quadrics passing through p 1 , p 2 , and lines L ⊂ F that pass through F P . The set of bilinear forms that are not of full rank is denoted by F 2 . There are several different ways a line l through F P can intersect F 2 , namely: When we have a finite number of intersections: • Three distinct, real intersection points (one of which is F P ).
• Three distinct intersection points, two of which are complex conjugates.
• Two distinct intersection points of rank 2, F P has multiplicity 2.
• Two distinct intersection points of rank 2, the one that is not F P has multiplicity 2.
• Only one intersection point: F P with multiplicity 3.
• Two distinct intersection points, F P and one which has rank 1. The latter must have multiplicity 2.
Then there are the ones where L lies in F 2 ; here we will look at the kernels, and at the number of rank 1 forms on L: • Two distinct, real, rank 1 forms (implies all bilinear forms share the same left-and right kernel).
• Two distinct, complex, rank 1 forms (implies all bilinear forms share the same left-and right kernel).
• One form of rank 1, which has multiplicity 2 (implies all bilinear forms share the same left-and right kernel) • One form of rank 1, which has multiplicity 1 (implies all bilinear forms share the same left-OR right kernel, but never both) • Let PGL(3, R) be the projective general linear group of degree 3 (the group of real, invertible 3 × 3 matrices up to scale). This group acts on F with an action that can be represented by matrix multiplication. Let PGL(3) F P be the subgroup of PGL(3) that fixes F P . This gives us a group that acts on the P 7 of lines passing through F P . This group will never take a line with one of the 12 configurations above to a line with a different configuration. For instance, a real invertible matrix will never take three distinct real points, to anything other than three distinct real points. In particular, this means that the 12 configurations above all lie in distinct orbits under this group action.
Recall the isomorphism between the set of quadrics through p 1 , p 2 and the set of lines through F P . It gives us a similar group action on the set of quadrics passing through the camera centers p 1 , p 2 , namely the subgroup of PGL(4) that fixes the two camera centers. Here too, the camera/quadric configurations fall into 12 different orbits (not listed). Now we only need to check one representative from each orbit to find the exact correspondences.
By Proposition 2.10, the only property of the cameras that matters when considering critical configurations is their center. Furthermore, one pair of distinct points in P 3 is no different from any other pair. This means that when we make computations for the two view case, we are free to pick any two cameras P 1 , P 2 with distinct centers. We will use The fundamental matrix of these two cameras is Now we can go ahead and check the 12 possible configurations listed earlier. This will be done by picking a fundamental matrix F 0 such that the line l spanned by F P and F 0 intersects F 2 the way we want, and checking what quadric it corresponds to. The full results are given in Table 2. Table 2 Matrix l lies in F 2 , and contains two distinct, real, rank 1 forms [3] Union of two planes, camera centers lie on the intersection of the planes   1 0 0 0 1 0 0 0 0   l lies in F 2 , and contains two distinct, complex, rank 1 forms [3] Union of two complex conjugate planes, both cameras on intersection.
  1 0 0 0 0 0 0 0 0   l lies in F 2 , and contains one form of rank 1, which has multiplicity 2 [3] A single plane with multiplicity 2   0 0 1 0 0 0 0 0 0   l lies in F 2 , and contains one form of rank 1, which has multiplicity 1 [4] Union of two planes, one camera on the intersection Union of two planes, cameras in same plane, neither at the intersection [3] This implies all bilinear forms share the same left and right kernel [4] This implies all bilinear forms share the same left or right kernel