Inclusion Matrices and the MDS Conjecture

Let F_q be a finite field of order q with characteristic p. An arc is an ordered family of at least k vectors in (F_q)^k in which every subfamily of size k is a basis of (F_q)^k. The MDS conjecture, which was posed by Segre in 1955, states that if k<= q, then an arc in (F_q)^k has size at most q+1, unless q is even and k=3 or k=q-1, in which case it has size at most q+2. We propose a conjecture which would imply that the MDS conjecture is true for almost all values of k when q is odd. We prove our conjecture in two cases and thus give simpler proofs of the MDS conjecture when k<= p, and if q is not prime, for k<= 2p-2. To accomplish this, given an arc G of (F_q)^k and a nonnegative integer n, we construct a matrix M_G^{\uparrow n}, which is related to an inclusion matrix, a well-studied object in combinatorics. Our main results relate algebraic properties of the matrix M_G^{\uparrow n} to properties of the arc G and may provide new tools in the computational classification of large arcs.


Introduction
Let F q be a finite field of order q with characteristic p. An arc in F k q is an ordered family of at least k vectors in which every subfamily of size k is a basis of F k q . Most authors define an arc, equivalently, as an unordered set of points in the corresponding projective space. For the techniques developed in this article, however, we find it more convenient to define arcs as ordered families of vectors. On the other hand, we will denote arcs with set notation rather than tuple notation as this is more natural.
Given an arc G ⊂ F k q and a basis B of F k q , let M(G, B) be the matrix whose columns are the vectors in G written with respect to the basis B in the order given by G. If G, G ′ ⊂ F k q are two arcs, then we say that G is linearly equivalent to G ′ if the matrix M(G, B) can be transformed into the matrix M(G ′ , B) using only elementary row operations, column permutations, and multiplication of columns by nonzero scalars.
A natural question is to determine how large an arc in F k q can be.
Question 1.1 What is the maximum size g(k, q) of an arc in F k q ? Question 1.1 interests the coding theory, algebraic geometry, and finite geometry communities, and its importance is highlighted by a $1000 prize offered for its solution by the Information Theory and Applications (ITA) center at UCSD [18].
If (e 1 , . . . , e k ) is a basis for F k q , then a natural arc in F k q of size k + 1 is given by {e 1 , . . . , e k , e 1 + · · · + e k }, (1.1) which proves that g(k, q) k + 1. A straightforward argument shows that g(k, q) = k + 1 when k q, and moreover if S ⊂ F k q is an arc of size k + 1, then S is linearly equivalent to (1.1). This result was first proved by Bush [5] in 1952. Question 1.1 becomes difficult to answer, however, when k < q. In this case, we can construct arcs that are larger than the arc in (1.1). For example, the normal rational curve R k ⊂ F k q , which is defined by is an arc of size q + 1. The normal rational curve R k shows that g(k, q) q + 1, and in 1955, Segre [16] conjectured that this lower bound is tight in most cases when k q.
Conjecture 1.2 (Segre, [16]) If k q, then the maximum size g(k, q) of an arc in F k q is g(k, q) = q + 1 if q is odd or k / ∈ {3, q − 1} q + 2 if q is even and k ∈ {3, q − 1}. Conjecture 1.2 is called the MDS conjecture or the main conjecture for maximum distance separable codes, and was first posed by Segre as a question.
By the well-known principle of duality, if S ⊂ F k q is an arc of size s > k, then up to linear equivalence, we can associate a unique dual arc S ⊥ ⊂ F s−k q of size s. This has two immediate implications. First, it explains why in Conjecture 1.2, exceptions occur for both k = 3 and k = q − 1 when q is even. Second, it shows that if g(k, q) = q + 1, then g(q + 2 − k, q) = q + 1. As a result, if q is odd and g(k, q) = q + 1 when k (q + 2)/2, then g(k, q) = q + 1 for all k q. Duality thus allows us to prove Conjecture 1.2 when q is odd by restricting to the case k (q + 2)/2.
Ball [1] proved that g(k, q) = q + 1 when k p = char (F q ), and thus verified Conjecture 1.2 when q is prime. For a complete list of when Conjecture 1.2 is known to hold for q non-prime, see [10] and [11]. The best-known bounds up to first-order of magnitude (c i are constants), are that for q an odd non-square, we have g(k, q) = q + 1 when k < √ pq/4 + c 1 p, which was proved by Voloch [19]. For q = p 2h , where p 5 is a prime, we have g(k, q) = q + 1 when k √ q/2 + c 2 , which was proved by Hirschfeld and Korchmáros [9]. Ball and De Buele [4] proved that g(k, q) = q + 1 when k 2 √ q − 2 and q = p 2 .
If k q and q is odd or k / ∈ {3, q − 1}, it is natural to ask if the normal rational curve R k is the unique arc in F k q of size q + 1 up to linear equivalence. By results of Kaneta and Maruta [12] and Seroussi and Roth [17], a positive answer to this question would imply Conjecture 1.2. For many values of k and q, the normal rational curve R k is the unique arc in F k q of size q + 1 up to linear equivalence [10], but Glynn [7] showed that this is not always true. The Glynn arc G ⊂ F 5 9 is an arc of size 10 and is defined by G = {(1, t, t 2 + ηt 6 , t 3 , t 4 ) | t ∈ F 9 } ∪ {(0, 0, 0, 0, 1)}, (1.3) where η ∈ F 9 satisfies η 4 = −1. Remarkably, the Glynn arc G is the only known arc in F k q of size q + 1 that is not linearly equivalent to the normal rational curve R k when k q and q is odd.

New Results
We propose a conjecture, Conjecture 1.9, which would imply that g(k, q) = q + 1 when where p = char (F q ). In Section 1, we noted that to prove Conjecture 1.2 when q is odd, it suffices to restrict to the case k (q + 2)/2 by duality. As p grows, the right hand side of (1.4) becomes very close to (q + 2)/2. Consequently, if Conjecture 1.9 is true, then Conjecture 1.2 is true for almost all values of k when q is odd. To state Conjecture 1.9, given an arc G ⊂ F k q and a nonnegative integer n, we define a matrix M ↑n G whose algebraic properties are related to properties of G. Definition 1.3 Let G ⊂ F k q be an arc and let 0 n |G| − k + 1. Let B be a basis of F k q and let M ↑n G be a matrix whose rows are indexed by G k−1 , whose columns are indexed by ordered pairs (U, A) where U ∈ G n and A ∈ G\U k−2 , and whose (C, (U, A))-entry is In (1.5), det(u, C) B denotes the determinant of the matrix whose first row is u written with respect to the basis B and whose last k − 1 rows are the elements of C written with respect to the basis B in the order inherited from G.
Although the matrices M ↑n G may seem unfamiliar, we claim that they are related to inclusion matrices, which are well-studied in combinatorics. Recall that the inclusion matrix I r (a, b) has its rows indexed by {1,...,r} a , its columns indexed by {1,...,r} b , and (A, B)-entry For example, when n = 0, the matrix M ↑0 G is the inclusion matrix I |G| (k − 1, k − 2). When n > 0, the matrix M ↑n G is formed by gluing together matrices which are equivalent to inclusion matrices. For a fixed U ∈ G n , let D U be a diagonal matrix whose rows and columns are indexed by G\U k−1 and whose (C, C)-entry is u∈U det(u, C) B . We then have that the submatrix M ↑n G (U) of M ↑n G whose rows are indexed by G\U k−1 and whose columns are indexed by ordered pairs (U, A), where A ∈ G\U k−2 , equals D U I |G\U | (k − 1, k − 2). It is easy to see that linear equivalence of the arcs G and G ′ induces equivalence of the corresponding matrices M ↑n G and M ↑n G ′ . More precisely, if G, G ′ ⊂ F k q are linearly equivalent arcs and B and B ′ are the bases of F k q used in the construction of the matrices M ↑n G and M ↑n G ′ respectively, then there exist invertible matrices N 1 and N 2 so that M ↑n Our main results relate algebraic properties of the matrix M ↑n G to properties of the arc G. For example, our first main result says that if G is an arc whose matrix M ↑n G has full row rank, then G cannot be extended to a larger arc of a specific size. Theorem 1.4 Let G ⊂ F k q be an arc and let n ∈ N be a natural number such that If the matrix M ↑n G has full row rank, then the arc G cannot be extended to an arc of size q + 2k − 1 + n − |G|.
The left-hand and right-hand sides of (1.7) respectively are required so that the matrix M ↑n G exists and so that the arc G has size strictly smaller than q + 2k − 1 + n − |G|. Suppose 0 n q − 2k + 4 so that 2k − 3 + n q + 1. Also, suppose we can show that for all arcs G ⊂ F k q of size 2k − 3 + n, the matrix M ↑n G has full row rank. If an arc of size q + 2 exists in F k q , then it would contain a subarc G of size 2k − 3 + n that can be extended to an arc of size q + 2k − 1 + n − |G|, which contradicts Theorem 1.4. Consequently, Theorem 1.4 allows us to eliminate the existence of arcs of size q + 2 in F k q by proving that for all arcs G ⊂ F k q of size 2k − 3 + n, the matrix M ↑n G has full row rank.
Corollary 1.5 If 0 n q − 2k + 4 and for every arc G ⊂ F k q of size 2k − 3 + n, the matrix M ↑n G has full row rank, then g(k, q) = q + 1. Since the matrices M ↑n G are related to inclusion matrices, knowing the ranks of inclusion matrices over F q will be crucial to verifying the condition in Corollary 1.5. Theorem 1.6 (Frankl [6], Wilson [20]) For fixed integers 0 b a r − b and a prime p = char (F q ), we have For example, when n = 0 and G ⊂ F k q is an arc of size 2k − 3, the matrix M ↑0 G is the inclusion matrix I 2k−3 (k − 1, k − 2). Theorem 1.6 thus implies the first assertion of Theorem 1.7. Theorem 1.7 If G ⊂ F k q is an arc of size 2k − 3, then the matrix M ↑0 G has full row rank exactly when k p. Hence g(k, q) = q + 1 when k p.
The second assertion of Theorem 1.7 follows from Corollary 1.5 when q is not prime. If q is prime, then Corollary 1.5 implies that g(k, q) = q + 1 when k (q + 4)/2 and hence the second assertion of Theorem 1.7 follows from duality. The second assertion of Theorem 1.7 was first proved by Ball [1].
In Section 7, we again use Theorem 1.6 to verify the condition in Corollary 1.5 when n = 1 and k 2p − 2 q.
2p − 2 q and G ⊂ F k q is an arc of size 2k − 2, then the matrix M ↑1 G has full row rank. Hence, if q is not prime, then g(k, q) = q + 1 when k 2p − 2. The bound k 2p − 2 in the first assertion of Theorem 1.8 cannot be improved because one can check using a computer that if G ⊂ F 5 9 is a subarc of size 8 of the normal rational curve R 5 ⊂ F 5 9 , then the matrix M ↑1 G does not have full row rank. The second assertion of Theorem 1.8 follows from Corollary 1.5 and was first proved by Ball and De Buele [4].
Recalling that p = char (F q ), we conjecture that if 0 n q and then the condition in Corollary 1.5 holds.
Conjecture 1.9 If 0 n q, k satisfies (1.9), and G ⊂ F k q is an arc of size 2k − 3 + n, then the matrix M ↑n G has full row rank. Observe that Theorem 1.7 and Theorem 1.8 prove Conjecture 1.9 when n = 0 and n = 1.
For larger values of n, we have computational evidence to support Conjecture 1.9.

Classification
The matrices M ↑n G are also useful for determining when the normal rational curve R k ⊂ F k q is the unique arc of size q + 1 up to linear equivalence. The second main result of this article is that if 0 n q − 2k and for any arc G ⊂ F k q of size 2k − 2 + n, the matrix M ↑n G contains a certain vector in its column space, then the normal rational curve is the unique arc of size q + 1 up to linear equivalence.
To state our theorem precisely, we define a matrix H ↑n G that is equivalent to the matrix M ↑n G so that the vector we require in the column space has a nice form. Recall that we have defined arcs to be ordered sets and that if (X, <) is an ordered set then A ⊂ X is smaller than B ⊂ X in colex order if the largest element of the symmetric difference A△B lies in B. Definition 1.11 Let G ⊂ F k q be an arc, let 0 n |G| − k + 1, and let B be the basis of F k q fixed in Definition 1.3. For each C ∈ G k−1 , let L C ∈ G\C n be the last n-subset of G\C n in colex order. Let J ↑n G be a diagonal matrix with rows and columns indexed by Define the matrix H ↑n G = J ↑n G M ↑n G and put the rows of the matrix H ↑n G in colex order. Observe that the entries of the matrix H ↑n G are independent of the basis B. We restate our second main result precisely using the matrices H ↑n G .
Theorem 1.12 If 0 n q − 2k and for every arc G ⊂ F k q of size 2k − 2 + n, the column space of the matrix H ↑n G contains a vector v ∈ F . . , k} and v i = 0 otherwise, then the normal rational curve R k is the unique arc in F k q of size q + 1 up to linear equivalence.
When n = 0 and G ⊂ F k q is an arc of size 2k − 2, the matrix H ↑0 G equals the inclusion matrix I 2k−2 (k − 1, k − 2), so we can easily verify that the column space of the matrix H ↑0 G contains the required vector when k p = char (F q ).
Theorem 1.13 If k p = char (F q ) and G ⊂ F k q is an arc of size 2k − 2, then the column space of the matrix H ↑0 G contains a vector v ∈ F . . , k} and v i = 0 otherwise. Hence, if k p and k = (q + 1)/2, then the normal rational curve R k is the unique arc in F k q of size q + 1 up to linear equivalence. It is easy to see that the bound k p in the first assertion of Theorem 1.13 cannot be improved. The second assertion of Theorem 1.13 was first proved by Ball in [1], although the condition k = (q + 1)/2 was missing there.
We conjecture in Conjecture 1.14 that if k 2p − 2 q and G ⊂ F k q is an arc of size 2k, then the column space of the matrix H ↑2 G contains the required vector in Theorem 1.12. We have computational evidence to support Conjecture 1.14, and we note that if Conjecture 1.14 is true, then the normal rational curve R k is the unique arc in F k q of size q + 1 up to linear equivalence when k 2p − 2 q.

Verifying Conjecture 1.2 and Classifying Large Arcs Computationally
An important benefit of the conditions in Corollary 1.5 and Theorem 1.12 is that they can be checked with a computer. Corollary 1.5 and Theorem 1.12 may consequently be of use in verifying Conjecture 1.2 and classifying large arcs computationally. For example, if one could classify arcs in F k q of size 2k − 2 up to linear equivalence, then one could test the rank of the matrix M ↑1 G for a representative G from each linear equivalence class. If the matrix M ↑1 G has full row rank, then Corollary 1.5 would rule out the possibility that any arc in the linear equivalence class of G could be extended to an arc of size q + 2. If the matrix M ↑1 G does not have full row rank, then one could extend G to an arc H of size 2k − 1 and check if the matrix M ↑2 H has full row rank. This should dramatically reduce the space of possible subarcs of arcs of size q + 2. In the same way, Theorem 1.12 can be used to check if the normal rational curve R k is the unique arc in F k q of size q + 1 up to linear equivalence. These algorithms should be possible to implement because the question of classifying arcs up to linear equivalence has already been considered in [8] and [13].

Important Remarks and Outline of Paper
The results in this paper are joint work with Simeon Ball, but he has elected to write a separate exposition of some of these results in [3]. A straightforward consequence of the proof of Theorem 1.4 is Theorem 1.15, which shows that that the conclusion of Theorem 1.4 holds if the matrix M ↑n G satisfies the slightly weaker condition of having a vector of weight one in its column space. Theorem 1.15 is the main result of [3].
q be an arc and let n ∈ N be a natural number such that (1.12) If the matrix M ↑n G has a vector of weight one in its column space, then the arc G cannot be extended to an arc of size q + 2k − 1 + n − |G|.
For the most interesting application of Theorem 1.4, namely Corollary 1.5, we do not believe that Theorem 1.15 offers any benefit over Theorem 1.4. In other words, we believe that if 0 n q − 2k + 4 and if for every arc G ⊂ F k q of size 2k − 3 + n the matrix M ↑n G has a vector of weight one in its column space, then for every such arc G the matrix M ↑n G has full row rank. Indeed, the bound on k in our stronger Conjecture 1.9 matches exactly the bound on k in Ball's weaker Conjecture 1 in [3]. This paper builds on the methods initiated in [1], [2], and [4]. In order for this paper to be self-contained and correct, we repeat some proofs from [1], [2], and [4], although we often give different expositions using matrices so that we may extend the results. For example, the proofs of [1, Lemma 4.2], [2,Lemma 7.20], and [4, Lemma 3.1] are incorrect as written because the determinants in those results are evaluated with respect to many different bases yet treated as if they were evaluated with respect to the same basis. We fix this here in Corollary 4.1 and in Lemma 5.5 and in [3, Section 2]. The proof of [1, Theorem 1.8] is also incorrect as written in the case k = (q + 1)/2, and this is fixed here in Theorem 1.13.
Another important change in the proof approach of [1], [2], and [4] lies in the definition of certain parameters α A in Lemma 6.1. In [2,Chapter 7], the analogue of the parameter α A in Lemma 6.1 is referred to as Q(A, F ) and its definition is dependent on a smaller subarc of a larger arc. In Lemma 6.1 and in [3, Section 3], we define the parameters α A so that they no longer depend on the smaller subarc and only depend on the larger arc. This change is crucial to the proof of Theorem 1.4.
The three main ingredients in the proofs of Theorem 1.4 and Theorem 1.12 are duality, polynomial interpolation, and Segre's Lemma of Tangents. Section 2 discusses the properties of polynomial interpolation that we use. Section 3 explains the concept of tangent functions. In Section 4, we reduce our first main result Theorem 1.4 to Theorem 4.2.
In Section 5, we reduce Theorem 4.2 to Lemma 5.3. In Section 6, we state and prove Segre's Lemma of Tangents and use it to prove Lemma 5.3, thus completing the proofs of Theorem 1.4 and Theorem 4.2. In Section 7, we prove Theorem 1.8 and thus prove Conjecture 1.9 when n = 1. In Section 8, we prove Theorem 1.12 and Theorem 1.13.

Polynomial Interpolation
That one can uniquely determine a polynomial f ∈ F[X] in one variable of degree at most t over any field F from t + 1 of its values is well-known. Similarly, one can recover a homogeneous polynomial in two variables f ∈ F(X, Y ) of degree t by knowing values of f on the points of an arc . . , f (x t+2 , y t+2 )]. As P has more columns than rows, its columns are linearly dependent. Hence, there is a solution w = [w 1 , . . . , w t+2 ] T to P w = 0 and thus z w = 0 because cP = z. We now show in Theorem 2.1 that a solution w to P w = 0 and z w = 0 is given by Theorem 2.1 is a key ingredient in the proof of Theorem 1.4.
Proof. Using the definitions of P , w, and z from the preceding paragraph, let B be a square matrix whose columns are the first t + 1 columns of the matrix P . Let b be the last column of the matrix P . Note that a solution r = [r 1 , . . . , r t+1 ] T to B r = b gives a solution w to P w = 0 with w i = r i for i ∈ {1, . . . , t + 1} and w t+2 = −1.
is an arc of size t + 2 in F 2 , we may assume that y 1 , . . . , y t+1 are nonzero. Hence the matrix B is nonsingular, so by Cramer's Rule, a solution r to B r = b is given by r i = det(B i )/ det(B) where B i is the matrix formed by replacing the i th column of B with b. Using the formula for the determinant of a Vandermonde matrix, Hence, after a little algebraic manipulation, Multiplying the corresponding solution w to P w = 0 by − 1 l t+1 (x t+2 y l − x l y t+2 ) −1 yields the solution w to z w = 0 given by (2.13).

Tangent Functions
Let S ⊂ F k q be an arc. Given a subset A ⊂ S of size k − 2, we will define the tangent function at A, denoted f A,S : F k q → F q , which can be viewed as a homogeneous polynomial in two variables with respect to certain bases of F k q . We will then apply Theorem 2.1 to the tangent functions f A,S for various A ⊂ S to prove Theorem 1.4.
To define the tangent function at A, we first count in Lemma 3.1 the number of (k − 1)-dimensional subspaces of F k q that intersect S precisely in A.
Proof. Since A is a linearly independent set of size k−2, the number of (k−1)-dimensional subspaces of F k q that contain A is q + 1. Since S is an arc, a (k − 1)-dimensional subspace of F k q that contains A can contain at most one other vector of S \ A.
Given an arc S ⊂ F k q and a subset A ⊂ S of size k − 2, we now define the tangent function at A.
Observe that f A,S (x) = 0 precisely when x ∈ t i=1 H i A and that f A,S is defined up to a scalar factor.
Notation: Recall that an arc S ⊂ F k q is ordered. If R 1 , . . . , R l are subsets of S we use (R 1 , . . . , R l ) to mean write the vectors in R 1 in order first, and then the vectors in R 2 etc. When R i is a singleton set, we simply write the vector. For example, if x, y ∈ S \ A and B is a basis of F k q , we write det(x, y, A) B for the determinant of the matrix whose rows are the vectors x, y, and the elements of A in order written with respect to B.  Proof. With respect to the basis B, the linear functional β i A in (3.17) is linear in just the first two coordinates since its kernel contains A. Hence, the tangent function f A,S is a homogeneous polynomial in two variables of degree t, where t is given by (3.16). Since S is an arc, when we write the vectors in T in terms of the basis B, their first two coordinates form an arc of size t + 2 in F 2 q . Hence, we can apply Theorem 2.1 to f A,S and T , and note that with respect to B, we have det(x, y, A) B = x 1 y 2 − y 1 x 2 .
We will show in Lemma 5.5 that the product of determinants in If P ↑n G is a matrix whose rows are indexed by G k−1 , whose columns are indexed by ordered pairs (U, A) where U ∈ G n and A ∈ G\U k−2 , and whose (C, (U, A))-entry is Let us rewrite (4.19) so that it will be easier to express the system of equations given by (4.19) in matrix form. For any fixed U ∈ G n and A ∈ G\U k−2 , we have Consequently, letting P ↑n G be the matrix defined in Corollary 4.1, we see that we can write the system of equations given by (4.20) in matrix form as 1P ↑n G = 0. The equation 1P ↑n G = 0 contains a wealth of information about the arc S ⊂ F k q and is crucial to the proof of Theorem 1.4. At the moment, the matrix P ↑n G defined in Corollary 4.1 may seem ugly and difficult to analyze, but we claim that P ↑n G is equivalent to the much simpler matrix M ↑n G defined in (1.5), which depends only on the arc G.
Theorem 4.2 Let S ⊂ F k q be an arc and let G ⊂ S have size t + k + n, where t is defined by (3.16) and n 0. If P ↑n G is the matrix defined in Corollary 4.1, then there exist invertible diagonal matrices D 1 and D 2 so that D 1 P ↑n We now reduce Theorem 1.4 and Theorem 1.15 to Theorem 4.2.
Proof of Theorem 1.4 and Theorem 1.15. We prove the contrapositive: namely that if G ⊂ F k q can be extended to an arc S ⊂ F k q of size q + 2k − 1 + n − |G|, then the matrix M ↑n G cannot have full row rank or a vector of weight one in its column space. First we show the arc G satisfies the hypotheses of Corollary 4.1. As S has size q + 2k − 1 + n − |G|, the arc G has size t + k + n, where t is defined by (3.16). By Corollary 4.1, we have 1P ↑n G = 0 and so by Theorem 4.2, we have 0 = ( 1D −1 1 )M ↑n G . Since D 1 is an invertible matrix, all entries of 1D −1 1 are nonzero. Hence, the matrix M ↑n G cannot have full row rank or a vector of weight one in its column space.

Proof that Lemma 5.3 Implies Theorem 4.2
To prove Theorem 4.2, we first define matrices Q ↑n G , R ↑n G and I ↑n G given a subset G ⊆ S where S ⊂ F k q is an arc. The matrix I ↑n G is a signed inclusion matrix and to define the signing we need the following notation.
Definition 5.1 Let X be an ordered set, let A be an ordered subset of X, and let C be an ordered subset of X that contains A and has size |A| + 1. We define τ (A, C) to be the minimum number of transpositions needed to order (A, C \ A) as C.
where S is an arc, and let 0 n |G| − k + 1. Let Q ↑n G , R ↑n G , and I ↑n G respectively be matrices whose rows are indexed by G k−1 , whose columns are indexed by ordered pairs (U, A) where U ∈ G n and A ∈ G\U k−2 , and whose (C, (U, A))-entries respectively are where f A,S is defined by Definition 3.2, τ (A, C) is defined by Definition 5.1, and t is defined by (3.16).
We will prove in Section 6 that if S ⊂ F k q is an arc, then the matrix Q ↑0 S defined in (5.21) is equivalent to the matrix I ↑0 S defined in (5.23).

Lemma 5.3
Let S ⊂ F k q be an arc. If Q ↑0 S is the matrix defined in (5.21) and I ↑0 S is the matrix defined in (5.23), then there exist invertible diagonal matrices E 1 and E 2 such that E 1 Q ↑0 S E 2 = I ↑0 S . We now use Lemma 5.3 to prove that if S ⊂ F k q is an arc and G ⊂ S satisfies the constraints of Corollary 4.1 then the matrix Q ↑n G defined in (5.21) is equivalent to the matrix I ↑n G defined in (5.23).

Lemma 5.4
Let G ⊂ S ⊂ F k q where S is an arc. If Q ↑n G is the matrix defined in (5.21), then there exist invertible diagonal matrices F 1 and F 2 such that F 1 Q ↑n G F 2 = I ↑n G . Proof. Recall that, by Lemma 5.3, there exist invertible diagonal matrices E 1 and E 2 such that E 1 Q ↑0 S E 2 = I ↑0 S . Let F 1 be the submatrix of E 1 whose rows and columns are indexed by G k−1 . Let F 2 be the submatrix of E 2 whose rows and columns are indexed by ordered pairs (U, A), where U ∈ G n and A ∈ G\U k−2 . As the entries of Q ↑n G and I ↑n G don't depend on U ∈ G n , we have F 1 Q ↑n G F 2 = I ↑n G .
We now prove that if S ⊂ F k q is an arc and G ⊂ S satisfies the constraints of Corollary 4.1 then the matrix R ↑n G defined in (5.22) is equivalent to the Hadamard product I ↑n G • M ↑n G .
Lemma 5.5 Let S ⊂ F k q be an arc and let G ⊂ S satisfy the hypotheses of Corollary 4.1. If R ↑n G is the matrix defined in (5.22), then there exist invertible diagonal matrices F 3 and  6 Proof of Lemma 5.3 In this section, we prove Lemma 5.3 and hence complete the proofs of Theorem 1.4 and Theorem 4.2. In Lemma 6.1, we show that Lemma 5.3 holds if we can find a vector in the nullspace of a certain matrix L all of whose coordinates are nonzero. In Lemma 6.2, we state and prove Segre's Lemma of Tangents, which we use in Lemma 6.3 to show that the matrix L does not have full column rank. Consequently, L has nonzero vectors in its nullspace and we prove Lemma 5.3 by showing that any nonzero vector in the nullspace of L must have all coordinates nonzero.
Recall that an arc S ⊂ F k q is ordered and that if (X, <) is an ordered set then A ⊂ X is smaller than B ⊂ X in lex order if the smallest element of the symmetric difference A△B lies in A. Lemma 6.1 Let S ⊂ F k q be an arc. Let L be a matrix whose columns are indexed by S k−2 and whose rows are indexed by ordered pairs (A, where t is defined by (3.16). If there exists a vector α ∈ F ( |S| k−2 ) q in the nullspace of L all of whose coordinates are nonzero, then Lemma 5.3 holds.

Proof. We write the coordinates of α as α
S is the matrix defined in (5.21), C ∈ S k−1 , A, A ′ ∈ C k−2 , and t is defined by (3.16), then Define E 2 to be a diagonal matrix with rows and columns indexed by S k−2 and (A, A) entry E 2 (A, A) = α A . Since the coordinates of α are nonzero and Q ↑0 S (C, A) = 0 when by (6.31). Consequently, defining E 1 to be a diagonal matrix with rows and columns indexed by S k−1 and (C, C)-entry E 1 (C, C) = α −1 C , we see that Lemma 5.3 holds.
To prove the existence of a vector α ∈ F ( |S| k−2 ) q satisfying the hypotheses of Lemma 6.1, we first show in Lemma 6.3 that the matrix L defined in (6.30) does not have full column rank over F q . For this, we need Lemma 6.2, which is called Segre's Lemma of Tangents and gives a relationship between values of different tangent functions. Lemma 6.2 (Ball [1,2]) Let S ⊂ F k q be an arc and let t be defined by (3.16). For a subset D ⊂ S of size k − 3 and a subset {u, v, w} ∈ S \ D, we have Proof. Observe that B = (u, v, w, D) is a basis of F k q because S is an arc. For x ∈ F k q , let x = (x 1 , . . . , x k ) be the coordinates of x with respect to B. By (3.17), Our first goal is to show that Since the product of the nonzero elements of a finite field F q equals −1, (6.35) implies Multiplying (6.37), (6.38), and (6.39), and canceling x∈S\B x 1 x 2 x 3 from both sides, we see that (6.33) holds.
Now we use Lemma 6.2 to show that the matrix L defined in (6.30) does not have full column rank over F q . To accomplish this, we must order the rows of L. First, list the rows of R and then list the remaining rows in lex order. We will show that each row of L that is not in R can be written as a linear combination of two rows of L that precede it. Hence, by induction, every row of L can be written as a linear combination of rows in R.
Let L (A,A ′ ) be a row of L that is not in R. We distinguish two cases. and is a linear combination of L (Â,A) and L (Â,A ′ ) : Proof of Lemma 5.3 By Lemma 6.3, the matrix L defined in (6.30) does not have full column rank over F q , so there exists a nonzero vector α in the nullspace of L. The coordinates α A of α satisfy (6.31) and we now show that they are all nonzero. Suppose, for a contradiction, that there existsÂ ∈ S k−2 such that αÂ = 0. By (6.31), α A ′ = 0 for all A ′ ∈ S k−2 such thatÂ ∪ A ′ ∈ S k−1 . Repeating this argument, we see that α A = 0 for all A ∈ S k−2 , which contradicts that α = 0. Therefore, all coordinates of α are nonzero so Lemma 5.3 holds by Lemma 6.1.
Let F be the subset consisting of the first k − 2 elements of S. For a subset A ∈ S k−2 , let D = A∩F , let A\F = {x 1 , . . . , x r }, let F \A = {z 1 , . . . , z r }, and let s be the minimum number of transpositions required to order (F ∩ A, F \ A) as F . Let t be defined by (3.16).
One can show that an explicit solution for a nonzero vector α ∈ F ( |S| k−2 ) q in the nullspace of L is given by 7 Proof of Theorem 1.8 Let G ⊂ F k q be an arc of size 2k −2. For C ∈ G k−1 , let e(C) ∈ F ( 2k−2 k−1 ) q be the C-coordinate vector; that is e(C) C ′ = 1 if C = C ′ and e(C) C ′ = 0 otherwise. To prove that the matrix M ↑1 G defined in (1.5) has full row rank over F q when k 2p − 2 q, we will show that for each C ∈ G k−1 , the C-coordinate vector e(C) lies in the column space of M ↑1 G . For a fixed U ∈ G 1 , recall that we noted in Section 1.1 that the submatrix M ↑1 . Hence, to understand the column space of M ↑1 G , we must understand the column space of I 2k−3 (k − 1, k − 2). By Theorem 1.6, the inclusion matrix I 2k−3 (k − 1, k − 2) is invertible over F q exactly when k p = char (F q ) so our first goal is to determine a spanning set for the orthogonal space of the column space of I 2k−3 (k − 1, k − 2) over F q when k > p. This will allow us to prove that a vector y lies in the column space of I 2k−3 (k − 1, k − 2) over F q by showing that y is orthogonal to every vector in the spanning set.
We now define some special vectors in F ( 2k−3 k−1 ) q . We will show in the proof of Theorem 1.8 that variants of these vectors lie in the column space of M ↑1 G .
We now show that if k > p = char (F q ), the vector v k−p (X, Y, ∆) defined in (7.45) lies in the column space of I 2k−3 (k − 1, k − 2) over F q . , let I 2k−3 (k +p−2, k −1) H be the row of the inclusion matrix I 2k−3 (k + p − 2, k − 1) corresponding to H. We want to show that Moreover, define F to be the family of (k − 1)-subsets C of {1, . . . , 2k − 3} such that I 2k−3 (k + p − 2, k − 1) (H,C) = 0 and v k−p (X, Y, ∆) C = 0. If H does not contain ∆ or if R(X) and R(Y ) have nonempty intersection, then F = ∅ and thus (7.46) holds. Otherwise, the elements of F are of the ) and observe that W = ∅ because R(X) and R(Y ) are disjoint and because |H| = k − p − 1. Since the left hand side of (7.46) equals the lemma follows.
For a fixed U ∈ G 1 , recall that we noted in Section 1.1 that the submatrix M ↑1 Observe that the vector v k−p (U, X, Y, ∆) is the vector D U v k−p (X, Y, ∆) padded with zeroes in all coordinates C ∈ G k−1 that have nonempty intersection with U. Consequently, when k > p, the vector v k−p (U, X, Y, ∆) lies in the column space of M ↑1 G for any choice of U ∈ G 1 and any choice of X, Y , and ∆ satisfying the constraints in Definition 7.4. In the proof of Theorem 1.8, we show that each of the C-coordinate vectors e(C) are linear combinations of the vectors v k−p (U, X, Y, ∆), and hence lie in the column space of M ↑1 G . To specify the linear combination, we need the following lemma.
Lemma 7.5 Let G ⊂ F k q be an arc of size 2k − 2 and let B be the basis of F k q fixed in Definition 1.3. Let C ∈ G k−1 and suppose that ∆ ⊂ C. Let W ⊂ G \ ∆ have size k − |∆|. For any u ∈ G, we have same for all w ∈ W , Observe that the support of v i−1 (U, X ′ , Y ′ , ∆ ′ ), denoted S U , is a subset of the support S w from (7.50), Consequently, to prove (7.51), we must show that the C-coordinates of the left and right hand sides of (7.51) are equal, We see that (7.53) follows from Lemma 7.5 because if C ∈ S w then ∆ ⊂ C and W ⊂ G\∆ has size k − |∆| so the left hand side of (7.53) equals If C ∈ S w \ S U , then i ∈ τ which implies that U ∈ C and hence det(U, C) B = 0.

Classification
To prove Theorem 1.12, we first state a sufficient condition for an arc S ⊂ F k q of size q + 1 to be linearly equivalent to the normal rational curve R k . Lemma 8.1 (Roth-Lempel [15]) Suppose that S ⊂ F k q is an arc of size q + 1 and let B = (e 1 , . . . , e k ) ⊂ S be a basis of F k q . For x ∈ S \ B, let x = (x 1 , . . . , x k ) be the coordinates of x when written with respect to B. Let W S,B be a matrix whose columns are the vectors (x −1 1 , . . . , x −1 k ) ⊤ for x ∈ S \ B. If rank W S,B = 2, then S is linearly equivalent to the normal rational curve R k ⊂ F k q . Suppose that S ⊂ F k q is an arc of size q+1 and that there exists a nonnegative integer n for which the hypothesis of Theorem 1.12 is satisfied. Moreover, let B = (e 1 , . . . , e k ) ⊂ S be a basis of F k q . To prove that the matrix W S,B defined in Lemma 8.1 has rank W S,B = 2, we will show that any three columns of W S,B are linearly dependent. Given three columns of W S,B , we will show they are dependent by constructing a (k − 2) × k matrix Z with rank Z = k −2 so that the three columns of W S,B lie in the nullspace of Z. In other words, we want to find k − 2 independent vectors in F k q that are orthogonal to each of the three given columns of W S,B . Using the notation of Lemma 8.1, observe that for x ∈ S \ B and 1 j k, we have The expression det(x, B \{e j }) B has appeared before, for example in (5.29), which suggests how to find the required vectors. The following lemma makes this intuition precise.
Lemma 8.2 Suppose that 0 n q − 2k and that for every arc G ⊂ F k q of size 2k − 2 + n, the column space of the matrix H ↑n G defined in Definition 1.11 contains a vector v ∈ F ( 2k−2+n . . , k} and v i = 0 otherwise. If S ⊂ F k q is an arc of size q + 1 and B = (e 1 , . . . , e k ) ⊂ S is a basis of F k q , then there exist nonzero constants c 1 , . . . , c k ∈ F q such that for any where y = (y 1 , . . . , y k ) is written with respect to the basis B.
Proof. Let A ⊂ S \ B be a subset of size k − 2 and letL ⊂ S \ (B ∪ A) be a subset of size |L| = n. Define an arc G and its ordering by G = (B, A,L). Reorder the arc S so that G is the first 2k − 2 + n vectors of S. Since |S| = q + 1, we have t = k − 2, where t is defined by (3.16). Observe that |G| = t + k + n and that |S \ G| 1 since 0 n q − 2k. Since the arc G ⊂ S satisfies the hypotheses of Recalling that D 1 = F 1 F 3 where F 1 from Lemma 5.4 is defined by the matrix E 1 in Lemma 6.1 and F 3 is defined by (5.29), we see that the C-coordinate of 1(J ↑n G D 1 ) −1 is where L C is the last n-subset of G\C n in colex order. Note that L C =L for all C ∈ B k−1 since B is the first k elements of G so We now prove Theorem 1.12.
Proof of Theorem 1.12 Let S ⊂ F k q be an arc of size q+1 and let B = (e 1 , . . . , e k ) ⊂ S be a basis of F k q . Since S is an arc, the matrix W S,B defined in Lemma 8.1 has rank W S,B 2 because if a column of W S,B is a multiple of another column of W S,B then two vectors in S are linearly dependent. To prove rank W S,B 2, we will show that any three columns of W S,B are linearly dependent. Let w, x, z ∈ S \ B. We will show that there exists a (k − 2) × k matrix Z with rank Z = k − 2 such that the columns of W S,B corresponding to w, x, z ∈ S \ B are in the nullspace of Z. As nullity Z = 2, this proves that the columns of W S,B corresponding to w, x, z ∈ S \ B are linearly dependent.
To we see that (8.60) implies that the column of W S,B corresponding to w lies in the nullspace of Z. Repeating the argument above, we similarly have that the columns of W S,B corresponding to x and z lie in the nullspace of Z as well.
Finally we prove Theorem 1.13.
Proof of Theorem 1. 13 We first show that if k p = char (F q ) and G ⊂ F k q is an arc of size 2k − 2, then the column space of the matrix H ↑0 G contains a vector v ∈ F and entries β A = β A . Let w = I 2k−2 (k − 1, k − 2) β.
(8.62) On the other hand, if C ⊂ B, then all A ∈ C k−2 satisfy l A = k − 2 so which is nonzero since k p. The first part of Theorem 1.13 is proved by setting v = ((−1) k−2 /(k − 1)!) w since the rows of H ↑0 G are in colex order. By Theorem 1.12 if k min{p, q/2}, the normal rational curve R k is the unique arc in F k q of size q + 1. By the well-known principle of duality, this implies that if k p and k = (q + 1)/2, then the normal rational curve R k is the unique arc in F k q of size q + 1.