The action of a mirabolic subgroup on a symmetric variety

Let F be a local field of character zero. Let E be a quadratic field extension of F. We show that any P-invariant linear functional on a GL(n,E)-distinguished irreducible smooth admissible representation of GL(2n,F) is also GL(n,E)-invariant where P is a mirabolic subgroup of GL(n,E).


Introduction
Let F be a local field of characteristic zero. Let E = F [δ] be a quadratic field extension of F with δ 2 ∈ F × \ (F × ) 2 . Let M at n,n (F ) (resp. M at n,n (E)) denote the set of all n × n matrices over F (resp. E). Let GL n (F ) act on M at n,n (F ) by inner conjugation. Let P F be the mirabolic subgroup of GL n (F ) consisting of matrices with last row vector (0, · · · , 0, 1). Bernstein [Ber84] proved that any P F -invariant distribution on M at n,n (F ) must be GL n (F )-invariant when F is non-archimedean. Baruch [Bar03] proved that any P F -invariant eigendistribution (with respect to the center of the the universal enveloping algebra of gl n (F )) is GL n (F )-invariant when F is archimedean, which has been proved completely in [AG09b,SZ12]. It is expected that there is a more general phenomenon related to the mirabolic subgroup P F . Let H p,n−p = GL p (F ) × GL n−p (F ). Gurevich [Gur17] investigeted the role of the mirabolic subgroup P F of GL n (F ) on the symmetric variety GL n (F )/H p,n−p when F is non-archimedean. Then Gurevich proved that any H 1,n−1 ∩P Finvariant linear functional on an H 1,n−1 -distinguished irreducible smooth representation of GL n (F ) is also H 1,n−1 -invariant (see [Gur17, Theorem 1.1]). It is expected that it holds for all H p,n−p . The case when n − p = p + 1 has been verified in [Lu20] if F is non-archimedean (see [Lu20,Theorem 6.3]). Let P E denote the mirabolic subgroup of GL n (E). Then P E ∩ GL n (F ) = P F . Offen and Kemarsky proved that any P E ∩ GL n (F )-invariant linear functional on a GL n (F )-distinguished irreducible smooth representation of GL n (E) is also GL n (F )-invariant. (See [Off11, Theorem 3.1] for the p-adic case and [Kem15, Theorem 1.1] for the archimedean case.) This paper studies the role of the mirabolic subgroup P of GL 2n (F ) on the symmetric variety GL 2n (F )/GL n (E).
There is a natural group embedding GL 2n (F ) ֒→ GL 2n (E) such that each element in the image of GL 2n (F ) is of the form A B BĀ where A, B ∈ M at n,n (E) and the Galois action on M at n,n (E) is given by Let θ be an involution of GL 2n (F ) given by for g ∈ GL 2n (F ). Then the fixed points of θ in GL 2n (F ) coincide with GL n (E). Denote by gl 2n (F ) the Lie algebra of GL 2n (F ). Then any g in gl 2n (F ) is of the form where a, b ∈ M at n,n (E). Let p denote the Lie algebra of P which is given by a b bā : a = (a i,j ), b = (b i,j ) for i, j ∈ {1, 2, · · · , n} a n,j =b n,j for j ∈ {1, 2, · · · , n} .
The main result in this paper is the following: The geometry of closed GL n (E)-orbits on the symmetric space GL 2n (F )/GL n (E) is well known due to Guo [Guo97] and Carmeli [Car15]. Then we will use the Harish-Chandra descent techniques developed in [AG09a] to show the following identity of distributions (defined in Lemma 4.3), (1.1) will lead to a proof of Theorem 1.1. In fact, we will prove that any element in D(GL 2n (F )/GL n (E)) GLn−1(E) is invariant under transposition. Here we identify the symmetric variety GL 2n (F )/GL n (E) with the space of matrices and the transpose acts on X n . Thus the transpose acts on D(X n ) = D(GL 2n (F )/GL n (E)) as well.
Theorem 1.2. One has D(X n ) GL n (E)∩P = D(X n ) GL n (E) .
The key idea in the proof of Theorem 1.2 is to reduce a question on the distribution spaces of X n = GL 2n (F )/GL n (E) to that of distributions on its tangenet space.
We may identify the linear version of X n = GL 2n (F )/GL n (E) with the space of matrices x x for x ∈ M at n,n (E) (see [Guo97]), denoted by L n . Let C (L n ) denote the tempered generalized functions on L n . Let GL n (E) act on L n by the twisted conjugation, i.e., for A ∈ GL n (E) and let gl n (E) act on L n by its differential. More precisely, for a, x ∈ M at n,n (E). Then there is an analogue for the Lie algebra version.
Remark 1.4. Here we study the tempered generalized functions space C (L n ) instead of the generalized functions or distributions D(L n ) on L n because we will use the Fourier transform on C (L n ). Moreover, there is not so much difference between C (L n ) and D(L n ) due to [AG09a,Theorem 4 There is a brief introduction to the proof of Theorem 1.3. We will use the result of Aizenbud-Gourevitch (see Theorem 2.2) to reduce the problem on the tempered generalized functions supported on the nilpotent cone. If F is non-archimedean, then we will pick up a sl 2 (F )-triple {h, e, f } (see (3.1)) and use Chen-Sun's method [CS20] to study some special nilpotent orbits O ∋ e. If F = R, then we will use the machine of D-modules to show the vanishing theorem. Note that GL n−1 (E) is a proper subgroup of P E = GL n (E) ∩ P . It turns out that each GL n−1 (E)-invariant tempered generalized function on L n supported on O is invariant under transposition (see Theorem 3.1), which implies Theorem 1.3.
The paper is organized as follows. In §2, we introduce some notation from the algebraic geometry. Then we will use Chen-Sun's method (resp. the machine of D-modules) to prove Theorem 1.3 when F is nonarchimedean (resp. F = R) in §3. In §4.1, we will give a proof to Theorem 1.2. The proof of Theorem 1.1 will be given in §4.2.

Preliminaries and notation
Let X be an ℓ-space (i.e. locally compact totally disconnected topological spaces) if F is non-archimedean or a Nash manifold (see [AG09a]) if F = R. Let C (X) denote the tempered generalized functions on X. Let a reductive group G(F ) act on an affine variety X. Let x ∈ X such that its orbit G(F )x is closed in X. We denote the normal bundle by N X be the stalizer subgroup of x.
Theorem 2.1. [AG09a, Theorem 3.1.1] Let G(F ) act on a smooth affine variety X. Let χ be a character of G(F ). Suppose that for any closed orbit Gx in X, we have If X is a finite dimensional representation of G(F ), then we denote the nilpotent cone in X by Let Q G (X) := X/X G and R G (X) := Q(X) \ Γ(X).
Theorem 2.2. Let X be a finite dimensional representation of a reductive group G(F ). Let K ⊂ G(F ) be an open subgroup and let χ be a character of K. Suppose that for any closed orbit G(F )x such that Proof. See [AG09a, Corollary 3.2.2].
2.1. D-modules and singular support. In this subsection, assume that F = R. Let X be a Nash manifold. Denote by S(X) the space of Schwartz functions on X. Denote by C (X) the linear dual space to S(X), i.e. the tempered generalized functions on X. All the materials in this subsection come from [AG09a, AG09b, Aiz13].
2.1.1. Coisotropic variety. Let M be a smooth algebraic variety and ω be a symplectic form on it. Let Z ⊂ M be an algebraic subvariety. We call it M -coisotropic if T z Z ⊃ (T z Z) ⊥ for a generic smooth point z ∈ Z, where (T z Z) ⊥ denotes the orthogonal complement to T z Z in T z M with respect to ω. Note that every non-empty M -coisotropic variety is of dimension at least 1 2 dim M . For a smooth algebraic variety X, we always consider the standard symplectic form on the cotangent bundle T * X. Also, we denote by p X : T * X → X the standard projection.
Lemma 2.3. Let X be a smooth algebraic variety. Let a group G act on X which induces an action on T * X. Let S ⊂ T * X be a G-invariant subvariety. Then the maximal T * X-coisotropic subvariety of S is also G-invariety.
Let Y be a smooth algebraic variety. Let Z ⊂ Y be a smooth subvariety. Let R ⊂ T * Y be any subvariety. We define the restriction 2.1.2. Singular support. Let X be a smooth algebraic variety. Let D X denote the algebra of polynomial differential operators on X. Let GrD X be the associated graded algebra of D X . Then GrD X ∼ = O(T * X).
Let ξ ∈ C (X). Denoted by SS(ξ) the singular support of the right D X -module generated by ξ. Then Appendix B] for more details.) Let V be a vector space over F . Let B be a non-degenerate bilinear form on V . Then B defines Fourier transform with respect to the self-dual Haar measure on V , denoted by F V . For any Nash manifold M , we also denote by Proposition 2.6. (i) Let ξ ∈ C (X). Then the Zariski closure of Supp(ξ) is p X (SS(ξ)).
(ii) Let an algebraic group G act on X. Let g denote the Lie algebra of G.
2.1.3. Distributions on non-distinguished nilpotent orbits. Let V be an algebraic finite dimensional representation of a reductive group G. Let Γ(V ) be the nilpotent cone of V .
Gx,x ⊂ Γ(V * ). We will call a G-orbit G-distinguished if all its elements are G-distinguished.
In the case when G = GL n (R) and V = M at n,n (R) the set of G-distinguished elements is exactly the set of regular nilpotent elements.
Proposition 2.8. Let W := Q(V ) and let A be the set of non-distinguished elements in Γ(V ). Then there are no non-empty W × W * -coisotropic subvarieties of A × Γ(V * ).

A vanishing result of generalized functions
In this section, we shall prove Theorem 1.3. Let Let H n := GL n (E). DenoteH n := H n ⋊ σ where σ acts on H n by the involution The groupH n acts on L n by aā · x = axā −1 and σ · x =x t for x ∈ L n . Let χ be the sign character ofH n , i.e. χ| Hn is trivial and Let H n−1 be a natural subgroup of H n through the embedding g → g 0 0 1 for g ∈ GL n−1 (E). Then H n−1 is a proper subgroup of P E = P ∩ GL n (E).
Then Theorem 1.3 follows from Theorem 3.1 due to the fact that the subgroups P E and its transpose P t E generate the whole group GL n (E). Consider the decomposition ) For the proof of Theorem 3.1, we will prove the following.
Then Theorem 3.1 follows from Theorem 3.2. Define a non-degenerate symmetric F -bilinear form on gl 2n (F ) by z, w gl 2n (F ) := the trace of zw as a F -linear operator.
Note that the restriction of this bilinear form on L n is still non-degenerate. Fix a non-trivial unitary character ψ of F . Denote by F : C (L n ) −→ C (L n ) the Fourier transform which is normalized such that for every Schwartz function ϕ on L n , for z ∈ L n , where dw is the self-dual Haar measure on L n . If L n can be decomposed into a direct sum of two quadratic subspaces U 1 ⊕ U 2 such that each U i is non-degenerate with respect to −, − | Ui , then we may define the partial Fourier transform . Similarly for F U2 (ϕ). It is clear that the Fourier transform F intertwines the action ofH n−1 . Thus we have the following lemma. 3.1. Reduction within the null cone. Recall that Following [CS20, Proposition 3.9], we shall prove the following proposition when F is non-archimedean in this subsection.
Proposition 3.5. Let f be a H n−1 -invariant generalized function on L n−1 ⊕ V ⊕ V * such that f , its Fourier transform F(f ) and its partial Fourier transforms F V ⊕V * (f ), F Ln−1 (f ) are all supported on N n . Then f = 0.
Remark 3.6. We will postpone the proof of Proposition 3.5 when F = R until the next subsection, which involves the machine of D-modules.
Then e ∈ N . Moreover, we may assume that x ∈ M at n−1,n−1 (F ) due to Lemma 3.4. Recall that every e ∈ L n−1 can be extended to a sl 2 -triple {h, e, f } (see [KR71,Proposition 4]) in the sense that Let F * act on C (L n−1 ⊕ V ⊕ V * ) by The orbit O is invariant under dilation and so F × acts on C Nn (L n−1 ⊕ V ⊕ V * ) Hn−1 as well. Suppose that the sl 2 (F )-triple (3.1) integrates to an algebraic homomorphism Denote by D t the image of t t −1 in H n−1 ∩ GL 2n−2 (F ). Let be a closed subgroup in H n−1 × F × which fixes the element e. Define a quadratic form on V ⊕ V * as follow: The following lemma is similar to [CS20, Lemma 3.13].
Lemma 3.7. Let η be an eigenvalue for the action of Remark 3.8. There is a more general version of Lemma 3.7; see Theorem 3.10.
Proof. Consider the map 0 v for all non-negative integers k} for e = x 0 x 0 ∈ L n−1 . It is easy to see that the representation C {0} (L f n−1 ) of T is complete reducible and every eigenvalue has the form where γ is an eigenvalue for the action of T on C E(e) (V ⊕ V * ). In order to compute γ, we will restrict γ to a smaller subspace for ϕ ∈ S(V ⊕ V * ) and x ∈ V ⊕ V * . We may extend ω ψ from the Schwartz space S(V ⊕ V * ) to the tempered generalized function space C (V ⊕ V * ). Note that X X −1 = 1 n −X 1 n 1 n X −1 1 n 1 n 1 − X 1 n 1 n −1 n 1 n 1 n 1 n 1 n holds for any X ∈ GL n (F ). Here we only need the case that X is a diagonal matrix.
. Then the action of D t on V ⊕ V * is given by Thus D t does not contribute to γ. Therefore γ has the form and so η(t) 2 = |t| Consider M at n−1,n−1 (F ) as a representation of sl 2 (F )-triple (3.1). Decompose it into irreducible representations Lemma 3.9. Assume n ≥ 3. One has Therefore tr(2 − h)| L f n−1 − 2(n − 1) 2 = 2d > 3 due to the fact that d ≥ 2. Let Q be a quadratic form on L n−1 ⊕ V ⊕ V * defined by Theorem 3.10. Let I be a non-zero subspace of C Z(Q) (L n−1 ⊕ V ⊕ V * ) such that for every f ∈ I, one has that F(f ) ∈ I and (ψ • Q) · f ∈ I for all unitary character ψ of F . Then I is a completely reducible F × -subrepresentation of C (L n−1 ⊕ V ⊕ V * ), and it has an eigenvalue of the form | − | 1 2 dim F (Ln−1⊕V ⊕V * ) . Now we are prepared to prove Proposition 3.5 when F is non-archimedean.
Proof of Proposition 3.5 when F is non-archimedean. Denote by I the space of all tempered generalized functions f on L n−1 ⊕ V ⊕ V * with the properties in Proposition 3.5. Assume by contradiction that I is nonzero. If n − 1 = 1, i.e. n = 2, then and so Proposition 3.5 follows from [Aiz13, Lemma 6.3.4] that if there exists an such that both F Ln−1 (f ) and F V ⊕V * (f ) are supported on N n , then f = 0. Here the action of E × is given by where g ∈ E × , x, v, v * ∈ E. Assume that n ≥ 3. Then by Lemma 3.7 and Theorem 3.10, one has dim F (L n−1 ⊕ V ⊕ V * ) = tr(2 − h)| L f n−1 + 4n − 4 and so tr(2 − h)| L f n−1 = 2(n − 1) 2 which contradicts the equality (3.3). This finishes the proof.
3.2. Proof of Proposition 3.5 when F = R. This subsection focuses on the proof of Proposition 3.5 when F = R. We will follow [Aiz13, §6] to prove that SS(f ) is not coisotropic for any non-zero tempered generalized function f satisfying the conditions in Proposition 3.5, which implies that f must be zero.

Note that the orthogonality condition can be replaced by
Thus [Aiz13,Lemma 6.3.4], f must be zero. This finishes the proof.
Remark 3.12. One may follow [Aiz13, §6] to give a uniform proof for Proposition 3.5 which involves more techniques and more notation when F is non-archimedean. Lemma 3.13. [Car15, Theorem 6.15] Every descendant of the pair (GL 2n , R E/F GL n ) is a product of pairs of the form (R L1/F GL r , R L2/F GL r ), (GL r × GL r , △GL r ) and (GL 2r , R E/F GL r ) for some r < n, where L 2 is a finite field extension over F and L 1 is a quadratic extension of L 2 . Proof of Theorem 3.2. It is enough to show that Pick any nilpotent orbit O in N n . Thanks to Lemma 3.3 and Proposition 3.5, we have This finishes the proof.
Finally, we give the proof of Theorem 3.1.
Proof of Theorem 1.2. Recall that X n = GL 2n (F )/GL n (E). From the proof of Lemma 4.2, we obtain that Note that H n−1 ⊂ P E and that the mirabolic subgroup P E and its transpose P t E generate GL n (E). Thus one has D(X n ) PE = D(X n ) GL n (E) . This finishes the proof. 4.2. Proof of Theorem 1.1. This subsection focuses on the proof of Theorem 1.1. Let us recall the following lemma appearing in [Off11,Kem15].