The cycle polynomial of a permutation group

The cycle polynomial of a finite permutation group $G$ is the generating function for the number of elements of $G$ with a given number of cycles: \[F_G(x) = \sum_{g\in G}x^{c(g)},\] where $c(g)$ is the number of cycles of $g$ on $\Omega$. In the first part of the paper, we develop basic properties of this polynomial, and give a number of examples. In the 1970s, Richard Stanley introduced the notion of reciprocity for pairs of combinatorial polynomials. We show that, in a considerable number of cases, there is a polynomial in the reciprocal relation to the cycle polynomial of $G$; this is the orbital chromatic polynomial of $\Gamma$ and $G$, where $\Gamma$ is a $G$-invariant graph, introduced by the first author, Jackson and Rudd. We pose the general problem of finding all such reciprocal pairs, and give a number of examples and characterisations: the latter include the cases where $\Gamma$ is a complete or null graph or a tree. The paper concludes with some comments on other polynomials associated with a permutation group.

In particular, we see that if G contains no odd permutations, then F G (x) vanishes only at x = 0. However, for permutation groups containing odd permutations, there may be negative roots of F G . Proof F G (−1) = 0, since G and N have equally many orbits (namely 1) on colourings with a single colour. Now suppose that F G (−a) = 0, so that G and N have equally many orbits on colourings with a colours; thus every G-orbit is an N-orbit. Now every colouring with a − 1 colours is a colouring with a colours, in which the last colour is not used; so every G-orbit on colourings with a − 1 colours is an N-orbit, and so F G (−a + 1) = 0. The result follows.
The property of having a root −a is preserved by overgroups: Proposition 1.6 Suppose that G 1 and G 2 are permutation groups on the same set, with G 1 ≤ G 2 . Suppose that F G 1 (−a) = 0, for some positive integer a. Then also F G 2 (−a) = 0.
Proof It follows from the assumption that G 1 (and hence also G 2 ) contains odd permutations. Let N 1 and N 2 be the subgroups of even permutations in G 1 and G 2 respectively. Then N 2 ∩ G 1 = N 1 , and so N 2 G 1 = G 2 . By assumption, G 1 and N 1 have the same orbits on a-colourings. Let K be an a-colouring, and g ∈ G 2 ; write g = hg ′ , with h ∈ N 2 and g ′ ∈ G 1 . Now Kh and Khg ′ are in the same G 1 -orbit, and hence in the same N 1 -orbit; so there exists h ∈ N 1 with Kg = Khg ′ = Khh ′ . Since hh ′ ∈ N 2 , we see that the G 2 -orbits and N 2 -orbits on a-colourings are the same. Hence F G 2 (−a) = 0.
The cycle polynomial has nice behaviour under direct product, which shows that the product of having negative integer roots is preserved by direct product. Proposition 1.7 Let G 1 and G 2 be permutation groups on disjoint sets Ω 1 and In particular, the set of roots of F G is the union of the sets of roots of F G 1 and F G 2 .
Proof This can be done by a calculation, but here is a more conceptual proof. It suffices to prove the result when a positive integer a is substituted for x. Now a G-orbit on a-colourings is obtained by combining a G 1 -orbit on colourings of Ω 1 with a G 2 -orbit of colourings of Ω 2 ; so the number of orbits is the product of the numbers for G 1 and G 2 .
The result for the wreath product, in its imprimitive action, is obtained in a similar way.
Proof Again it suffices to prove that, for any positive integer a, the equation is valid with a substituted for x.
Let ∆ be the domain of H, with |∆| = m. An orbit of the base group G m on a-colourings is an m-tuple of G-orbits on a-colourings, which we can regard as a colouring of ∆, from a set of colours whose cardinality is the number F G (a)/|G| of G-orbits on a-colourings. Then an orbit of the wreath product on a-colourings is given by an orbit of H on these F G (a)/|G|-colourings, and so the number of orbits is (1/|H|)F H (F G (a)/|G|). Multiplying by |G ≀ H| = |G| m |H| gives the result. Corollary 1.9 If n is odd, m > 1, and G = S n ≀ S m , then F G (x) has roots −1, . . . , −n.
Substituting x = −n and recalling that n is odd, this is −n! + n! = 0.
The next corollary shows that there are imprimitive groups with arbitrarily large negative roots. Corollary 1.10 Let n be odd and let G be a permutation group of degree n which contains no odd permutations. Suppose that F G (a)/|G| = k. Then, for m > k, the polynomial F G≀Sm (x) has a root −a.
Proof By Proposition 1.3, F G (−a)/|G| = −k. Now for m > k, the polynomial F Sm (x) has a factor x + k; the expression for F G≀Sm shows that this polynomial vanishes when x = −a.
The main question which has not been investigated here is: What about non-integer roots?
It is clear that F G (x) has no positive real roots; so if G contains no odd permutations, then F G (x) has no real roots at all, by Proposition 1.3. Is there any restriction on where the non-real roots lie? For example computer calculations suggest that F An (a) = 0 implies Re(a) = 0, but we have been unable to prove this.
2 Some examples Proposition 2.1 For each n we have, Proof By induction and Proposition 1.6, F Sn (−a) = 0 for each 0 ≤ a ≤ n − 2. Since F Sn (x) is a polynomial of degree n and the coefficient of Several other proofs of this result are possible. We can observe that the number of orbits of S n on a-colourings of {1, . . . , n} (equivalently, ntuples chosen from the set of colours with order unimportant and repetitions allowed) is n+a−1 n . Or we can use the fact that the number of permutations of {1, . . . , n} with k cycles is the unsigned Stirling number of the first kind u(n, k), whose generating function is well known to be n k=1 u(n, k)x k = x(x + 1) · · · (x + n − 1).

Proposition 2.2 For each n we have
Proof Clear.

Proposition 2.3
Let p be an odd prime and G be the group P GL 2 (p) acting on a set of size p + 1. Then F G (x) is given by Proof We count the elements in G of order k. When k = 2, there are two conjugacy classes of involutions of lengths p(p−1) 2 and p(p+1) 2 with the first class consisting of elements which act fixed-point freely and the second consisting of elements which fix exactly two points. For ǫ = ±1, and each non-trivial odd divisor k of p + ǫ, there are Finally, there is a single conjugacy class of elements of order p of length p 2 −1.
Putting this altogether, we obtain Simplifying gives the result.

Reciprocal pairs
Richard Stanley, in a 1974 paper [5], explained (polynomial) combinatorial reciprocity thus: A polynomial reciprocity theorem takes the following form. Two combinatorially defined sequences S 1 , S 2 , . . . andS 1 ,S 2 , . . . of finite sets are given, so that the functions f (n) = |S n | and f (n) = |S n | are polynomials in n for all integers n ≥ 1. One then We will see that, in a number of cases, the cycle polynomial satisfies a reciprocity theorem.

The orbital chromatic polynomial
First, we define the polynomial which will serve as the reciprocal polynomial in these cases. A (proper) colouring of a graph Γ with q colours is a map from the vertices of Γ to the set of colours having the property that adjacent vertices receive different colours. Note that, if Γ contains a loop (an edge joining a vertex to itself), then it has no proper colourings. Birkhoff observed that, if there are no loops, then the number of colourings with q colours is the evaluation at q of a monic polynomial P Γ (x) of degree equal to the number of vertices, the chromatic polynomial of the graph. Now suppose that G is a group of automorphisms of Γ. For g ∈ G, let Γ/g denote the graph obtained by "contracting" each cycle of g to a single vertex; two vertices are joined by an edge if there is an edge of Γ joining vertices in the corresponding cycles. The chromatic polynomial P Γ/G (q) counts proper q-colourings of Γ fixed by g. If any cycle of g contains an edge, then Γ/g has a loop, and P Γ/g = 0. Now (with a small modification of the definition in [3]) we define the orbital chromatic polynomial of the pair (Γ, G) to be The Orbit-Counting Lemma immediately shows that P Γ,G (q)/|G| is equal to the number of G-orbits on proper q-colourings of Γ. Now, motivated by Stanley's definition, we say that the pair (Γ, G), where Γ is a graph and G a group of automorphisms of Γ, is a reciprocal pair if where n is the number of vertices of Γ.
Problem Find all reciprocal pairs. This problem is interesting because, as we will see, there are a substantial number of such pairs, for reasons not fully understood. In the remainder of the paper, we present the evidence for this, and some preliminary results on the above problem.
A basic result about reciprocal pairs is the following. Proof Whitney [7] showed that the leading terms in the chromatic polynomial of a graph Γ with n vertices and m edges are Equating the two expressions gives m = t(G) + t 0 (G), as required.
We remark that the converse to Lemma 3.1 does not hold: consider the group G = S 3 ≀ S 3 acting on 3 copies of K 3 (see Proposition 1.8: but note that (3K 3 , S 3 ≀ C 3 ) is a reciprocal pair, by Proposition 3.6). We also observe the following corollary to Lemma 3.1.

edges.
Proof If Γ has n 2 − δ edges then by Lemma 3.1, It is well-known that a permutation group of degree n containing at least n−1 2 + 1 transpositions must be the full symmetric group. But this implies that Γ is a complete graph, a contradiction. According to Lemma 3.1, if Γ is not a null graph and (Γ, G) is a reciprocal pair, then G contains transpositions. Now as is well known, if a subgroup G of S n contains a transposition, then the transpositions generate a normal subgroup N which is the direct product of symmetric groups whose degrees sum to n. (Some degrees may be 1, in which case the corresponding factor is absent.) A G-invariant graph must induce a complete or null graph on each of these sets. Moreover, between any two such sets, we have either all possible edges or no edges.
Suppose that n 1 , . . . , n r are the N-orbits carrying complete graphs and m 1 , . . . , m s the orbits containing null graphs, then Lemma 3.1 shows that the total number of edges of the graph is The first term counts edges within N-orbits, so the second term counts edges between different N-orbits. (b) Let Γ be a complete graph, and G a subgroup of the symmetric group S n . Then P Γ,G (x) = x(x + 1) · · · (x + n − 1), independent of Γ.

Examples
Proof (a) The chromatic polynomial of a null graph on n vertices is x n . So, if g ∈ G has c(g) cycles, then Γ/g is a null graph on c(g) vertices. Thus (b) Suppose that Γ is a complete graph. Any proper colouring of Γ has all colours disjoint, so any permutation group G on proper a-colourings is semiregular, and has a(a − 1) · · · (a − n + 1)/|G| orbits. Thus the orbital chromatic polynomial of G is x(x − 1) · · · (x − n + 1), independent of G.

Corollary 3.4 (a) If Γ is a null graph, then (Γ, G) is a reciprocal pair if and only if G contains no odd permutations. (b) If Γ is a complete graph, then (Γ, G) is a reciprocal pair if and only if G is the symmetric group.
Proof (a) This follows from Proposition 1.3.
(b) We saw in the preceding section that, if G = S n , then F G (x) = x(x + 1) · · · (x + n − 1). Thus, we see that (G, Γ) is a reciprocal pair if and only if G is the symmetric group.
Proposition 3.5 Let Γ be the disjoint union of graphs Γ 1 , . . . , Γ r , and G the direct product of groups G 1 , . . . , G r , where G i ≤ Aut(Γ i ). Then In particular, if (Γ i , G i ) is a reciprocal pair for i = 1, . . . , r, then (Γ, G) is a reciprocal pair.
The proof is straightforward; the last statement follows from Proposition 1.7. The result for wreath products is similar: Proof Given q colours, there are P ∆,G (q)/|G| orbits on colourings of each copy of ∆; so the overall number of orbits is the same as the number of orbits of H on an m-vertex null graph with P ∆,G (q)/|G| colours available. For the last part, the hypotheses imply that P ∆,G (q) = (−1) n F G (−q), and that the degree of each term in F H is congruent to m mod 2. So the expression evaluates to |G| m F H (F G (−x)/|G|) if either m or n is even, and the negative of this if both are odd; that is, We now give some more examples of reciprocal pairs. Example 1 Let Γ be a 4-cycle, and G its automorphism group, the dihedral group of order 8. There are 4 edges in Γ, and 2 transpositions in G, each of which interchanges two non-adjacent points (an opposite pair of vertices of the 4-cycle); so the equality of the lemma holds. Direct calculation shows that so P Γ,G (x) = (−1) n F G (−x) holds in this case. The 4-cycle is also the complete bipartite graph K 2,2 . We note that, for n > 2, (K n,n , S n ≀ S 2 ) is not a reciprocal pair. This can be seen from the fact that F Sn≀S 2 (x) has factors x, x + 1, . . . , x + n − 1, whereas K n,n has chromatic number 2 and so x − 2 is not a factor of P Kn,n,G (x) for any G ≤ S n ≀ S 2 . Also (K n,m , S n × S m ) is not a reciprocal pair for n, m > 2 since the equality of Lemma 3.1 is easily seen to have no solutions in this range.
We do not know whether other complete multipartite graphs support reciprocal pairs. Example 2 Let Γ be a path with 3 vertices and G its automorphism group which is cyclic of order 2. Direct calculation shows that This graph is an example of a star graph, for which we give a complete analysis in the next section.
Example 3 Let Γ be the disjoint union of K m and N n together with all edges in between and set G = S m × S n ≤ Aut(Γ). Then Γ has m 2 + mn edges and G has m 2 + n 2 transpositions of which n 2 correspond to nonedges in Γ. Thus, according to Lemma 3.1 we need n = m + 1. Now the only elements g ∈ G which give non-zero contribution to P Γ,G (x) lie in the S n component. We get: Using Proposition 3.3(b) and m = n − 1 this becomes which is equal to −F G (−x) by Proposition 1.7.

Reciprocal pairs containing a tree
In this section we show that the only trees that can occur in a reciprocal pair are stars, and we determine the groups that can be paired with them. Γ is a tree and (Γ, G) is a reciprocal pair. Let n be the number of vertices in Γ and assume n ≥ 3. The following hold: (a) n is odd;

Conversely any pair (Γ, G) which satisfies conditions (a)-(c) is reciprocal.
In what follows we assume the following: • Γ is a tree; • (Γ, G) is a reciprocal pair; • n is the number of vertices in Γ and n ≥ 3.
Note that any two vertices interchanged by a transposition are nonadjacent. For suppose that a transposition flips an edge {v, w}. If the tree is central, then there are paths of the same length from the centre to v and w, creating a cycle. If it is bicentral, then the same argument applies unless {v, w} is the central edge, in which case the tree has only two vertices, a contradiction.

Lemma 4.2 If n is odd then (Γ, G) is a reciprocal pair if and only if
(1) Proof The chromatic polynomial of a tree with r vertices is easily seen to be x(x − 1) r−1 . Hence for each g ∈ G and then Rearranging (and using that n is odd) yields the Lemma. Lemma 4.3 G has n−1 2 transpositions; in particular, n is odd. Proof As in Lemma 3.1, let t(G) be the number of transpositions in G and t 0 (G) be the number of transpositions (i, j) in G for which i ∼ j in Γ. If G fixes an edge (u, v) of Γ then (u, v) ∈ G implies n = 2, a contradiction. Thus (u, v) / ∈ G, and every transposition in G is a non-edge. Hence t 0 (G) = t(G) so 2t(G) = n − 1 by Lemma 3.1.
Proof The transpositions in a permutation group G generate a normal subgroup H which is a direct product of symmetric groups. If there are two non-disjoint transpositions in G, one of the direct factors is a symmetric group with degree at least 3, and hence F H (x) has a root −2 by Propositions 2.1 and 1.7. Then by Proposition 1.6, F G (x) has a root −2. By Lemma 4.2 with x = 2, a contradiction. So the transpositions are pairwise disjoint, and generate a subgroup (C 2 ) k with n = 2k + 1 by Lemma 4.3. Thus the conclusion of the lemma holds.

Lemma 4.5 Γ is a star;
Proof Let v be the unique fixed point of G. By Lemma 4.4, for each u = v there exists a unique vertex u ′ with (u, u ′ ) ∈ G. This is possible only if each u has distance 1 from v. Hence Γ is a star.
Proof [Proof of Theorem 4.1] (a),(b) and (c) follow from Lemmas 4.3, 4.5 and 4.4 respectively. Conversely, suppose that (a),(b) and (c) hold. Then G = C 2 ≀ K for some permutation group K of degree k. By Proposition 1.8, so that (1) holds and we deduce from Lemma 4.2 that (Γ, G) is a reciprocal pair. Our proof is complete.
Given a set of reciprocal pairs (Γ 1 , G 1 ), . . . (Γ m , G m ) with with each Γ i a star we can take direct products and wreath products (using Propositions 3.5 and 3.6) to obtain reciprocal pairs (Γ, G) with Γ a forest of stars. We do not know whether all such pairs arise in this way.

Connection with other polynomials
The cycle index of a permutation group G of degree n is a polynomial in n variables which keeps track of all the cycle lengths of elements, not just the total number of cycles. If the variables are s 1 , . . . , s n , then the cycle index is given by where c i (g) is the number of cycles of length i in the cycle decomposition of G. (It is customary to divide this polynomial by |G|; but, for compatibility with our earlier polynomial and comparison with other polynomials we do not do so here.) Clearly the cycle polynomial is given by F G (x) = Z G (x, x, . . . , x).
Harden and Penman studied the fixed point polynomial P G (x) of a permutation group G (the generating function for fixed points of elements of G), given by P G (x) = Z G (x, 1, . . . , 1).
Another related construction is the Parker vector of G, usually presented as an n-tuple rather than a polynomial: its kth entry is obtained by differentiating Z G with respect to s k , and putting all variables equal to 1 (and dividing by |G|). It is relevant to computing the Galois group of an integer polynomial. See [2, Section 2.8].
In [3], the orbital chromatic polynomial is extended to an orbital Tutte polynomial, for graphs or (more generally) representable matroids. It is not clear whether this is related to the cycle polynomial.
Also, the cycle index of finite permutation groups can be extended to oligomorphic (infinite) permutation groups, see [2,Section 5.7]. However, the specialisation which gives the cycle polynomial fails to be defined in the infinite case.