Computing normalisers of intransitive groups

The normaliser problem takes as input subgroups $G$ and $H$ of the symmetric group $S_n$, and asks one to compute $N_G(H)$. The fastest known algorithm for this problem is simply exponential, whilst more efficient algorithms are known for restricted classes of groups. In this paper, we will focus on groups with many orbits. We give a new algorithm for the normaliser problem for these groups that performs many orders of magnitude faster than previous implementations in GAP. We also prove that the normaliser problem for the special case $G=S_n$ is at least as hard as computing the group of monomial automorphisms of a linear code over any field of fixed prime order.


Introduction
Given generators for subgroups G and H of the symmetric group S n , the normaliser problem asks one to compute a generating set for the normaliser N G (H).The fastest known bound for this problem, in general, is Wiebking's simply exponential bound 2 O(n)  [20].Better bounds are known for various cases: for example, quasipolynomial 2 O(log 3 n) if N Sn (H) is primitive [16,5], and polynomial if G has restricted composition factors by work of Luks and Miyazaki [13].In terms of practical computation, there are many algorithms to solve special cases of the normaliser problem (see [8,11,14] for examples).
As a consequence of Babai's quasipolynomial solution to the string isomorphism problem [1], the intersection of permutation groups can be computed in quasipolynomial time.So, with a quasipolynomial cost, it suffices to compute N Sn (H), then N G (H) = N Sn (H) ∩ G. Furthermore, in practice, computing intersections is much faster than computing normalisers.We shall therefore focus on the following problem.
To better understand its worst-case complexity, it is helpful to study the case where the problem seems to be the hardest.In this paper, we will consider intransitive groups, for which Norm-Sym is not known to be solvable in quasipolynomial time.Since N Sn (H) may only permute permutation isomorphic H-orbits, the following case seems likely to be hardest.Definition 1.2.Let A ≤ S m be transitive.Let InP(A) consist of all groups H ≤ S mk , for any k, such that H has orbits O = {Ω 1 , Ω 2 , . . ., Ω k } and each restriction H| Ω i is permutation isomorphic to A.
In particular, we will consider InP(A) with A the regular representation of the cyclic group C p or the natural representation of the dihedral group D 2q of order 2q, where p and q are primes and q is odd.
Luks proved that the graph isomorphism problem is polynomial-time reducible to Norm-Sym [12], which is a special case of the normaliser problem.However, not much more is known about where Norm-Sym fits into the complexity hierarchy.In particular, we do not know its relation to the intersection problem, or whether it is polynomial-time equivalent to the general normaliser problem.In 1993, Luks asked [12] for a decision problem polynomial-time equivalent to Norm-Sym, but this remains open.Notably, we do not know if Conj-Sym, the problem of deciding if subgroups H 1 and H 2 of S n are conjugate in S n , is polynomial-time equivalent to Norm-Sym.
Let MAut be the problem of computing the monomial automorphism group of a linear code C, and let MEq be the problem of deciding if linear codes C 1 and C 2 are monomially equivalent (see Definition 3.4 for details).Petrank and Roth showed that MEq for codes of length k over F p is at least as hard as the graph isomorphism problem [15], and MEq has time complexity bounded by 2 O((p−1)k) , assuming constant time field operations [17,2].Our first main complexity result is the following.
Theorem 1.3.For a fixed prime p, Norm-Sym and Conj-Sym for groups in InP(C p ) are polynomial-time equivalent to MAut and MEq, respectively, for codes over F p .
As a consequence, Norm-Sym is at least as hard as MAut for codes over any fixed field of prime order (see Corollary 3.6).
As Wiebking's simply exponential algorithm [20] is not implemented, the fastest implemented algorithm to compute N Sn (H) has a runtime complexity of 2 O(n log n) .Using methods based on the work of Feulner [6], we shall bound the complexity of Norm-Sym for H ∈ InP(C p ) by 2 O( n p log n p +log n) .We show that we may also bound the depth of the search tree (see Section 2.1) in S n by n/(2p), using dual codes.Our second main complexity result will then follow immediately from Proposition 4.2 and Theorem 4.10.Current practical methods to solve Norm-Sym for groups in InP(C p ) are very slow.The implementation in the computer algebra system GAP [7] struggles to compute N Sn (H) even when H has very small degree (median time of more than 10 minutes for degree around 30, see Section 8).This paper also develops an effective practical algorithm to solve Norm-Sym for groups in InP(C p ), using the above ideas.In Section 8, we provide evidence that our new algorithm performs far better than the one currently implemented in GAP.
The paper structure is as follows.In Section 2, we give some background and present general results for computing N Sn (H) for intransitive groups H.In Section 3, we prove Theorem 1.3 and in Section 4, we shall prove Theorem 1.4.In Section 5, we present several techniques for speeding up a backtrack search to compute N Sn (H) for H ∈ InP(C p ).We will describe our algorithm to compute N Sn (H) for such H in Section 6.In Section 7, we extend our methods to class InP(D 2q ), where q is an odd prime.Finally, in Section 8, we present runtime data.

Background and preliminaries
In this section, we consider Norm-Sym for an arbitrary intransitive group H ≤ S n with orbits O = {Ω 1 , Ω 2 , . . ., Ω k }.In Section 2.1, we give some preliminary complexity results and present a short overview of normaliser computation using backtrack search.In Section 2.2, we discuss equivalent orbits, then in Section 2.3, for a transitive group A, we construct a natural subgroup L of S n containing N Sn (H) for H ∈ InP(A).

Permutation group algorithms
In this section, let G ≤ S n be given by a generating set X and let Ω = {1, 2, . . ., n}.It is well known that in time O(|X|n 2 + n 5 ), we may replace X by a generating set of size at most n (see, for example, [18]).Therefore, we assume that |X| ≤ n and we measure the complexity of permutation group problems in terms of n.
A base We shall assume that all homomorphisms are given by generator images.
The following results are standard and we refer the reader to [9,18] for further details.
1.In polynomial time, we can: compute the orbits of G; compute the order |G|; compute the restriction G| ∆ of G to a given G-invariant set ∆; compute a base for G; compute the centraliser C Sn (G).
2. Given a permutation group K, a homomorphism φ : G → K, and permutations g ∈ G and k ∈ Im(φ), in polynomial time we can compute the image φ(g) and a pre-image of k.
We compute normalisers using backtrack search (see, for example, [9,18] for more details).Suppose that we want to compute N G (H).The search tree T of G with respect to B is a rooted tree of depth m, where the root node is labelled by the empty tuple ( ) and nodes at depth d are labelled with elements of the orbit (β 1 ,β 2 , . . .,β d ) G such that each node (α 1 ,α 2 , . . .,α d ) has parent (α 1 ,α 2 , . . .,α d−1 ).We can associate each node of T with a coset of a point stabiliser in G by defining We traverse the search tree T depth-first and gather the elements of N G (H) that we find in a group N , which is updated as search progresses.
When solving problems with backtrack search, the runtime is correlated to the number of nodes in the search tree we visit.We reduce this number in two ways.Firstly, if there exists a proper subgroup S of G containing N G (H), we search in the tree of S with respect to a base of S instead.We will describe how we can find such an S for H ∈ InP(C p ) ∪ InP(D 2p ) in Section 2.3.Secondly, if we can deduce for some node τ of T that Ψ(τ ) ∩ N G (H) is empty, or that all elements of Ψ(τ ) ∩ N G (H) is contained in the group N of normalising elements we have already found, then we can skip traversing the subtree rooted at τ .We call this skipping pruning T .
Methods for deducing that Ψ(τ ) ∩ N G (H) is a non-empty subset of N are known, see for example [18,Section 9.1], but we present new, efficient, techniques for deducing that Ψ(τ ) ∩ N G (H) = ∅ for H ∈ InP(C p ) in Sections 4 and 5.We will also use the following elementary lemma (whose proof is clear) to deduce that Throughout the paper, for all subsets ∆ of Ω, we shall consider subgroups of Sym(∆) as subgroups of Sym(Ω) with support ∆.

Equivalence of orbits
In this subsection, we define an equivalence relation on the set O of H-orbits, and show how it is used in centraliser and normaliser computation.
We say that ψ witness the equivalence.
For Ω i and Ω j in O and a bijection ϕ : Ω i → Ω j , we denote by ϕ the involution in Sym(Ω i ∪Ω j ) such that α ϕ = ϕ(α) for all α ∈ Ω i .Hence Ω i and Ω j are equivalent if and only if there exists In particular, the elements ψ ij and C Sn (H) can be computed in polynomial time.
By Lemma 2.2, we can deduce that there are no normalising elements in a subtree of T by showing that some restrictions or stabilisers are not conjugate in S n .We can use orbit equivalence to show that subgroups of S n are not conjugate.Lemma 2.5.Let R and Q be subgroups of S n such that Q = R σ for some σ ∈ S n , and let B be an ≡ R -class of orbits.Then {∆ σ | ∆ ∈ B} is an ≡ Q -class.Hence, with H as in Lemma 2.4, the group N Sn (H) acts on {B 1 , B 2 , . . ., B a }.
Proof.If ∆ and Γ are equivalent R-orbits, then by (2) there exists an involution µ in Sym(∆∪Γ) such that r| Γ = (r| ∆ ) µ for all r ∈ R. Let q ∈ Q.Then there exists r ∈ R such that r σ = q.So Next, we show that if ≡ H is not the equality relation then the computation of N Sn (H) reduces in polynomial time to computing the normaliser of a group H 1 of a smaller degree for which ≡ H 1 is equality.For a partition P of a set S, we denote by [s], or [s] P if we wish to emphasise P , the cell of P containing s.For s, s ∈ S, we write s ∼ s (or s ∼ P s ) to mean that s and s are in the same cell of P , so [s] = [s ].
Proposition 2.6.Norm-Sym for H ≤ S n reduces in polynomial time to the special case where the H-orbits are pairwise inequivalent.
Since the permutations ψ ij and C Sn (H) can be computed in polynomial time by Lemma 2.4, the complexity claim is immediate.
Hence in much of the remainder of the paper, we shall assume that the ≡ H -relation is trivial.

Normalisers of groups in InP(A): overgroups
We now assume that there exists an integer m and a transitive subgroup , where we identify the direct product with the corresponding subgroup of S n .We call G the enveloping group of H.
Next we define a subgroup L ≤ S n , analyse its structure and show that L contains N Sn (H).
Lemma 2.8.Then κ setwise stabilises each Ω i .Fix i, let α ∈ Ω i , and consider κ as a word in the φ j .Since Ω i is moved only by φ i , if φ i does not occur in κ then α is fixed by κ, so assume otherwise.As α κ is also in Ω i , there exists a subword κ of κ such that α κ = α φ i κ φ i , where α φ i and α φ i κ are in Ω 1 .Now since each Ω j is moved only by φ j , there exist (not necessarily distinct) integers We first show that L is imprimitive with block system O.Let α ∈ Ω i and β ∈ Ω j .Then α φ i and β φ j are points in Ω 1 , so there exists g ∈ G ≤ L such that α φ i g = β φ j , and hence L is transitive.Since B and K preserve O, it follows that O is a block system for L.
Consider the kernel J of the action of L on O. Since the action of [10, §11], so this follows from Part 2. Part 4: The element l induces a permutation σ of the set O of orbits.Then κ = ξ −1 (σ) can be computed in polynomial time by Lemma 2.1.2,and lκ −1 fixes each H-orbit, so is in B.

InP(C p ) and automorphisms of codes
In this section, let p be prime.For the rest of this section, we shall assume that the following hypothesis holds.
Let G be the enveloping group of H and let g 1 be a permutation in Sym(Ω 1 ) generating G 1 .For 1 ≤ i ≤ k, let φ i : Ω 1 → Ω i witness the permutation isomorphism between G 1 and G i , and let g i = g φ i 1 .
We now set up an isomorphism γ from H to a linear code.Then, in Section 3.1, we shall prove that computing N Sn (H) is polynomial-time equivalent to computing the monomial automorphism group of γ(H).
Denote the set of all s × k matrices over the field F p by M(s, k, p).For M ∈ M(s, k, p), we denote by M i, * and M * ,j the i-th row and the j-th column of M respectively.For a tuple I of distinct elements of {1, 2, . . ., s}, we denote by M I, * the matrix of dimension |I| × k such that the i-th row of M I, * is M I i , * , and similarly for columns.We denote the row space of M by M .
A linear code C over F p is a subspace of F k p for some k, denoted by for some s whose rows form a basis for C. We shall assume that all linear codes are given by generator matrices.The matrix M is in standard form if its first s columns form the identity matrix I s .For more information on linear codes, refer to, for example, [19].Since throughout the paper we shall be moving between exponential and additive notation, we shall identify elements of F p with integers {0, 1, . . ., p − 1}.We denote by F * p the multiplicative group of F p .Definition 3.2.Let γ be the isomorphism from G to F k p defined by p , so γ(H) is linear code of length k and dimension s over F p .Our assumption that |(H| ∪ i≤s Ω i )| = p s means that no column permutations are needed to put a generator matrix of γ(H) in standard form, so let M be such a generator matrix.
We show that, in polynomial time, we can decide if a given subgroup of S n is in InP(C p ), and if so compute some accompanying data.
, in polynomial time, we can compute: an ordering Ω 1 , Ω 2 , . . ., Ω k of O, the bijections φ i and the generators g i from Hypothesis 3.1; the isomorphism γ from Definition 3.2; and a generator matrix The ordering of the H-orbits can be obtained in polynomial time by Lemma 2.1.1.Choices for g i can be computed in polynomial time since any non-trivial restriction x| ∆ of x ∈ X generates H| ∆ .Since conjugacy of permutations in symmetric groups can be computed in polynomial time, so can the bijections φ i .We construct γ by mapping each g i to the i-th standard basis vector of F k p .Finally, M can be obtained by finding a row reduced basis for γ(x)

The normaliser of H as an automorphism group of a linear code
In this subsection we prove Theorem 1.3, but first we define the actions of certain subgroups of GL k (p) on γ(H).
Let D and P be the groups of all diagonal and permutation matrices in GL k (p), respectively, and let W := D, P .The natural action of W on p are monomially equivalent if Cw = C for some w ∈ W (we denote vector-matrix multiplication by concatenation).
p , and hence show that computing N Sn (H) is polynomial-time equivalent to computing MAut(γ(H)).Lemma 3.5.Define a homomorphism ρ : K → P by ρ(κ) i,j = 1 if and only if Then the following statements hold.
1. Ξ is an epimorphism with kernel G.
4. Given l ∈ L and w ∈ W , we can compute Ξ(l) and a preimage of w under Ξ, in time polynomial in n.
) κ .Recall the involutions φ i ∈ Sym(Ω 1 ∪ Ω i ) from Hypothesis 3.1.Since K is generated by the φ i and conjugation by φ i swaps g 1 and g i , it follows that g κ i = g j .So (g l )| Ω j = g r i d i j and γ(g l ) j = r i d i .Hence for all j, , and j such that Ω κ i = Ω j .As k ≤ n, the images ζ(b) and ρ(κ), and hence Ξ(l), can be computed in time polynomial in n.
To show that we can find an element with Ξ-image w in polynomial time, for 1 ≤ i ≤ k, let d i be the non-zero entry of w i, * , and let Now we find an element σ of S n such that σ −1 g i σ = g d i i , in time polynomial in n since the g i have disjoint supports and conjugation in Sym(Ω i ) can be solved in time polynomial in Next, in time polynomial in k, we construct the element σ of S k such that the image i σ is the position of the non-zero entry in q i, * .Letting ξ be as in the proof of Lemma 2.8.1, κ := ξ −1 (σ) is the element of K with ρ(κ) = q, which can be computed in time polynomial in n by Lemma 2.1.2.Therefore, in time polynomial in n, we can compute an element σκ of L with Ξ-image w.For the backward reduction, let C ≤ F k p be a linear code given by a generator matrix M ∈ M(s, k, p).Let g i = (p(i − 1) + 1, . . ., pi) ∈ S pk for 1 ≤ i ≤ k and let G = g 1 , g 2 , . . ., g k .Let γ be as in Definition 3.2 and let H = γ −1 (M i, * ) | 1 ≤ i ≤ s .Assume that we can compute a generating set Y for N S pk (H) in time polynomial in k.Then by Lemma 3.5.3,Proof.The graph isomorphism problem is polynomial-time reducible to MEq for p-ary codes [15], which is polynomial-time reducible to MAut p by Theorem 1.3.The result now follows as Norm-Sym for InP(C p ) is a special case of Norm-Sym.

Complexity results
Let H be as in Hypothesis 3.1, and let L = B K be as in Lemma 2.8.In Section 4.1, we show that N Sn (H) can be computed in time 2 O( n 2p log n) .In Section 4.2, we show that for each κ ∈ K, if there exists b ∈ B such that bκ ∈ N Sn (H), then we can find such a b in polynomial time.In Section 4.3, we first show that N B (H) can be computed in polynomial time, then we see how the results in this section come together to reduce the search space for N Sn (H) to searching in K ∼ = S k instead of L ∼ = (C p C p−1 ) S k , and hence prove Theorem 1.4.

Limiting the depth of the search tree
In this subsection, we show how we can reduce the depth of the search tree using a possibly smaller group H ⊥ , which we define now. The where '•' denotes the standard dot product.Let H ⊥ ≤ S n be γ −1 (γ(H) ⊥ ).We shall show that there exists a bijection between N Sn (H) and N Sn (H ⊥ ).Recall from Lemma 2.8 that N Sn (H) ≤ L. Proof.Let the epimorphism Ξ : L → W be as in Lemma 3.5.
Since P permutes the coordinates of F k p , the product γ(g)Ξ(l) • γ(η)Ξ(b −1 κ) is also 0. Now by Lemma 3.5.2,Proof.Notice that |H ⊥ | = p k−s , so m is the minimum of log p (|H|) and log p (|H ⊥ |).We shall show that N Sn (H) can be computed in time 2 O((s+1) log n) .Then if s > k/2, using Lemmas 2.8.4 and 4.1, we may compute N Sn (H) in time 2 O((m+1) log n) by computing N Sn (H ⊥ ) instead.From here, since m ≤ k/2, the last assertion will follow.
By Proposition 2.6, without loss of generality, we may assume that ≡ H is the equality relation.By Lemma 3.3, in polynomial time, we can check that H ∈ InP(C p ), and obtain a generator matrix M of γ(H) in standard form.If w ∈ W is in MAut(γ(H)), then M := M w satisfies M = M .Since M is in standard form, the elements of M are uniquely determined by their first s coordinates.So to test whether a given w ∈ W is in MAut(γ(H)), it suffices to describe which columns of M are mapped onto the first s columns of M and how they are scaled, then the rest of the action of w is determined.We find all such w For each such J and v, let M be the partially defined matrix with Then each partial row of M extends to a unique vector in M , which must be the corresponding row of M .If w ∈ MAut(γ(H)) then (M w) * ,j is a scalar multiple of a column of M .Therefore we test whether J and v yield an element of MAut(γ(H)) by testing, for all j > s, if there exists a If for some j there are no such M * ,j κ −1 and d j κ −1 , then this choice of J and v does not extend to an element of MAut(γ(H)), and we move on to the next.Note that since the H-orbits are pairwise inequivalent, for each j, there are at most one possible choice for j κ −1 .If we have succeeded in choosing j κ −1 and d j κ −1 for all j, then and so w = diag(d 1 κ −1 , . . ., d k κ −1 )ρ(κ) ∈ MAut(γ(H)).There are k(p − 1) ≤ n choices of d j κ −1 M * ,j κ −1 for each j ∈ {s + 1, . . ., k}, so for each J and r, this step can be done in polynomial time.
Let Y be the set of all elements w ∈ MAut(γ(H)) found as above.We claim that For the complexity claim, since there are O(k s (p − 1) s ) ∈ O(n s ) choices for J and v, the set Y can be computed in time 2 O(s log n+log n) .Now by Lemma 3.5.4,the set Ξ −1 (Y ) can be computed in 2 O((s+1) log n) time, and by Lemma 2.1.2,C Sn (H) can be computed in time 2 O(log n) .Therefore N Sn (H) can be computed in time 2 O((s+1) log n) .
While searching for elements of MAut(γ(H)) in W , the above result limits the depth of the search tree.In practice, we search for MAut(γ(H)) in P ∼ = S k , as we shall see in Section 4.3.

Normalising elements that act non-trivially on O
Let L = B K be as in Lemma 2.8 and recall that N Sn (H) ≤ L. In Proposition 4.6, we show that given a κ ∈ K, we can decide in polynomial time if there exists an element of N Sn (H) which induces the same permutation as κ on the set O of H-orbits.We shall decide if M ∈ M F by computing certain F -orbit representatives, which we now define.For A, A ∈ M(s, k, p), let A R * ,i denote A * ,i reversed.We define A ≺ A if there exists a j such that A * ,i = A * ,i for 1 ≤ i < j and A R * ,j < lex A R * ,j .We choose the representative of A F to be the least element under the ordering ≺.
Feulner gives an algorithm to compute such F -orbit representatives [6, Algorithm 1].Since we will be proving its complexity, we will present our minor variation on the algorithm here.A key ingredient is the support partition, which we define next.We denote the tuple (1, 2, . . ., i) by i.
Definition 4.4.Let M ∈ M(s, k, p) be in standard form.For 1 ≤ j ≤ k, the support partition Q j is the finest partition of {1, 2, . . ., s} such that for 1 ≤ i ≤ j, there exists a cell Q of Q j that contains Supp(M * ,i ).
We present a simplified version of [6, Algorithm 1] as Algorithm 1.For 1 ≤ j ≤ k, let D [j] be the subgroup of D consisting of matrices of the form diag(d 1 , d 2 , . . ., d j , 1, . . ., 1), where d i ∈ F * p .Algorithm 1 iteratively computes the orbit representative under GL j (p) × D [j] , for increasing values of j.Theorem 4.5.Let A be a matrix in M(s, k, p) in standard form.Then, assuming constant time field operations, Algorithm 1 returns the representative of A F in time polynomial in k.
Proof.Algorithm 1 and [6, Algorithm 1] are identical, except: Feulner considers codes over arbitrary finite fields, but we restrict to fields of prime order; Feulner's algorithm performs row reduction, computes the partitions Q j and computes the F -orbit representative simultaneously, but we assume A is in standard form, and first compute all the Q j .Feulner's algorithm also computes the orbit representative under GL j (p) × D [j] for j < s.We observe that since A is in standard form, A * ,s is an identity matrix, so A is the orbit representative of A under the action of GL s (p) × D [s] .Hence the correctness of Algorithm 1 follows from the analysis in [6].
For the complexity claim, since s ≤ k and the analysis of Lines 2-10 of Algorithm 1 is straightforward, it remains only to show that the Q j can be computed in time polynomial in sk ∈ O(k 2 ).As A s,s is an identity matrix, if j ≤ s then Q j is the trivial partition of {1, 2, . . ., s} with s cells.Now let j ≥ s + 1 and suppose that we have constructed Q j−1 .To construct Q j , we merge all cells of Q j−1 that have non-trivial intersection with Supp(A * ,j ).Since Q j−1 has at most s cells, we can do this in time polynomial in sk.
if a = 0 and A u,j+1 = 0 for all u ∈ Q then

Computing N Sn (H) by searching in K
In this subsection, we will first show that we can efficiently compute N B (H), then we prove that N Sn (H) can be computed in time O(( n p )! + n c ) for some constant c.Definition 4.7.Let M ∈ M(s, k, p) be the generator matrix of γ(H) in standard form.For 1 ≤ i ≤ s, we call x i := γ −1 (M i, * ) the standard generators of H.We replace the original generating set of H by the standard generating set X = {x 1 , x 2 , . . ., x s }.
We now show that each element of N B (H) conjugates elements of X with non-disjoint supports to the same power.Recall that we identify F p with the integers {0, 1, . . ., p − 1}.Lemma 4.8.Let b ∈ N B (H), and let x and y be standard generators for H with non-disjoint supports.Then there exists an a ∈ F * p such that x b = x a and y b = y a .Proof.As x b ∈ H, the image γ(x b ) is a linear combination of the rows of M .Since M * ,s is an identity matrix and each element of M is completely defined by its first s coordinates, there exists an a ∈ F * p such that γ(x b ) = γ(x)a and so x b = x a .Similarly, there exists an a such that y b = y a .Now consider an H-orbit Ω i contained in Supp(x) ∩ Supp(y) and let g i be the corresponding generator of the enveloping group G. Since b fixes each Ω i setwise, g b i = g a i = g a i and so a = a . For groups R i have pairwise disjoint supports, and each R i cannot be written as a non-trivial direct product of subgroups with disjoint supports.We use such decompositions to compute the subgroup of N Sn (H) which setwise stabilises each H-orbit.Proposition 4.9.Let H be a subgroup of S n in class InP(C p ) with orbits Ω 1 , Ω 2 , . . ., Ω k , and let Proof.Let t be a primitive element of F * p , and let and let σ i ∈ Sym(Γ i ) be such that g σ i j = g t j for all generators g j of G such that Ω j ⊆ Γ i .We first show that i for some a, and so (h Since t is independent of Ω i , the permutation h σ j j = h t j is in R j .For i = j, since h i and σ j have disjoint supports, h and let X be the standard generating set of H.By Lemma 4.8, there exists t l ∈ F * p such that x b| Γ i = x t l for all x ∈ X with support intersecting non-trivially with so the result follows.For the complexity result, in time polynomial in p, we can determine a primitive t ∈ F * p .In time polynomial in n, we can compute the unique finest disjoint direct decomposition of H by [4] and construct the g i by Lemma 3.3.The Sym(Ω i )-conjugacy of permutations can be solved in polynomial time, so we can construct the σ i in polynomial time.
Lastly, we show that to compute N Sn (H) for H ∈ InP(C p ), it suffices to search in K. Our implementation to compute N Sn (H) which will be described in Section 6 uses the procedure described in the following proof.
Theorem 1.4 now follows.Note that Proposition 4.2 gives better complexity than Theorem 4.10 when p ≤ k.However, the algorithm for Proposition 4.2 requires the checking of all elements of (F * p ) m , which we do not have useful pruning methods for.When p and m are large, this becomes infeasible (see Table 1).

Pruning techniques
Recall that we compute N Sn (H) using backtrack search.In this section, we present some methods to compute with H efficiently using γ(H), including some pruning techniques.We will see how to apply these results in Section 6. Throughout, let H be as in Hypothesis 3.1.
First we show that stabilisers and the relation ≡ H from Definition 2.3 can be computed using a generating matrix M for γ(H).Recall that we denote the rows and columns of M by M i, * and M * ,j , and let s = {1, 2, . . ., s}.
Lemma 5.1.Let β be a point in an H-orbit Ω j .If M * ,j is non-zero in a unique row, i say, then γ(H (β) ) = M s\i, * .
The assumption on M * ,j in the previous lemma can be achieved by performing row operations.Hence any point stabiliser can be computed by row operations.Lemma 5.2.Let M be a generator matrix of γ(H), and let a i and a j be the first non-zero entries of M * ,i and M * ,j respectively.Then Proof.We show that Ω i ≡ H Ω j if and only if M * ,j = aM * ,i for some a ∈ F * p , from which the result follows.⇐: Recall the p-cycles g i from Hypothesis 3.1.Fix α ∈ Ω i and β ∈ Ω j .Define a mapping ψ : Ω i → Ω j by setting ψ(α g u i ) = β g au j for 0 ≤ u ≤ p − 1.Since g i and g a j are isomorphic and regular, ψ is a bijection, and the reader may check that ψ witnesses the equivalence of Ω i and Ω j .⇒: By Definition 2.3, there exists an involution σ ∈ Sym(Ω i ∪ Ω j ) such that for all h ∈ H, we have h| Next, we show how we can use linearly dependent columns of M to prune the search tree.For a subset J of {1, 2, . . ., k}, we denote the union ∪ j∈J Ω j by Ω J .A set V of linearly dependent vectors is minimally linearly dependent if no proper subset U ⊂ V is linearly dependent.Lemma 5.3.Let I and J be subsets of {1, 2, . . ., k} such that there exists ν ∈ N Sn (H) with Ω ν I = Ω J .Then the rank of M * ,I is equal to the rank of M * ,J .Hence the columns of M * ,I are minimally linearly dependent if and only if the columns of M * ,J are minimally linearly dependent.
Proof.The assumption that ν exists implies that H| Ω I and H| Ω J are conjugate in Sym(Ω I ∪Ω J ).Let r I and r J be the ranks of M * ,I and M * ,J respectively.Then Observe that the columns of M * ,I are minimally linearly dependent if and only if M * ,I has rank |I| − 1 and each proper subset S of columns has rank |S|.Since rank is preserved by conjugation, the final claim follows.
The following lemma is elementary but extremely useful for pruning the search tree T .Two columns M * ,i and M * ,j of M are equivalent, denoted by M * ,i ≡ M M * ,j , if Ω i ≡ H Ω j .The weight enumerator w(C) of a linear code C ≤ F k p is the polynomial w(C) = k i=1 w i x i , where w i is the number of codewords of weight i. Lemma 5.4.
3. Let V be the set of non-zero minimum weight vectors in γ(H).Then N Sn (H) setwise stabilises γ −1 (V ).Hence N Sn (H) setwise stabilises the partition of {1, 2, . . ., k} where i ∼ j if and only if Proof.For Part 1, by Lemma 2.2.1, the existence of ν implies that H (∆) and H (Γ) are conjugate in S n , so the required bijection exists by Lemma 2.5.For Parts 2 and 3, notice that γ(h) i = 0 if and only if h| Ω i = 1.So the weight of γ(h) is the number of p-cycles of h.
There is no known polynomial-time algorithm for computing the weight enumerator, so we use a simple heuristic to determine when we use Part 2 (see Algorithm 5).
Lastly, we give a technical lemma that can be used to prune the search tree T , whose rationale is as follows.Assume we are at a node τ at depth m > s in T .If there exists a κ ∈ K represented by a leaf below τ such that bκ ∈ N Sn (H), then there exists d ∈ D such that dρ(κ) ∈ MAut(γ(H)).Let M := M dρ(κ).Then since m > s, at node τ we know, up to s unknown scalars from F * p , the first s columns of M .Since rows of M are linear combinations of the rows of M and the elements of M are defined by their first s coordinates, we now know the whole of M , up to s unknown scalars.So if we can deduce that some entries of M must be zero, then we can sometimes show that no such κ exists, and hence we can backtrack from τ .Lemma 5.5.Let M ∈ M(s, k, p) be a generator matrix of γ(H) in standard form.Fix m ∈ {s, s + 1, . . ., k − 1} and let J = {1, 2, . . ., m} ∪{u} ⊆ {1, 2, . . ., k}.Let f : J → {1, 2, . . ., k} be an injection.If there exists an i ∈ {1, 2, . . ., s} such that M i,f (u) = 0 and M i,f (j) M j,u = 0 for all j ∈ J ∩ {1, 2, . . ., s}, then there does not exist ν ∈ N Sn (H) such that Ω ν j = Ω f (j) for all j ∈ J.
Proof.Aiming for a contradiction, assume that such an i and ν exist.Then by Lemma 3.
, for all j ∈ J and all i.
In particular, M i,u = d f (u) M i,f (u) , which is non-zero.
Each row of M is a linear combination of the rows of M , and M is in standard form, so where the second equality follows from (4).Therefore by (3), M i,u = s l=1 d f (l) M i,f (l) M l,u = 0, a contradiction.Algorithm 3: Initialising the domains with domainsInit, makeDualEquiv-Partn, makeStabsPartn and makeInvSetPartn DomainsInit returns a list of sets doms, where each entry doms[i] is a subset of {1, 2 . . ., k} such that if there exists ν ∈ N Sn (H) with Ω ν i = Ω j then j ∈ doms[i].We shall compute partitions P ⊥ e , P s , P ⊥ s , P I of {1, 2, . . ., k}, where each partition represents a partition of O = {Ω 1 , Ω 2 , . . ., Ω k } preserved by N Sn (H).Hence the meet P * of these partitions is also preserved by N Sn (H) and we initialise the domain of i to be [i] P * .
We have assumed that the H-orbits are pairwise inequivalent, but the relation ≡ H ⊥ may be non-trivial.makeDualEquivPartn exploits this and computes P ⊥ e , which N Sn (H) preserves by Lemma 2.5.Since known subgroups of N Sn (H) can be used to prune the search tree, and centralising elements for H ⊥ that act non-trivially on the set of H ⊥ -orbits yield elements of N Sn (H)\B, we also update N with such normalising elements.
makeStabsPartn considers the stabiliser of each H-and H ⊥ -orbit, then uses the sizes of these stabilisers' ≡-classes to make partitions P s and P ⊥ s , which are preserved by N Sn (H) by Lemmas 5.4.1 and 4.1.
makeInvSetPartn first computes the set invSet of minimal weight vectors of γ(H).The algorithm systematically considers linear combinations of rows of M with increasingly many non-zero coefficients.Since M s,s is the identity matrix, if v is a linear combination with nonzero coefficients of i rows of M , then wt(v) ≥ i.As we are only interested in vectors with weight at most m, we only consider all such linear combinations v of up to m rows of M .The group N Sn (H) preserves P I by Lemma 5.4.3.

Algorithm 4: Search with recurseSearch
Now we shall describe the recursive search, which is initialised in Line 10 of Algorithm 2. We shall traverse the search tree depth first, using the domains doms to guide our search.Note that N is a global variable that stores the group generated by all elements of N Sn (H) we have found so far.The pruning tests used in Lines 10-16 are presented as Algorithm 5. Line 23 Since the images of O must be distinct, whenever the domains are changed, we can further refine the domains.For each i, let We remove elements of doms[i] from doms[t] for all t ∈ J i .
Lines 24-25 If a domain becomes empty, we backtrack.Otherwise, we continue the depth-first search, branching using doms.
Algorithm 5: Pruning functions checkLDS and compareStabs checkLDS uses minimally linearly dependent columns of M to prune the search tree.If the condition in Line 6 is satisfied then there is no normalising element under the current node that sends Ω I [1] to Ω i by Lemma 5.3, so we remove i from the domain of I [1].compareStabs uses conjugacy of point stabilisers to prune the search tree, as in Lemma 2.2.1.If any of the conditions in Lines 13, 15 or 18 is satisfied then there are no normalising elements under the current node, by Lemmas 2.5, 5.4.1 and 5.4.2 respectively.Compute the (≡ H ⊥ )-classes using Lemma 5.2

8:
for each pair of equivalent orbits Ω i and Ω j do 9: end for  In this section, we consider H ∈ InP(D 2p ), where p is an odd prime and D 2p is the dihedral group of order 2p and degree p.We show that N Sn (H) can be found by computing the normalisers of its Sylow subgroups, which can be identified with groups in classes InP(C p ) and InP(C 2 ).
We will assume the following throughout this section.Let n = pk and Ω = {1, 2, . . ., n}.Let H be a subgroup of S n in class InP(D 2p ) with orbits and let H q be a Sylow q-subgroup of H, where q ∈ {2, p}.end if 27: end procedure 2. H p is a subdirect product of G, and so H p ∈ InP(C p ).
Proof.Part 1: As G is the unique Sylow p-subgroup of D, it follows that H p ≤ H ∩ G, so H p = H ∩ G. Since G D, the group H p is characteristic in H, so the last assertion follows.Part 2: Since H is a subdirect product of D, for all i there exists h ∈ H such that 1 = h| Ω i ∈ G i .Then, for all j, the restriction h 2 | Ω j is of order 1 or p.So h 2 | Ω j ∈ G j for all j and therefore h 2 ∈ G ∩ H = H p .Since h| Ω i is a p-cycle, h 2 | Ω i = 1, so H p | Ω i = G i .Part 3: For all i, there exists an involution r i ∈ D i such that H 2 | Ω i ≤ r i .Since every involution in D i fixes a point, there exists α i ∈ Ω i such that r i fixes α i .So H 2 | Ω i ≤ (D i ) (α i ) .If H 2 pointwise stabilises Ω i , then by Part 1, H| Ω i = (H p H 2 )| Ω i = H p | Ω i = G i , but H is a subdirect product of D, a contradiction, and so H 2 | Ω i = (D i ) (α i ) .
1.I ≤ T and so I = N T (H 2 ) ∩ N T (H p ).
2. Let t ∈ F * p be primitive.For all i ≤ k, let g i ∈ Sym(Ω i ) be a generator of G i , and let c i be an element of Sym(Ω i \{α i }) such that g c i i = g t i .Then N i = c i and so T ∼ = C p−1 S k .Proof.Part 1: Let ι ∈ I. Since L ∼ = N Sym(Ω 1 ) (G 1 ) S k by Lemma 2.8.2, there exists κ ∈ K such that ικ −1 fixes each Ω i setwise and (ικ −1 )| Ω i ∈ N Sym(Ω i ) (G i ).To show that (ικ −1 )| Ω i normalises H 2 | Ω i , let j be such that Ω j = Ω ι i .Then κ witnesses the permutation isomorphism from D i to D j and maps α i to α j , so κ conjugates Hence (ικ −1 )| Ω i ≤ N i and so ι ∈ T .Part 2: Observe that N Sym(Ω i ) (G i ) = g i , c i , so N i ≤ g i c i .Assume for a contradiction that there exists a non-trivial g ∈ g i and a c ∈ c i such that gc ∈ N i .Letting r i be the generator for H 2 | Ω i ∼ = C 2 , note that r gc i = r i .Since g moves α i and c fixes only α i , the points α gc i and α i are distinct.But (α gc i ) r i = (α gc i ) (r gc i ) = α r i gc i = α gc i , a contradiction since r i only fixes α i , so N i ≤ c i .
To see that N i ≥ c i , first notice that c i ∈ N Sym(Ω i ) (G i ).Since r i is an element of N Sym(Ω i ) (G i ) which fixes α i , it is in c i , and so c i normalises H 2 | Ω i .
Lastly the isomorphism follows from Lemma 2.8.2.

Theorem 1 . 4 .
The Norm-Sym problem for H in class InP(C p ) can be solved in time min 2 O( n p log n p +log n) , 2 O( n 2p log n) .
Define a map Ξ : L → W by writing each l ∈ L as bκ for some b ∈ B and κ ∈ K and mapping l = bκ → ζ(b)ρ(κ).

Finally we prove Theorem 1. 3 .
Proof of Theorem 1.3.To reduce Norm-Sym for groups H ≤ S pk in class InP(C p ) to MAut, first notice that by Lemma 3.3, in polynomial time, we can compute the enveloping group G of H and an isomorphism γ : G → F k p .Assume that we can compute a generating set Y for MAut(γ(H)) in time polynomial in k.Then by Lemma 3.5, N Sn (H) = {Ξ −1 (y) | y ∈ Y }, G , where each Ξ −1 (y) denotes a pre-image of y under Ξ.
Now we prove a more precise version of the upper bound in Theorem 1.4.Proposition 4.2.Let H = X be a subgroup of S n in class InP(C p ), with n = pk and |H| = p s , and let m := min{s, k − s}.Then N Sn (H) can be computed in time 2 O((m+1) log n) .Hence N Sn (H) can be computed in time 2 O( n 2p log n) .

Lemma 4 . 3 .
Let F := GL s (p) × D. Define an action of F on M(s, k, p) by M (R,d) = R −1 M d for all R ∈ GL s (p) and d ∈ D. Let κ ∈ K and let M, M ∈ M(s, k, p) be generator matrices of γ(H) and γ(H κ −1 ) respectively.Then there exists a b ∈ B such that bκ ∈ N Sn (H) if and only if there exists an f ∈ F such that M f = M .Proof.Such a b exists if and only if H and H κ −1 are in the same B-orbit, or equivalently, by Lemma 3.5, γ(H) and γ(H κ −1 ) are in the same D-orbit.The result now follows from the fact that M = M if and only if M and M are in the same orbit under left multiplication by GL s (p).

Proposition 4 . 6 .
Let H be a subgroup of S n in InP(C p ), let L = B K be as in Lemma 2.8, and let κ ∈ K. Then in time polynomial in n, we can determine if there exists b ∈ B such that bκ ∈ N Sn (H), and if so, output b.Proof.By Lemma 3.3, in polynomial time, we can verify that H ∈ InP(C p ) and construct generator matrices M, M ∈ M(s, k, p) for γ(H) and γ(H κ −1 ) respectively, in standard form.By Lemma 4.3, such a b exists if and only if M and M have the same F -orbit representative.So by Theorem 4.5, in time polynomial in n/p, we can decide if such a b exists.Furthermore, by keeping track of the changes in Lines 5 and 7 of Algorithm 1, we can simultaneously obtain elements (R 1 , d 1 ) and (R 2 , d 2 ) of F which map M and M to their F -orbit representatives.Let ζ : B → D be as in Lemma 3.5.Then H ζ −1 (d 1 d −1 2 ) = H κ −1 and hence we can construct b = ζ −1 (d 1 d −1 2 ), in time polynomial in n by Lemma 3.5.4.

Theorem 4 . 10 .
Norm-Sym for H = X ≤ S n , where H ∈ InP(C p ), can be solved in time 2 O( n p log n p +log n) .Proof.By Lemma 3.3, in polynomial time 2 O(log n) , we can recognise that H ∈ InP(C p ), and obtain a generating set φ i | 2 ≤ j ≤ k for K, an isomorphism γ : G → F k p as in Definition 3.2, and a generator matrix M for γ(H) in standard form.Next we compute C Sn (H) and N B (H), in time polynomial in n by Lemma 2.1.1 and Proposition 4.9 respectively.Initialise N as C Sn (H), N B (H) .For each κ ∈ K, we determine if there exists b ∈ B such that bκ ∈ N Sn (H), in time polynomial in n by Proposition 4.6.If such a b exists, we update N as N, bκ .This takes time O(( n p )!n c ) = 2 O( n p log n p +log n) , for some constant c.By the end of the procedure, we have N ≤ N Sn (H).To show N Sn (H) ≤ N , let ν ∈ N Sn (H).Then since ν ∈ L = B K by Lemma 2.8.3, we can find κ ∈ K and b ∈ B such that ν = bκ by Lemma 2.8.4.If κ = 1, then ν ∈ N B (H) and so ν ∈ N .Suppose now that κ = 1.We have already found an element b

Figure 1 :
Figure 1: Median log time (secs) to compute N Sn (H) for 10 instances of H ≤ S n in InP(C p ).

Figure 2 :
Figure 2: Median log time (secs) to compute N Sn (H) for 10 instances of H ≤ S n in InP(D 2p ).
One can check that θ(u) is indeed in S n and that θ is a homomorphism.To see that N Sn (H) contains θ(N U (H| Γ )), C Sn (H) , it suffices to show that it contains the image of N U (H| Γ ) under θ.Let h ∈ H and u ∈ N U (H| Γ ).Then there exists an h ∈ H such that h Proof.Part 1: Let ξ : K → S k be the permutation representation of K on O. Then ξ is surjective since φ i induces the permutation (Ω 1 , Ω i ) on O.For injectivity, let κ ∈ K be such that ξ(κ) = 1.
which can be computed in time polynomial in k by Lemma 3.5.4.For the equivalence of Conj-Sym for groups in class InP(C p ) and MEq for codes over F p , let H 1 and H 2 be subgroups of S pk and let C 1 = γ(H 1 ) and C 2 = γ(H 2 ) be codes of length k over F p .It follows from Lemma 3.5.1-2that H 1 and H 2 are conjugate in S n if and only if C 1 and C 2 are monomially equivalent.The rest of the proof is similar to that of the equivalence of Norm-Sym and MAut.Corollary 3.6.Fix a prime p. Denote MAut for p-ary codes by MAut p .Then the graph isomorphism problem is polynomial-time reducible to MAut p , which is polynomial-time reducible to Norm-Sym.
Lastly, we see how we use Algorithm 1 to decide if there exists an element of N Sn (H) which induces a given permutation of the H-orbits.