Asymptotic and exact results on the complexity of the Novelli--Pak--Stoyanovskii algorithm

The Novelli--Pak--Stoyanovskii algorithm is a sorting algorithm for Young tableaux of a fixed shape that was originally devised to give a bijective proof of the hook-length formula. We obtain new asymptotic results on the average case and worst case complexity of this algorithm as the underlying shape tends to a fixed limit curve. Furthermore, using the summation package Sigma we prove an exact formula for the average case complexity when the underlying shape consists of only two rows. We thereby answer questions posed by Krattenthaler and M{\"u}ller.

In this paper we prove Conjectures 1 and 3. While we were unable to prove Conjecture 2 in full generality, we do provide a proof for a large class of sequences of partitions, adding further evidence to its validity. The article is structured as follows: In Section 1 we review the combinatorics of the NPS algorithm.
In Sections 2-4 we engage sequences of partitions that converge under a balanced scaling, that is, by a factor of √ n in both dimensions. Section 2 contains mostly preparatory results and includes a precise definition of convergence. Section 3 treats the worst case complexity. We derive an exact combinatorial formula for W (λ) for any partition λ in Theorem 3.1 by proving that a trivial upper bound is tight. Moreover Theorem 3.2 provides an asymptotic result on W (λ (n) ) in the balanced case. Section 4 treats the average case complexity in the balanced case.
Here we give an asymptotic lower bound for C(λ (n) ) in Theorem 4.3. It is a consequence of our results that both C(λ (n) ) and W (λ (n) ) are of order λ (n) 3/2 , which is in accordance with Conjecture 1.
In Section 5 we turn to sequences that converge when subjected to an imbalanced scaling, that is, by a factor of n 1/p in one direction and a factor of n 1/q in another. Theorem 5.1 verifies both Conjectures 1 and 2 in the imbalanced setting. More precisely, we show that C(λ (n) ) and W (λ (n) ) are both of order n (p+1)/p if p < q and of order n (q+1)/q if p > q, and that the leading coefficient of the average case complexity is one half of the leading coefficient of the worst case complexity.
In Section 6 we prove Conjecture 3 employing the summation package Sigma [Sch07] in a non-trivial fashion. Here we first provide an alternative representation C(λ) in terms of five nontrivial double sums and show that this expression corresponds to the single sum expression given in Conjecture 3. The underlying machinery is based on the summation paradigms of creative telescoping, recurrence solving and the zero-recognition problem for the class of (indefinite) nested sums over hypergeometric products. As a by-product we provide alternative representations of C(λ) = C(λ 1 , λ 2 ) that enable one to calculate C(λ) efficiently if one keeps λ 1 symbolic and specializes λ 2 to a concrete value, or if one keeps λ 1 symbolic and specializes the distance λ 1 − λ 2 to a specific non-negative integers. In particular, we discover a particularly nice formula for the special case λ 1 = λ 2 .
In this section we recall some definitions concerning partitions and Young tableaux as well as the needed facts about the NPS algorithm.
Let n ∈ N be a non-negative integer. A partition λ of n is a weakly decreasing sequence λ 1 ≥ λ 2 ≥ · · · ≥ λ k > 0 of positive integers such that |λ| = λ i = n. We call |λ| the size of λ. The length l(λ) is the number of summands λ i . We identify a partition with its Young diagram λ = {(i, j) : 1 ≤ i ≤ l(λ), 1 ≤ j ≤ λ i }. The elements (i, j) are called the cells of the partition λ.
The conjugate partition λ ′ of λ is given by the Young diagram {(j, i) : (i, j) ∈ λ}. Visually we imagine a partition as a left justified array of n boxes, with λ i boxes in the i-th row counting from top to bottom as in Figure 1. Thus λ ′ is obtained from λ by a flip along the main diagonal.
Define the arm of a cell arm λ (i, j) = λ i − j as the number of cells in the same row as (i, j) and strictly to the right of (i, j). Define the leg of a cell leg λ (i, j) = λ ′ j − i as the number of cells in the same column as (i, j) and strictly below (i, j). Furthermore, define the hook length of a cell as h λ (i, j) = λ i + λ ′ j − i − j + 1. We call a cell (i, j) a corner of λ if h λ (i, j) = 1. An integer filling of a partition is a map T : λ → Z assigning an entry T (i, j) to each cell (i, j). The partition λ is called the shape of T . A tableau is a bijection T : λ → {1, 2, . . . , n}. A standard Young tableau (SYT) is a tableau that increases along rows from left to right as well as down columns, that is, . Note that somewhat counter-intuitively a hook tableau is not a tableau. We denote the set of all tableaux, standard Young tableaux and hook tableaux of shape λ by T(λ), SYT(λ) and H(λ), respectively.
Such a construction was found by Novelli, Pak and Stoyanovskii [PS92,NPS97]. We are now going to describe the map Φ. See Figure 2 for an example.
First note that given a permutation σ ∈ S n and a tableau T ∈ T(λ), we obtain a new tableau T ′ = σ • T by setting T ′ (i, j) = σ(T (i, j)). In particular if σ = (k, m) is a transposition, then (k, m) • T arises from T by exchanging the two entries k and m.
Impose the reverse lexicographic order on the cells of λ, that is, (i, j) (i ′ , j ′ ) if j < j ′ or if j = j ′ and i ≤ i ′ . Let (i 1 , j 1 ) ≻ (i 2 , j 2 ) ≻ · · · ≻ (i n , j n ) be the cells of λ in decreasing order and set k r = T (i r , j r ). Moreover, set T 0 = T and let H 0 be the hook tableau with all entries equal to zero.
Given a pair tableau (T r−1 , H r−1 ) we first construct the tableau T r from T r−1 as follows: E4: If k r < m then return T r = T . E5: Otherwise m < k r . In this case exchange the entries m and k r , that is, set T = (k r , m) • T , and return to step E1. Note that T r is obtained from T r−1 by applying a jeu-de-taquin-like move to the entry k r . Next we construct H r from H r−1 as follows: H3: Return H r = H. These rules give rise to a sequence (T 0 , H 0 ), (T 1 , H 1 ), . . . , (T n , H n ) of pairs of a tableau and a hook tableau. We define Φ(T ) = (T n , H n ). While it is not too difficult to show that T n is a standard Young tableau and that H n is a hook tableau, it takes considerably more effort to prove that Φ is a bijection. For details we refer to [Kra99,NPS97,Sag01].
Given a tableau T we denote by n(T ) the number of exchanges performed during the application of the NPS algorithm, that is, the number of times step E5 is visited during the construction of all tableaux T 1 , . . . , T n . The average case complexity of the NPS algorithm is now defined as The worst case complexity is defined as Let λ and µ be two partitions. We write µ ⊆ λ if the Young diagram of µ is contained in the Young diagram of λ. In this case the skew diagram λ/µ consists of the cells of λ that are not cells of µ. A tableau of skew shape λ/µ is simply a bijection T : λ/µ → {1, . . . , |λ| − |µ|}. A standard Young tableau of skew shape λ/µ is a tableau of this shape that increases along rows and down columns. In the proof of Theorem 3.1 we apply the NPS algorithm to a tableau of skew shape. We remark that this poses no difficulties. The output will even be a pair of a standard Young y v u x Figure 3. The partition λ = (4, 4, 2, 1, 1, 1) depicted in Russian convention.
tableau and a hook tableau of the same skew shape. The bijectivity of the NPS algorithm is, however, lost.

Convergence and hook coordinates
We shall work with the following coordinate system corresponding to a rotation by π/2 and a rescaling by Given a partition λ of n define We define the boundary function γ : R → R of λ via The set D λ and function γ describe the (rescaled) partition λ in the so-called Russian convention.
Let (λ (n) ) n∈N be a sequence of partitions, such that λ (n) is a partition of n and has the boundary function γ n . Denote by Γ the set of all 1-Lipschitz functions γ : R → R such that there exists an interval (a, b) with γ(x) = |x| for all x / ∈ (a, b). Moreover, denote by Γ 1 the set of all functions γ ∈ Γ such that R γ(x) − |x| dx = 1.
Clearly γ n ∈ Γ 1 for all n ∈ N. We say the sequence λ (n) converges uniformly to a limit curve γ ∈ Γ 1 if lim n→∞ sup x∈R |γ(x) − γ n (x)| = 0 and there exists a uniform interval (a, b) such that γ n (x) = |x| for all x / ∈ (a, b) and all n ∈ N. For any γ ∈ Γ set and let (x, y) be an interior point of D γ . We define three functions via Geometrically a γ (x, y) is the distance from (x, y) to the curve γ in u-direction, ℓ γ (x, y) is the distance from (x, y) to the curve γ in v-direction, and d γ (x, y) is half of the maximal perimeter among all rectangles confined in D γ with sides parallel to the u-and v-axes, whose lower corner is (x, y). Extend a γ , ℓ γ and d γ from the interior of D γ to R 2 by assigning zero to all other points. With regard to the subsequent sections we need to give the following question some thought. Suppose γ, η ∈ Γ are close with respect to the supremum norm, then what can be said about the relationship between a γ and a η on their common domain D γ ∩ D η ? The following example demonstrates that ||a γ − a η || ∞ does not need to be small. Let Then ||γ − η|| ∞ = √ 2 n but ||a γ − a η || ∞ = ||ℓ γ − ℓ η || ∞ = n − 1 n . We show, however, in Lemma 2.2 that when γ and η agree outside of a fixed interval (a, b), then the exceptional set of points on which a γ and a η diverge is small when ||γ − η|| ∞ is small. The proof of Lemma 2.2 is based on a geometric argument. In Lemma 2.3 and Lemma 2.4 we deduce analogous results for ℓ γ and d γ , which causes little effort once Lemma 2.2 is established.
For a (measurable) subset A of R n , let |A| denote the Lebesgue measure of A.
Lemma 2.1. Let γ, η ∈ Γ such that γ(x) = |x| = η(x) for all x / ∈ (a, b). Then the Lebesgue measure of the symmetric difference of the sets D γ and D η is bounded by Proof. It follows immediately from the definition of D γ that The inequality is strict since γ and η agree at a and b and are continuous.
Lemma 2.2. Let ε > 0 and (a, b) be an interval. Then there exists a constant K such that for all functions γ, η ∈ Γ with γ(x) = |x| = η(x) for all x / ∈ (a, b) we have We first prove the claim under the assumption that γ(x) ≤ η(x) for all x ∈ (a, b). To this end Furthermore, subdivide the curve of γ into segments σ 1 , . . . , σ k such that Define Thus we should investigate the set of points {(x, |x|) : a ≤ x ≤ 0}. Note that the sets P (σ i ) cover the set {(x, |x|) : a ≤ x ≤ 0} in such a way that P (σ i ) ∩ P (σ i+1 ) consists of a single point, and P (σ i ) ∩ P (σ j ) is empty unless all sets P (σ i+1 ), . . . , P (σ j−1 ) collapse to a single point. Suppose , then let i j+1 be the minimal i with this property. Otherwise terminate both sequences.
The analogous result on ℓ γ follows easily.
Lemma 2.3. Let ε > 0 and (a, b) be an interval. Then there exists a constant K such that for Proof. The claim follows directly from Lemma 2.2 and the symmetry Finally there is a similar result for the function d γ .
Lemma 2.4. Let ε > 0 and (a, b) be an interval. Then there exists a constant K such that for Proof. The claim follows from Lemma 2.2 and Lemma 2.3 and the estimation We conclude this section by introducing the so called hook coordinates, which were named (to the best of the authors' knowledge) by Dan Romik, and appear naturally in the study of limit shapes of partitions, see for example [LS77]. Namely, we set which yields where the derivative γ ′ is defined almost everywhere since γ is 1-Lipschitz.

Worst case complexity
In this section we analyse the asymptotic behaviour of the worst case complexity of the NPS algorithm. We first demonstrate in Theorem 3.1 that a trivial combinatorial upper bound for the worst case complexity of the NPS algorithm on a fixed shape W (λ) is in fact tight. Furthermore, Theorem 3.2 provides the first order asymptotics of W (λ (n) ), where (λ (n) ) n∈N converges uniformly, in terms of the limit curve γ.
For a cell (i, j) ∈ λ denote by We first show that this upper bound is tight.
Theorem 3.1. Let T be a standard Young tableau of skew shape λ/µ. Then there exists a tableau F of skew shape λ/µ such that the NPS algorithm transforms F into T using exactly which is transformed into T by the NPS algorithm using w(i, j) exchanges. We prove this claim by induction on n = |λ/µ|.
The case n = 1 is trivial. To demonstrate the induction step we distinguish two cases. If there is a second corner (i ′′ , j ′′ ) of λ/µ then let k ′′ = T (i ′′ , j ′′ ). By induction there exists a tableau where T ′ is obtained from T by deleting the cell (i ′′ , j ′′ ) and reducing each entry in {k ′′ + 1, . . . , n} by one. Let F be the tableau of shape λ/µ obtained from F ′ by increasing each entry in {k ′′ , . . . , n − 1} by one, and then setting F (i ′′ , j ′′ ) = k ′′ . By construction F induces a SYT on λ/µ − {(i, j)} and NPS transforms F into T in w(i, j) steps.
If (i ′ , j ′ ) is the only corner of λ/µ then k ′ = n. Let F ′ be a tableau of shape λ/µ − {(i ′ , j ′ )} with F ′ (i, j) = n − 1, which induces a SYT on λ/µ − {(i, j), (i ′ , j ′ )}, and is transformed into T ′ by the NPS algorithm using w(i, j) − 1 exchanges, where T ′ is obtained from T by deleting the corner (i ′ , j ′ ). Then the desired skew tableau F is obtained from F ′ by setting F (i, j) = n and Approximating the right hand side of (3.1) by an integral and making use of the preparatory results in Section 2 we are able to draw conclusions on the asymptotics of the worst case complexity.
Theorem 3.2. Let λ (n) be a sequence of partitions converging uniformly to the limit shape γ ∈ Γ 1 . Then Before we turn to the proof let us state some remarks. First, let us argue the existence of the integral in (3.2). Since d γ (x, y) is bounded and D γ is the union of a compact set and a null set, the integral is proper. Furthermore, the function d γ (x, y) is decreasing in y and hence integrable. The function Secondly, since the right hand side of (3.2) is a priori not straight forward to compute, we offer the estimation which is obtained from d γ (x, y) ≤ a γ (x, y) + ℓ γ (x, y) by a substitution of hook coordinates.
Poof of Theorem 3.2. First rewrite W (λ (n) ) as an integral as follows: A cell (i, j) ∈ λ (n) corresponds to the square Let (x, y) be an interior point of Z(i, j).
Hence for all sufficiently large n. In order to do so choose an interval (a, b) such that γ n (x) = |x| for all x / ∈ (a, b) and all n ∈ N. It follows that also γ(x) = |x| outside of (a, b). By Lemma 2.1 the Lebesgue measure of the symmetric difference of the sets D γ and D n tends to zero as n tends to infinity. Since both d γ and d γn are bounded by the constant (b − a) √ 2, . On the other hand by Lemma 2.4 there exist sets A and B and a constant K such that D γ ∩ D n = A ∪ B, |d γ (x, y) − d γn (x, y)| < ε/2 for all (x, y) ∈ A and |B| < K||γ − γ n || ∞ /ε. Hence, if ||γ − γ n || ∞ is sufficiently small.

Average case complexity
The main result of this section is an asymptotic lower bound for the average case complexity of the NPS algorithm, which we obtain in three steps. Proposition 4.1 obtains a combinatorial bound for the average case complexity of the NPS algorithm on a fixed shape C(λ). Proposition 4.2 approximates this combinatorial bound by an integral. Finally, in Theorem 4.3 we derive an asymptotic bound for C(λ (n) ), where (λ (n) ) n∈N converges uniformly, in terms of the limit curve γ.
Given a hook tableau H of shape λ denote Proposition 4.1. Let λ be a partition of n, and H be a hook tableau of shape λ chosen uniformly at random. Then C(λ) > E(|H|).
Proof. During the application of the NPS algorithm to a tableau T of shape λ a hook tableau of the same shape is built from the zero tableau, that is, the hook tableau with |H| = 0. This is done by applying the following transformations. Suppose the entry of the cell (i, j) drops to the cell (i ′ , j ′ ). Then H(s, j) is set to H(s + 1, j) − 1 for s = i, . . . , i ′ − 1, and H(i ′ , j) is set to j ′ − j. Thereby |H| is increased by no more than i ′ − i + j ′ − j which is exactly the number of performed exchanges. Since the NPS algorithm produces each hook tableau equally often as T ranges over all possible tableaux of shape λ, we conclude the following lower bound on the average number of exchanges where the sum is taken over all hook tableaux of shape λ. The number of hook tableaux is given by the hook product By use of the hook-length formula, the right hand side of (4.1) is just the expected value of the random variable |H|.
In a next step we replace the combinatorial lower bound C(λ) > E(|H|) by an integral.
Proposition 4.2. Let λ be a partition of n with boundary γ ∈ Γ 1 . Then Proof. Recall that C(λ) > E(|H|), where H ranges over the hook tableaux of shape λ by Proposition 4.1. By linearity .
Throughout the remainder of this proof denote a ij = arm(i, j), The error term (4.2) vanishes by symmetry in z and w. We have The other two error terms, (4.3) and (4.4), could be neglected as the integrands are non-negative. However, there is no harm in showing that they are also small. This can be seen easily since the integrands converge uniformly to 2z and 0 respectively. Thus In particular, the integrals in (4.3) and (4.4) are uniformly bounded no matter how large h ij is. Approximating ℓ 2 ij /h ij by an analogous integral and summing over all cells of λ yields the claim.
Using the preparatory results of Section 2, we obtain the main theorem of this section.
Theorem 4.3. Let (λ (n) ) n∈N be a sequence of partitions converging uniformly to the limit curve γ ∈ Γ 1 . Then as n → ∞.
Before we give a proof let us argue that the right hand side of (4.5) is well-defined. First note that the integral is really taken over a compact set. Suppose that γ(x) = |x| for all x / ∈ (a, b) then γ ′ (s) = −1 for all s < a and γ ′ (t) = 1 for all t > b, and the integrand vanishes. Because γ is Lipschitz continuous, its derivative exists almost everywhere, is Lebesgue integrable and fulfils b a γ ′ (s) ds = γ(b) − γ(a). Thus the limit of the quotient (γ(t) − γ(s))/(t − s) as t tends to s exists almost everywhere. The integrand is therefore essentially bounded and integrable.
Proof of Theorem 4.3. Let γ n ∈ Γ 1 be the boundary function of λ (n) and choose an interval (a, b) such that γ n (x) = |x| for all x / ∈ (a, b). We begin by noting that dx dy → 0 (4.6) as n tends to infinity. To see this, first note that the functions a γn , a γ , ℓ γn and ℓ γ are all non-negative and bounded by (b − a) √ 2. Thus also the integrands in (4.6) are non-negative and bounded. By Lemma 2.1 it suffices to consider the common domain D γn ∩ D γ . It then follows from the Lemmata 2.2 and 2.3 that To finish the proof we use our hook coordinates. Substitution gives 1 2 Dγ a 2 γ (x, y) + ℓ γ (x, y) 2 a γ (x, y) + ℓ γ (x, y) dx dy Perhaps the only step that needs comment is the last one. Each pair (s, t) with s ≤ t gives rise to a unique point (x, y) ∈ D γ unless γ ′ (s) = −1 or γ ′ (t) = 1. This follows from the Lipschitz property of γ. However, the integrand vanishes when either of the two cases γ ′ (s) = −1 or γ ′ (t) = 1 occurs. Hence, we can relax the limits of the integral without altering its evaluation.

Imbalanced scaling
Some types of partitions, for example partitions with a fixed number of parts, do not converge to a limit curve γ ∈ Γ 1 in the sense of Section 2. However, they might converge if an alternative scaling, that is, not by a factor of √ n in both u-and v-direction, is chosen. In this section we study the asymptotic behaviour of the average case and worst case complexity of the NPS algorithm when the partitions under consideration converge after an imbalanced scaling.
For p, q ∈ N ∪ {∞} such that consider the coordinates given by To each partition λ of n we associate a set D λ and a p, q-boundary function γ, defined exactly as in (2.2) and (2.3) but with u and v now given by (5.1).
Let (λ (n) ) n∈N be a sequence of partitions such that λ (n) is a partition of n and has p, q-boundary γ n . We say (λ (n) ) n∈N converges p, q-uniformly to the limit curve γ ∈ Γ 1 if lim n→∞ sup x∈R |γ(x) − γ n (x)| = 0 and there exists an interval (a, b) such that γ n (x) = |x| for all x / ∈ (a, b) and all n ∈ N.
The main result of this section provides the leading terms of C(λ (n) ) and W (λ (n) ), where (λ (n) ) n∈N converges p, q-uniformly, in terms of the limit curve γ. They turn out to be of order n 1+max{1/p,1/q} as n tends to infinity.
By use of the Lemmata 2.1 and 2.2 it follows that Dγ n a γn (x, y) 2 a γn (x, y) + n 1/q−1/p ℓ γn (x, y) + n −1/p dx dy − Dγ a γ (x, y) dx dy → 0 as n → ∞. This establishes the asymptotic lower bound In order to obtain an upper bound for the average case complexity, recall the algorithm for constructing the hook tableau during the application of the NPS algorithm described in Section 1. The reason why |H| can be less than C(λ (n) ) is that there might be a cancellation in step H1. However, the total cancellation cannot exceed (i,j)∈λ (n) leg(i, j) such that Since the term (i,j)∈λ (n) leg(i, j) = n (q+1)/q Dγ n ℓ γn (x, y) dx dy + n 2 is of order less than n (p+1)/p as n → ∞, we conclude (5.2). Our starting point for the analysis of the worst case complexity is the inequality which is a trivial consequence of Theorem 3.1. The right hand side of (5.7) equals n (p+1)/p Dγ n a γn (x, y) dx dy + n (q+1)/q Dγ n ℓ γn (x, y) dx dy + n.
Again the term corresponding to the leg function is of lower order and can be dropped. Thus W (λ (n) ) is asymptotically equivalent to the left hand side of (5.7). Lemmata 2.1 and 2.2 imply that Dγ n a γn (x, y) dx dy − Dγ a γ (x, y) dx dy → 0 as n → ∞, which yields (5.3). The alternative formula in terms of hook coordinates is simply obtained by substitution, which completes the first part of the proof. The second part, that is, the case p > q, follows similarly.

Partitions with two parts
The main result of this section is a nice formula for the average case complexity of the NPS algorithm when the partition consists of only two rows.
Theorem 6.1. Let λ = (λ 1 , λ 2 ) be a partition with two parts. Then the average case complexity of the NPS algorithm on λ is given by Our starting point for proving Theorem 6.1 is the following formula for the average case complexity of the NPS algorithm for general partitions.
where the outer sum is taken over all cells x of the Young diagram of λ, |x| = i + j − 2 denotes the distance of a cell x = (i, j) to the top left cell, f λ denotes the number of SYT of shape λ, f λ (x, k) denotes the number of SYT of shape λ such that the cell x contains the entry k, and H n = n ℓ=1 1 ℓ denotes the n-th harmonic number. While the appearance of harmonic numbers in (6.2) is quite surprising, the most challenging expressions are the numbers f λ (x, k). For partitions with only two parts we derive a first explicit expression for the average case complexity in terms of five double sums, each of which contains a harmonic number in the summand. Lemma 6.3. Let λ = (λ 1 , λ 2 ) be a partition with two parts. Then Proof.
The formula for f λ is a simple consequence of the hook-length formula (1.1) Next note that Thus only the terms involving a harmonic numbers that depends on k remain. For a cell x ∈ λ let S(x) denote the set of all values k ∈ [n] such that f λ (x, k) > 0. We observe three cases, Suppose x = (1, j) is a cell of λ, k ∈ S(x), and let T be a SYT of shape λ with T (x) = k. Then the cells y ∈ λ with T (y) < k constitute the partitionμ = (j − 1, k − j). On the other hand the cells y ∈ λ with T (y) > k form the skew shape λ/µ where µ = (j, k − j). Moreover note that the set of SYT of shape λ in which the cell x contains the entry k is in bijection with pairs of a SYT of shapeμ and a SYT of skew shape λ/µ. Consequently While fμ is given by the hook-length formula, the number of SYT of skew shape f λ/µ can be computed using Aitken's determinant formula [Ait43] f λ/µ = (|λ| − |µ|)! · det In the present case over all cells x = (1, j) of the first row of λ, accounts for the first three double sums in (6.3). The case where x = (2, j) is a cell of the second row of λ is treated in the same way, except that nowμ = (k − j, j − 1) and µ = (k − j, j), and accounts for the fourth and fifth double sums in (6.3).
While the proof of Lemma 6.3 is not too complicated, we begin to appreciate how remarkably simple the expression in (6.1) really is, consisting of a single sum devoid of harmonic numbers.
In the following we will prove Theorem 6.1. More precisely, denoting the right hand sides of (6.1) and (6.3) by A(λ 1 , λ 2 ) and B(λ 1 , λ 2 ), respectively, we will show that (6.4) A(λ 1 , λ 2 ) = B(λ 1 , λ 2 ) holds for all λ 1 , λ 2 ∈ N with 0 ≤ λ 2 ≤ λ 1 . Looking at the given problem, one could be tempted to try the following summation tactic: compute for each of the sums a homogeneous recurrence relation in one of the discrete parameters, say λ 2 (using, e.g., the package MultiSum [Weg97]), and combine the found recurrences to one linear homogeneous recurrence for the expression (6.5) T (λ 1 , λ 2 ) := A(λ 1 , λ 2 ) − B(λ 1 , λ 2 ) (using, e.g., the Mathematica package GeneratingFunctions [Mal96]). However, in this particular situation this tactic seems rather clumsy: already the calculation of the linear recurrences for each single sum is a hard nut, and assembling the recurrences to a big recurrence for (6.5) is rather hopeless. Therefore we will follow an alternative and more suitable tactic using various features of the summation package Sigma [Sch07]. Namely, exploiting a constructive difference ring and field theory [Kar81,Sch16a,Sch16b], that is incorporated within Sigma, we will proceed as follows.
(DEF): Using Sigma's definite summation toolbox, we will find alternative sum representations where the occurring sums are indefinite nested w.r.t. to the discrete parameter λ 2 . In a nutshell, we will rewrite the expression T (λ 1 , λ 2 ) given in terms of 6 definite sums to an expression in terms of indefinite nested sums w.r.t. λ 2 . (IND): Using Sigma's indefinite summation toolbox, we will rewrite the expression (6.5) further such that no algebraic relations exist among the arising indefinite nested sums and products.
As we will see below, the derived expression of T (λ 1 , λ 2 ) will collapse to zero, which will prove (6.4) and thus will establish Theorem 6.1. We emphasize that this tactic is justified by the following central result stated in [Sch16b,Prop. 7.5]: Suppose that we are given an expression T (n) in terms of indefinite nested sums w.r.t. an integer parameter n such that there exist no algebraic relations among the sums and products. Then we can solve the zero-recognition problem trivially: there exists a δ ∈ N such that T (ν) = 0 holds for all ν ∈ N with ν ≥ δ if and only if the expression T (n) collapses to zero.