1 Introduction

1.1 Definitions and open questions

The theory of pattern avoiding permutations has seen tremendous progress during the last two decades. The key definition is the following. Let \(k\le n\), let \(p=p_1p_2\ldots p_n\) be a permutation of length \(n\), and let \(q=q_1q_2\ldots q_k\) be a permutation of length \(k\). We say that \(p\) avoids \(q\) if there are no \(k\) indices \(i_1<i_2<\cdots <i_k\) such that for all \(a\) and \(b\), the inequality \(p_{i_a}<p_{i_b}\) holds if and only if the inequality \(q_a<q_b\) holds. For instance, \(p=2537164\) avoids \(q=1234\) because \(p\) does not contain an increasing subsequence of length four. See [3] for an overview of the main results on pattern avoiding permutations.

A spectacular result of Adam Marcus and Gábor Tardos [6] shows that for all patterns \(q\), there exists a constant \(c_q\) such that \(S_n(q)\le c_q^n\) for all \(n\), and a short argument [1] then shows that this implies the existence of \(L(q)=\lim _{n\rightarrow \infty } \root n \of {S_n(q)}\).

The shortest pattern for which even some of the most basic questions are open is \(q=1324\), a pattern that has been studied for at least \(2\)0 years. For instance, there is no known exact formula for the number \(S_n(1324)\) of permutations of length \(n\) (or, in what follows, \(n\)-permutations) avoiding \(1324\). Even the value of \(L(1324)=\lim _{n\rightarrow \infty } \root n \of {S_n(1324)}\) is unknown. It is also known that no pattern of length four is easier to avoid than the pattern \(1324\), that is, for any pattern \(q\) of length four, the inequality \(S_n(q)\le S_n(1324)\) holds. The inequality is sharp unless \(q=4231\). See [3, Chapter 4] for a treatment of the series of results leading to these inequalities.

Recently, there has been some progress in the very challenging problem of determining \(L(1324)\). First, in 2011, Anders Claesson, Vit Jelínek and Einar Steingrímsson have proved that \(L(1324)\le 16\) [5]. A year later, in July of 2012, the present author improved that upper bound [4] showing that \(L(1324)\le 7+4\sqrt{3}<13.93\). As far as lower bounds go, in 2005 Albert et al. proved that \(L(1324)\ge 9.42\), which has recently been improved to \(L(1324)\ge 9.81\) by David Bevan [2].

In the present paper, we further refine the counting argument of [4], leading to the improved upper bound \(L(1324)\le 13.73718\). While this improvement is numerically not huge, the way in which it is obtained is interesting as we are able to exploit some dependencies between two words \(w(p)\) and \(z(p)\) related to the \(1324\)-avoiding permutation \(p\) that have previously been counted as though they were independent. A key element of our improved enumeration method is that we will consider pairs of words \((w,z)\) in which \(w\) and \(z\) do not have to be of the same length.

2 Words over a finite alphabet

2.1 Coloring

The starting point of our proof is the following decomposition of \(1324\)-avoiding permutations, given in [5] in a slightly different form, and then given in the present form in [4].

Let \(p=p_1p_2\ldots p_n\) be a \(1324\)-avoiding permutation, and let us color each entry of \(p\) red or blue as we move from left to right, according the following rules.

  1. (I)

    If coloring \(p_i\) red would create a \(132\)-pattern with all red entries, then color \(p_i\) blue, and

  2. (II)

    otherwise color \(p_i\) red.

It is then proved in [5] that the red entries form a \(132\)-avoiding permutation and the blue entries form a \(213\)-avoiding permutation. The following fact will be useful in the next section.

Proposition 2.1

[4] If an entry \(p_i\) is larger than a blue entry on its left, then \(p_i\) itself must be blue.

If \(p=p_1p_2\ldots p_n\) is a permutation, then we say that \(p_i\) is a left-to-right minimum if \(p_j>p_i\) for all \(j<i\). That is, \(p_i\) is a left-to-right minimum if it is smaller than all entries on its left. Similarly, we say that \(p_k\) is a right-to-left maximum if \(p_k>p_\ell \) for all \(\ell >k\). In other words, \(p_k\) is a right-to-left maximum if it is larger than all entries on its right. It is easy to see that \(p_1\) and \(1\) are always left-to-right minima, \(p_n\) and \(n\) are always right-to-left maxima, and both the sequence of left-to-right minima and the sequence of right-to-left maxima are decreasing.

In [4], the decomposition of \(1324\)-avoiding permutations given by rules (I) and (II) above was refined as follows. Let us color each entry of the \(1324\)-avoiding permutation \(p=p_1p_2\ldots p_n\) red or blue as above. Furthermore, let us mark each entry of \(p\) with one of the letters \(A, B, C\), or \(D\) as follows.

  1. (a)

    Mark each red entry that is a left-to-right minimum in the partial permutation of red entries by \(A\),

  2. (b)

    mark each red entry that is not a left-to-right minimum in the partial permutation of red entries by \(B\),

  3. (c)

    mark each blue entry that is not a right-to-left maximum in the partial permutation of blue entries by \(C\), and

  4. (d)

    mark each blue entry that is a right-to-left maximum in the partial permutation of blue entries by \(D\).

Call entries marked by the letter \(X\) entries of type \(X\). Let \(w(p)\) be the \(n\)-letter word over the alphabet \(\{A,B,C,D\}\) defined above. In other words, the \(i\)th letter of \(w(p)\) is the type of \(p_i\) in \(p\). Let \(z(p)\) be the \(n\)-letter word over the alphabet \(\{A,B,C,D\}\) whose \(i\)th letter is the type of the entry of value \(i\) in \(p\). In other words, \(w(p)\) is obtained by reading the letter marking the entries of \(p\) from left to right, while \(z(p)\) is obtained by reading these letters from the bottom up.

Example 2.2

Let \(p=3612745\). Then the subsequence of red entries of \(p\) is \(36127\), the subsequence of blue entries of \(p\) is \(45\), so \(w(p)=ABABBCD\), while \(z(p)=ABACDBB\). See Fig. 1 for an illustration.

Fig. 1
figure 1

Colors and labels for \(p=3612745\) (Color figure online)

The following important fact is proved in [4]. Let \(\mathrm{NO}_n(CB)\) be the set of all words of length \(n\) over the alphabet \(\{A,B,C,D\}\) that contain no \(CB\)-factors, that is, that do not contain a letter \(C\) immediately followed by a letter \(B\). Let \(\mathrm{Av}_n(1324)\) denote the set of all \(1324\)-avoiding permutations of length \(n\).

Theorem 2.3

The map \(f:\mathrm{Av}_n(1324)\rightarrow \mathrm{NO}_n(CB) {\times } \mathrm{NO}_n(CB)\) given by \(f(p)=(w(p),z(p))\) is an injection.

Consequently, \(S_n(1324)=|\mathrm{Av}_n(1324)| \le | \mathrm{NO}_n(CB)|^2\), leading to the upper bound

$$\begin{aligned} S_n(1324)\le (7+4\sqrt{3})^n . \end{aligned}$$
(1)

3 A connection between \(w(p)\) and \(z(p)\)

3.1 A small modification of the encoding

One wasteful element of the method used to prove the upper bound (1) that we sketched in the last section is that it treats \(w(p)\) and \(z(p)\) as two independent words for enumeration purposes, which leads to the bound \(|\mathrm{Av}_n(1324)|\le |\mathrm{NO}_n(CB)|^2\). The full extent of the dependencies between \(w(p)\) and \(z(p)\) is difficult to describe, but in this paper, we describe it well enough to improve the upper bound for the exponential growth rate of \(S_n(1324)\).

First, we modify the letter encoding of \(1324\)-avoiding permutations a little bit by adding the following last rule to the existing rules (a)–(d).

(d\('\)):

If an entry is a right-to-left maximum in all of \(p\), but not a left-to-right minimum in \(p\), then color it blue and mark it \(D\), regardless what earlier rules said.

In the rest of this paper, we will use rules (a)–(d) together with (d\('\)). With a slight abuse of notation, for the sake of brevity, we denote the pair of words encoding \(p\) by these rules by \(w(p)\) and \(z(p)\), just as we did before rule (d\('\)) was added. This will not create confusion, since the older, narrower set of rules will not be used. The advantage of adding rule (d\('\)) is that it ensures that every letter \(C\) is eventually followed by a letter \(D\) so that the entry corresponding to that \(D\) is larger than the entry corresponding to that given \(C\).

Note that if an entry changes its letter code because of this last rule, then that entry was previously a letter \(B\), and now it is a \(D\). It is straightforward to check that it is still true that the string of red entries forms a \(132\)-avoiding permutation (since their string simply lost a few entries) and the string of blue entries forms a \(213\)-avoiding permutation. Indeed, if \(u\) is a right-to-left minimum in \(p\) that has just become blue because of rule (d\('\)), then \(u\) could only play the role of \(3\) in a blue \(213\)-pattern \(yxu\). However, that would mean that before rule (d\('\)) was applied, the red entry \(u\) was on the right of the smaller blue entries \(x\) and \(y\), contradicting Proposition 2.1. It is also clear that if \(p\in \mathrm{Av}_n(1324)\), then neither \(w(p)\) nor \(z(p)\) contains a \(CB\)-factor (since the set of letters \(B\) shrank), and it is straightforward to show that the new map \(p\mapsto (w(p),z(p))\) is still injective. Indeed, given \(w(p)\) and \(z(p)\), we can first use the letters \(A\) and \(B\) in these words to recover the “red part” of \(p\), then we can use the letters \(C\) and \(D\) in these words to recover the ”blue part” of \(p\). See [4] for details.

3.2 A connection between \(w(p)\) and \(z(p)\)

We are now ready to describe some connections between \(w(p)\) and \(z(p)\). We define a segment in a finite word \(v\) over the alphabet \(\{A,B,C,D\}\) is a subword of consecutive letters that starts with an \(A\) and ends immediately before the next letter \(A\), or at the end of \(v\). For instance, the word \(v=ABBDCACDB\) has two segments. The key observation is the following.

Lemma 3.1

Let \(p\) be a \(1324\)-avoiding permutation, and let \((w(p),z(p))\) be its image under the coloring defined by rules (a)–(d) and \((\mathrm{d}')\). Then for all \(i\), the following holds. If the \(i\)th letter \(A\) from the right in \(w(p)\) is in the middle of a \(CAB\)-factor, then the \(i\)th segment of \(z(p)\) from the left must contain a letter \(B\).

Proof

Let us assume that the statement does not hold, that is, there is \(i\) that provides a counterexample. Let \(a\) be the \(i\)th smallest left-to-right minimum in \(p\). That means that \(a\) corresponds to the \(i\)th letter \(A\) from the left in \(z(p)\), and to the \(i\)th letter \(A\) from the right in \(w(p)\). Let us say that the letters corresponding to the entries \(cab\) of \(p\) form a \(CAB\)-factor in \(w(p)\). Find the closest entry \(d\) on the right of \(b\) that corresponds to a letter \(D\), and the closest entry \(a'\) on the left of \(c\) that corresponds to a letter \(A\). Then the entries \(a'cbd\) will form a \(1324\)-pattern in \(p\) unless \(b<a'\). However, that would mean that \(a<b<a'\), that is, there is a letter \(B\) in the \(i\)th segment of \(z(p)\) after all, contradicting the assumption that \(i\) is a counterexample. \(\square \)

The following simple proposition will be useful for us.

Proposition 3.2

Let \(s_n\) be the number of segments of length \(n\) that do not contain a \(CB\)-factor. Let \(S(x)=\sum _{n\ge 1} s_nx^n\). Then

$$\begin{aligned} S(x)=\frac{x}{x^2-3x+1}. \end{aligned}$$

Proof

By definition, \(s_0=0\) (since any segment must contain a letter \(A\) at its front), and \(s_1=1\). If \(n\ge 2\), then \(s_n=3s_{n-1}-s_{n-2}\), since we get a segment of length \(n\) if we affix a letter \(B, C\), or \(D\) at the end of any segment of length \(n-1\), except in the \(s_{n-2}\) cases when this results in a \(CB\)-factor at the end. \(\square \)

We are now ready to announce our main tool of this section. Let \(h_n\) be the number of the pairs of words \((w,z)\) over the alphabet \(\{A,B,C,D\}\) that satisfy the following requirements.

  1. (i)

    Both \(w\) and \(z\) start with the letter \(A\), and \(|w|+|z|=n\),

  2. (ii)

    the words \(w\) and \(z\) contain the same number of letters \(A\),

  3. (iii)

    neither \(w\) nor \(z\) contains a \(CB\)-factor, and

  4. (iv)

    for all \(i\), if the \(i\)th letter \(A\) from the right in \(w\) is in the middle of a \(CAB\)-factor, then the \(i\)th segment of \(z\) (from the left) contains a letter \(B\).

Note that the words \(w\) and \(z\) do not have to have the same length.

Theorem 3.3

Let \(h_n\) be as above and \(H(x)=\sum \nolimits _{n\ge 2} h_n x^n\). Then

$$\begin{aligned} H(x)=\sum _{n\ge 2}h_nx^{n}= \frac{x^2(1-2x)}{x^6-5x^5+14x^4-26x^3+22x^2-8x+1}. \end{aligned}$$

Proof

If a pair of words \((w,z)\) enumerated by \(H(x)\) contains a total of two letters \(A\), then both \(w\) and \(z\) must consist of a single segment, since each segment contains a letter \(A\). Let us now assume that \((w,z)\) contains more than two letters \(A\). Removing the last segment of \(w\) and the first segment of \(z\), we get another, shorter pair \((w',z')\) of words enumerated by \(H(x)\). On the other hand, inserting a new last segment \(S_1\) at the end of \(w'\) and a new first segment \(S_2\) at the front of \(z'\), we get a new pair of words enumerated by \(H(x)\) except when the newly inserted last letter \(A\) in \(w\) is in the middle of a \(CAB\)-factor, and the new first segment of \(z\) does not contain any letters \(B\).

This leads to the generating function identity

$$\begin{aligned} H(x)=S^2(x)+S^2(x)H(x) - \frac{x}{1-2x} \cdot \frac{x^2}{1-3x+x^2} \cdot xH(x). \end{aligned}$$
(2)

The last summand of the right-hand side is justified as follows. The generating function for \(S_2\) is \(x^2/(1-2x)\) since \(S_2\) starts with a letter \(A\), and then consists of letters \(C\) and \(D\) with no restrictions, while the generating function for \(S_1\) is \(x^2/(1-3x+x^2)\), since this segment starts with \(AB\), and then consists of a string over the alphabet \(\{B,C,D\}\) with no \(CB\)-factors, so it is just a segment with a letter \(B\) inserted into its second position. Finally, the rest of the pair \((w,z)\) is just a word counted by \(H(x)\) where a letter \(C\) is inserted at the end of \(w\).

Solving (2) for \(H(x)\), we get the statement of theorem. \(\square \)

Corollary 3.4

There exists a positive constant \(c\) such that \(h_n\le c{ \cdot } 3.709381^n\).

Proof

This is routine after noticing that the denominator of \( H(x)\) as given in Theorem 3.3 has a unique root of smallest modulus, namely \(\alpha \ge 0.2695867676\), and computing \(\beta =1/\alpha \le 3.709381\). \(\square \)

Now we can prove the main result of this section.

Theorem 3.5

\(L(1324)\le 13.76\).

Proof

This is immediate from the fact that if \(p\in \mathrm{Av}_n(1324)\), then the pair \(w(p),z(p)\) is in the set that is counted by \(h_{2n}\) and is defined by rules (i)–(iv). Indeed, \(w(p)\) and \(z(p)\) are both words of length \(n\) over the alphabet \(\{A,B,C,D\}\), they contain the same number of letters of each type, do not contain any \(CB\)-factors as shown in Theorem 2.3, and the pair \((w(p),z(p))\) satisfies condition (iv) by Lemma 3.1. Therefore, \(S_n(1324)\le h_{2n}\), and our result follows from Corollary 3.4 by computing \(\beta ^2\). \(\square \)

4 Extending the reach of our method

There are several other constraints that the pair \((w(p),z(p))\) must satisfy if \(p\) is a \(1324\)-avoiding permutation. The problem is that it is difficult to count pairs that satisfy all these constraints at once. On the other hand, taking only a few constraints into account results only in a small improvement of the result of Theorem 3.5.

For instance, Lemma 3.1 can be generalized in the following way. A \(CAB^k\)-factor in a word is a subword of \(k+2\) letters in consecutive positions, the first of which is a letter \(C\), the second of which is an \(A\), and all the others are \(B\).

Lemma 4.1

Let \(p\) be a \(1324\)-avoiding permutation, and let \((w(p),z(p))\) be its image under the coloring defined by rules (a)–(d) and \((\mathrm{d}')\). Then for all \(i\), and all \(k\), the following holds. If the \(i\)th letter \(A\) from the right in \(w(p)\) is in the middle of a \(CAB^k\)-factor, then the \(i\)th segment of \(z(p)\) from the left must contain at least \(k\) letters \(B\).

Proof

The proof is analogous to that of Lemma 3.1. The only change is that the role of \(b\) in that proof must be played by the \(k\)th letter \(B\) in the \(CAB^k\)-factor of the lemma. In order for \(p\) to be \(1324\)-avoiding, \(b<a'\) must hold (keeping the notation of the proof of Lemma 3.1). As \(b\) is clearly the largest of the \(k\) letters \(B\) immediately following \(a\), this implies that there are at least \(k\) letters \(B\) between \(a\) and \(a'\) in \(z(p)\). \(\square \)

Let us now check what numerical improvement we obtain if we enforce the constraint of Lemma 4.1 for \(k=2\).

Theorem 4.2

Let \(k_n\) be the number of pairs of words \((w,z)\) over the alphabet \(\{A,B,C,D\}\) that satisfy all of the rules (i)–(iv), and also the following rule.

  1. (v)

    For all \(i\), if the \(i\)th letter \(A\) from the right in \(w\) is in the middle of a \(CABB\)-factor, then the \(i\)th segment of \(z\) \((\)from the left\()\) contains at least two letters \(B\).

Let \(K(x)=\sum \limits _{n\ge 2}k_nx^n\). Then

$$\begin{aligned} K(x)=\frac{x^2(1-2x)^2}{1-10x+38x^2-70x^3+66x^4-33x^5+12x^6-6x^7+4x^8-x^9}. \end{aligned}$$

Proof

If a pair of words \((w,z)\) enumerated by \(K(x)\) contains a total of two letters \(A\), then both \(w\) and \(z\) must consist of a single segment. Otherwise, removing the last segment of \(w\) and the first segment of \(z\), we get another, shorter pair \((w',z')\) of words enumerated by \(K(x)\). On the other hand, inserting a new last segment \(S_1\) at the end of \(w'\) and a new first segment \(S_2\) at the front of \(z'\), we get a new pair of words enumerated by \(H(x)\) except in two disjoint cases. The first such case is when the newly inserted last letter \(A\) in \(w\) is in the middle of a \(CAB\)-factor, and the new first segment of \(z\) does not contain any letters \(B\). The second such case is when the newly inserted last letter \(A\) in \(w\) is in the middle of a \(CABB\)-factor, and the new first segment of \(z\) contains exactly one letter \(B\). This leads to the functional equation

$$\begin{aligned} K(x)&=\frac{x^2}{(1-3x+x^2)^2} \cdot (1+K(x)) - \frac{x}{1-2x} \cdot \frac{x^2}{1-3x+x^2} \cdot xK(x)\\&\qquad \qquad -\frac{x^3}{1-3x+x^2}\cdot x\cdot \biggl (\frac{x}{1-2x}+1\biggr )\cdot x\cdot \frac{1}{1-2x} \cdot xK(x). \end{aligned}$$

The first and second summand of the right-hand side can be explained just as in the proof of Theorem 3.3. The third summand counts the pairs \((w,z)\) described as the second case in the previous paragraph. In such pairs, \(S_2\) is a segment containing exactly one letter \(B\). Such segments are counted by the generating function

$$\begin{aligned} x\cdot \biggl (\frac{x}{1-2x}+1\biggr )\cdot x\cdot \frac{1}{1-2x}, \end{aligned}$$

since in such segments, a letter \(A\) is followed by a (possibly empty) sequence of \(C\)s and \(D\)s that ends in a letter \(D\) if it is not empty, then comes a letter \(B\), and then comes a sequence of \(C\)s and \(D\)s again. On the other hand, in such pairs, \(S_1\) is just a segment into which two letters \(B\) are inserted right after the first letter, leading to the generating function \(x^3/(1-3x+x^2)\). Finally, the rest of the pair \((w,z)\) is just a pair counted by \(K(x)\), with a letter \(C\) inserted at the end of the first word of the pair. Solving the last displayed equation for \(K(x)\), we get the statement of the theorem. \(\square \)

Corollary 4.3

There exists a constant \(c\) such that \(k_n\le c{\cdot } 3.70672^n\) holds for all \(n\).

Proof

This follows from the fact that \(K(x)\) is a rational function whose denominator has a unique root of smallest modulus, namely \(0.26978\). Taking its reciprocal, our claim is proved. \(\square \)

As \(S_n(1324)\le k_{2n}\) we have

Theorem 4.4

\(L(1324)\le 13.73977\).

It goes without saying that further improvements are possible if we enforce the restriction of Lemma 4.1 not only for \(k=2\), but for larger values of \(k\) as well. However, these improvements will be minuscule. Indeed, let us replace restrictions (iv) and (v) by the following general restriction.

  1. (vi)

    For all \(i\ge 1\) and all \(k\ge 1\), if the \(i\)th letter \(A\) from the right in \(w\) is in the middle of a \(CAB^k\)-factor, then the \(i\)th segment of \(z\) (from the left) contains at least \(k\) letters \(B\).

Theorem 4.5

Let \(t_n\) be the number of pairs \((w,z)\) that satisfy restrictions (i)–(iii) and (vi). Let \(T(x)=\sum _{n\ge 2}t_nx^n\). Then

$$\begin{aligned} T(x)=\frac{x^2(1-2x-x^2+x^3)}{1-8x+21x^2-19x^3-2x^4+11x^5-6x^6+x^7}. \end{aligned}$$

Proof

This follows from the functional equation

$$\begin{aligned} T(x)=\frac{x^2}{(1-3x+x^2)^2} \cdot (1+T(x)) - x\cdot T(x) \cdot \frac{x^3}{(1-3x+x^2)(1-2x-x^2+x^3)}, \end{aligned}$$

which can be proved in a way that is analogous to the proofs of Theorems 3.3 and 4.2. Note that the term \(1-2x-x^2+x^3\) in the denominator is obtained from the infinite sum

$$\begin{aligned} \sum _{k\ge 0} \biggl (x^2 \cdot \biggl (1+\frac{x}{1-2x}\biggr ) \biggr )^{\!k} = \sum _{k\ge 0} \biggl ( \frac{x^2-x^3}{1-2x} \biggr )^{\!k} =\frac{1-2x}{1-2x-x^2+x^3}, \end{aligned}$$

which is explained by the enforcement of restriction (vi) for all \(k\). \(\square \)

Corollary 4.6

\(L(1324) \le 13.73719\).

Proof

This is immediate if we notice that \(S_n(1324)\le t_{2n}\), and find the root of smallest modulus of the denominator of \(T(x)\). \(\square \)

Other minor improvements can be obtained if one considers \(CAAB\)-factors in \(w(p)\).

A more substantial improvement could possibly be obtained in the following way. The restrictions involving \(CAB\)-factors that we described have dual versions that involve \(CDB\)-factors. If one could find a way to count pairs of words that satisfy both kinds of restrictions at once, (that is, restrictions involving \(CDB\)-factors and restrictions involving \(CAB\)-factors), then a somewhat better upper bound could be obtained.