Co-lexicographically Ordering Automata and Regular Languages - Part I

Let \(\mathcal {N}\) be an NFA on an alphabet of cardinality \(\sigma\) with n states and \(|\delta |\) transitions, and let \(p=\text{width}(\mathcal {N})\). Let \(\le\) be a co-lex order of width p, \(\lbrace Q_i \;|\; 1 \le i \le p \rbrace\) be a \(\le\)-chain partition of the set of states, and for all \(1\le i \le p\) let \(Q_i = \lbrace v_{i,1},\dots , v_{i,n_i}\rbrace\), where \(n_i = |Q_i|\) and \(v_{i,k} \lt v_{i,k^{\prime }}\) whenever \(k\lt k^{\prime }\). Fix \(1\le i,j \le p\) and \(a\in \Sigma\), and consider all the transitions \((v_{i, k}, v_{j, l}, a)\) labeled with a that leave \(Q_i\) and reach \(Q_j\). We denote with \(e_{i,j,a}\) the number of such transitions; our goal is to establish an upper bound to this quantity for all \(i,j,a\). Sort these \(e_{i,j,a}\) edges \((v_{i,k}, v_{j, l}, a)\) by the index l of their destination state, breaking ties by the index k of their source state. Now, let us prove that the value \(k + l\) is strictly increasing with respect to this order. In order words, we want to prove that if we pick two edges \((v_{i, k}, v_{j, l}, a)\) and \((v_{i, k^{\prime }}, v_{j, l^{\prime }}, a)\) being consecutive with respect to the edge order, then \(k + l \lt k^{\prime } + l^{\prime }\). By the definition of the edge order, we have \(l \le l^{\prime }\). If \(l = l^{\prime }\), again by the definition of the edge order we have \(k \lt k^{\prime }\) and so \(k + l \lt k^{\prime } + l^{\prime }\). If \(l \lt l^{\prime }\), by Axiom 2 of co-lex orders we have \(k \le k^{\prime }\), and again we conclude \(k + l \lt k^{\prime } + l^{\prime }\). Since we have proved that \(k + l\) is strictly increasing with respect to the edge order, then from \(2\le k + l \le n_i+n_j\) we obtain \(e_{i,j,a} \le n_i+n_j-1\). Observing that \(\sum _{i=1}^p n_i = n\), we conclude: \( \begin{equation*} |\delta | = \sum _{a\in \Sigma }\sum _{i=1}^p\sum _{j=1}^p e_{i,j,a} \le \sum _{a\in \Sigma }\sum _{i=1}^p\sum _{j=1}^p (n_i+n_j-1) = 2\sigma p n - \sigma p^2 = (2n - p) p \sigma .\end{equation*} \) \(□\)

First, let us prove that \(\le _\mathcal {D}\) is a co-lex order on \(\mathcal {D}\).

To see that Axiom 1 holds assume that \(u \lt _\mathcal {D} v\): we must prove that \(e=\text{max}_{\lambda (u)} \preceq e^{\prime }=\text{min}_{\lambda (v)}\). Notice that it must be \(v \not= s\) because the empty string \(\varepsilon\) is in \(I_s\) and \(\varepsilon\) is co-lexicographically smaller than any other string. Hence, \(e^{\prime }\succ \#\) and if \(e= \#\) we are done. Otherwise, there are \(\alpha e \in I_u\) and \(\alpha ^{\prime } e^{\prime } \in I_{v}\), so that \(u \lt _\mathcal {D} v\) implies \(\alpha e \prec \alpha ^{\prime } e^{\prime }\) and therefore \(e \preceq e^{\prime }\). As for Axiom 2, assume that \(u \in \delta (u^{\prime }, a)\), \(v \in \delta (v^{\prime }, a)\), and \(u \lt _\mathcal {D} v\). We must prove that \(u^{\prime } \lt _\mathcal {D} v^{\prime }\). Fixing \(\alpha \in I_{u^{\prime }}\) and \(\beta \in I_{v^{\prime }}\), we must prove that \(\alpha \prec \beta\). We have \(\alpha a \in I_u\) and \(\beta a \in I_{v}\), hence from \(u \lt _\mathcal {D} v\) it follows \(\alpha a \prec \beta a\), and therefore \(\alpha \prec \beta\).

Let us now prove that \(\le _\mathcal {D}\) is the maximum co-lex order.

Suppose, reasoning for contradiction, that \(\le\) is a co-lex order on \(\mathcal {D}\) and for some distinct \(u,v \in Q\), \(u \lt v\), and \(u \not\lt _\mathcal {D} v\). Then, there exist \(\alpha \in I_u, \beta \in I_v\) with \(\beta \prec \alpha\). Let us fix u, v, \(\alpha\) and \(\beta\) with the above properties such that \(\beta\) has the minimum possible length. Notice that \(\beta\) cannot be the empty word, otherwise v would be the initial state s, while \(z\not\lt s\) for all \(z\in Q\) (see Remark 2.2). Hence, \(\beta =\beta ^{\prime } e\) for some \(e\in \Sigma\) and \(\beta \prec \alpha\) implies \(\alpha =\alpha ^{\prime }f\) for some \(f\in \Sigma\). We then have \(e\in \lambda (v), f \in \lambda (u)\), and by Axiom 1 of co-lex orders we get \(f\preceq e\). From \(\beta =\beta ^{\prime }e\prec \alpha =\alpha ^{\prime }f\) we conclude \(f=e\) and \(\beta ^{\prime }\prec \alpha ^{\prime }\). If \(u^{\prime },v^{\prime }\) are such that \(\delta (u^{\prime },e)=u, \delta (v^{\prime }, e)=v\) and \(\alpha ^{\prime }\in I_{u^{\prime }}, \beta ^{\prime }\in I_{v^{\prime }}\), then by Axiom 2 of co-lex orders and Remark 2.2 we get \(u^{\prime }\lt v^{\prime }\); however, the pair \(\alpha ^{\prime },\beta ^{\prime }\) witnesses \(u^{\prime }\not\le _\mathcal {D}v^{\prime }\), contradicting the minimality of \(\beta\).\(□\)

Let \(\alpha \in I_u\) and \(\beta \in I_v\) such that \(\lbrace \alpha , \beta \rbrace \not\subseteq I_u \cap I_v\). We must prove that \(\alpha \prec \beta\). Let \(\gamma \in \Sigma ^*\) be the longest string such that \(\alpha = \alpha ^{\prime } \gamma\) and \(\beta = \beta ^{\prime } \gamma\), for some \(\alpha ^{\prime }, \beta ^{\prime } \in \text{Pref}(\mathcal {L(N)})\). If \(\alpha ^{\prime } = \varepsilon\) the claim follows, therefore we can assume \(|\alpha ^{\prime } | \ge 1\).

Let \(\gamma = c_p \dots c_1\), with \(c_i \in \Sigma\) for \(i \in \lbrace 1, \dots , p \rbrace\) (\(p \ge 0\)), \(\alpha ^{\prime } = a_q \dots a_1\), with \(a_i \in \Sigma\) for \(i \in \lbrace 1, \dots , q \rbrace\) (\(q \ge 1\)), and \(\beta ^{\prime } = b_r \dots b_1\), with \(b_i \in \Sigma\) for \(i \in \lbrace 1, \dots , r \rbrace\) (\(r \ge 0\)).

Assume \(|\gamma | \gt 0\). Since \(\alpha \in I_u\) and \(\beta \in I_v\), then there exist \(u_1, v_1 \in Q\) such that \(\alpha ^{\prime } c_p \dots c_2 \in I_{u_1}\), \(\beta ^{\prime } c_p \dots c_2 \in I_{v_1}\), \(u \in \delta (u_1, c_1)\) and \(v \in \delta (v_1, c_1)\). By Axiom 2, we obtain \(u_1 \le v_1\). However, it cannot be \(u_1 = v_1\) because this would imply \(\lbrace \alpha , \beta \rbrace \subseteq I_u \cap I_v\), so it must be \(u_1 \lt v_1\). By iterating this argument, we conclude that there exist \(u^{\prime }, v^{\prime } \in Q\) such that \(\alpha ^{\prime } \in I_{u^{\prime }}\), \(\beta ^{\prime } \in I_{v^{\prime }}\) and \(u^{\prime } \lt v^{\prime }\), and the same conclusion holds if \(\gamma =\varepsilon\) as well.

Now, it cannot be \(r = 0\) because this would imply \(v^{\prime } = s\), and \(u^{\prime } \lt s\) contradicts Remark 2.2. Hence, it must be \(|\beta | \ge 1\). By Axiom 1, it must be \(a_1 \preceq b_1\). At the same time, the definition of \(\gamma\) implies that it cannot be \(a_1 = b_1\), so we obtain \(a_1 \prec b_1\) and we can conclude \(\alpha \prec \beta\).\(□\)

Suppose \(u,z \in I_\alpha\) and let \(v\in Q\) be such that \(u\lt v\lt z\). We have to prove that \(v\in I_\alpha\). If this were not true, we would have \(\alpha \in (I_u \cap I_z) \setminus I_v\). Consider any \(\beta \in I_v\). By Lemma 2.13 we would have \(\alpha \prec \beta \prec \alpha\), a contradiction.\(□\)

It is enough to prove that \(\text{ent}(\mathcal {D}_{\mathcal {L}})\le \text{ent}(\mathcal {D})\), for any DFA \(\mathcal {D}\) such that \(\mathcal {L}(\mathcal {D}_{\mathcal {L}})= \mathcal {L}(\mathcal {D})\). Suppose \(u_1, \dots , u_k\) are pairwise distinct states which are entangled in \(\mathcal {D}_{\mathcal {L}}\), witnessed by the monotone sequence \((\alpha _i)_{i\in \mathbb {N}}\). Since \(\mathcal {D}_{\mathcal {L}}\) is minimum, each \(I_{u_j}\) is a union of a finite number of \(I_v\), with \(v\in Q_{\mathcal {D}}\). The monotone sequence \((\alpha _i)_{i\in \mathbb {N}}\) goes through \(u_j\) infinitely often, so there must be a state \(v_j\in Q_{\mathcal {D}}\) such that \(I_{v_j}\subseteq I_{u_j}\) and \((\alpha _i)_{i\in \mathbb {N}}\) goes through \(v_j\) infinitely often. Then \((\alpha _i)_{i\in \mathbb {N}}\) goes through the pairwise distinct states \(v_1,\dots , v_k\) infinitely often and \(v_1, \dots , v_k\) are entangled in \(\mathcal {D}\).\(□\)

The existence of such a decomposition is guaranteed by Theorem A.5 of Appendix A, applied to \((Z,\le)=(\text{Pref} (\mathcal {L}(\mathcal {D})), \preceq)\) and \(\mathcal {P}= \lbrace I_u \;|\; u\in Q\rbrace\).\(□\)

We already proved that \(\text{ent}(\mathcal {D})\le \text{width}(\mathcal {D})\) in Lemma 4.8. In order to prove the reverse inequality, let \(\text{width}(\mathcal {D}) =p\) and let \(u_1, \dots , u_p\) be p \(\preceq _{\mathcal {D}}\)-incomparable states. If we prove that \(u_1, \dots , u_p\) are entangled, we are done. Fix \(i\in \lbrace 1, \dots ,p\rbrace\) and denote by \(W_i\) the convex set \(V_{h}\cup V_{h+1}\cup \dots \cup V_{k}\) where \(I_{u_i}\subseteq V_{h}\cup V_{h+1}\cup \dots \cup V_{k}\) and \(V_j\cap I_{u_i}\ne \emptyset\), for all \(h\le j\le k\). From the incomparability of the \(u_i\)’s it follows that \(W_i\cap W_j\ne \emptyset\), for all pairs \(i,j\), so that \(\bigcap _i W_i\ne \emptyset\) by Lemma A.13 in Appendix A. Since all \(W_i\)’s are unions of consecutive elements in the partition \(\mathcal {V}\), there must be an element \(V\in \mathcal {V}\) with \(V\subseteq \bigcap _i W_i\). Such a V must contain an occurrence of every \(u_i\), and since V is an entangled set, we conclude that \(u_1, \dots , u_p\) are entangled.\(□\)

Consider an e.c. decomposition \(\mathcal {V}=\lbrace V_1, \dots , V_m\rbrace\) of \(\mathcal {D}\), whose existence is guaranteed by Theorem 4.11. Since all \(V_i\)’s are entangled convex sets in \(\text{Pref} (\mathcal {L}(\mathcal {D}))\), two words belonging to the same \(V_i\) and ending in the same state belong to the same \(\sim\)-class. Hence, the number of \(\sim\)-classes is at most \(m\times |Q|\). Moreover, \(\alpha \sim \beta\) implies \(\delta (s,\alpha)=\delta (s,\beta)\). Hence, \(\alpha \in \mathcal {L}(\mathcal {D})\) and \(\alpha \sim \beta\) imply \(\beta \in \mathcal {L}(\mathcal {D})\), proving that \(\mathcal {L}(\mathcal {D})\) is equal to the union of some \(\sim\)-classes.\(□\)

Let \(\mathcal {L}=\mathcal {L}(\mathcal {D})\), assume \(\alpha \sim \alpha ^{\prime }\) and let \(a\in \Sigma\) be such that \(\alpha a \in \text{Pref} (\mathcal {L})\). We must prove that \(\alpha ^{\prime } a\in \text{Pref} (\mathcal {L})\) and \(\alpha a\sim \alpha ^{\prime } a\). Since \(\alpha \sim \alpha ^{\prime }\) we know that \(\delta (s,\alpha)= \delta (s, \alpha ^{\prime })\) and there exist entangled convex sets \(C_1, \dots , C_n\) such that \([\alpha , \alpha ^{\prime }]^\pm \subseteq C_1 \cup \dots \cup C_n\) and \(C_i\cap I_{\delta (s,\alpha)}\ne \emptyset\) for all \(i=1, \dots ,n\). We must prove that \(\alpha ^{\prime } a\in \text{Pref} (\mathcal {L})\), \(\delta (s,\alpha a)= \delta (s, \alpha ^{\prime } a)\), and there exist entangled convex sets \(C^{\prime }_1, \dots , C^{\prime }_{n^{\prime }}\) such that \([\alpha a, \alpha ^{\prime } a]^\pm \subseteq C^{\prime }_1 \cup \dots \cup C^{\prime }_{n^{\prime }}\) and \(C^{\prime }_i\cap I_{\delta (s,\alpha a)}\ne \emptyset\) for all \(i=1, \dots ,n^{\prime }\).

From \(\delta (s,\alpha)= \delta (s, \alpha ^{\prime })\) and \(\alpha a \in \text{Pref} (\mathcal {L})\) we immediately obtain \(\alpha ^{\prime } a \in \text{Pref} (\mathcal {L})\) and \(\delta (s,\alpha a)= \delta (s, \alpha ^{\prime } a)\). Moreover, from \([\alpha , \alpha ^{\prime }]^\pm \subseteq C_1 \cup \dots \cup C_n\) we obtain \([\alpha a, \alpha ^{\prime } a]^\pm = [\alpha , \alpha ^{\prime }]^\pm a \subseteq C_1a \cup \dots \cup C_na\), and from \(C_i\cap I_{\delta (s,\alpha)}\ne \emptyset\) we obtain \(C_i a \cap I_{\delta (s,\alpha a)}\ne \emptyset\). We are only left with showing that every \(C_i a =\lbrace \gamma a \; | \: \gamma \in C_i\rbrace\) is an entangled convex set. The fact that they are convex follows directly from the definition of co-lex ordering. Let us prove that the \(C_i a\)’s are entangled. Fix i and consider a monotone sequence \((\alpha _j)_{j \in \mathbb {N}}\) witnessing that \(C_i\) is entangled. Then \((\alpha _j a)_{j \in \mathbb {N}}\) is a monotone sequence witnessing that \(C_i a\) is entangled.\(□\)

To prove that \(\alpha \sim \alpha ^{\prime }\) holds under the above hypotheses, it is sufficient to recall that \(V_i , \dots , V_j\) are entangled convex sets and apply Definition 4.13.

Let us prove the reverse implication. Pick \(\alpha \sim \alpha ^{\prime } \in \text{Pref}(\mathcal {L(D)})\) and let \(C_1, \dots ,C_n\) be entangled convex sets such that \([\alpha , \alpha ^{\prime }]^{\pm } \subseteq \bigcup _{i=1}^n C_i\) and \(C_i\cap I_u\ne \emptyset\), for every \(i=1, \dots ,n\), where \(u = \delta (s,\alpha)= \delta (s, \alpha ^{\prime })\). By Lemma A.12 of Appendix A we can assume that \(C_1\prec C_2 \prec \dots \prec C_n\). Let \(i\le j\) be such that \([\alpha , \alpha ^{\prime }]^{\pm } \subseteq \bigcup _{h=i}^j V_h\) and \(V_h\cap [\alpha , \alpha ^{\prime }]^{\pm } \ne \emptyset\) for all \(h\in \lbrace i, \dots , j\rbrace\). We just have to prove that \(V_h\cap I_{u}\ne \emptyset\), for all \(h\in \lbrace i, \dots , j \rbrace\). From \(V_h\cap [\alpha , \alpha ^{\prime }]^{\pm } \ne \emptyset\) it follows that either \(V_h\) contains \(\alpha\) or \(\alpha ^{\prime }\), or \(V_h\subseteq [\alpha , \alpha ^{\prime }]^{\pm } \subseteq \bigcup _{i=1}^n C_i\). In the first case we have \(V_h\cap I_{u}\ne \emptyset\) because \(\alpha , \alpha ^{\prime }\in I_u\), while in the second case \(V_h\cap I_{u}\ne \emptyset\) follows from Lemma A.9 of Appendix A.\(□\)

In order to prove that an e.c. decomposition of minimum size \(V_1\prec \dots \prec V_r\) of \(\mathcal {D}\) is also an e.c. decomposition of \(\mathcal {D}^{\prime }\) we just have to check that \(V_h\) is entangled in \(\mathcal {D}^{\prime }\), for all \(h=1, \dots ,r\). Let \(u_1^{\prime }, \dots , u^{\prime }_k\) be the pairwise distinct \(\mathcal {D}^{\prime }\)-states occurring in \(V_h\). Notice that by the definition of \(\delta ^{\prime }\) we have \(\lbrace u_1^{\prime }, \dots , u^{\prime }_k\rbrace =\lbrace \delta ^{\prime }(s^{\prime },\alpha) \;|\; \alpha \in V_h\rbrace = \lbrace [\alpha ]_\sim \;|\; \alpha \in V_h\rbrace\). Hence, for every \(j = 1, \dots , k\) there exists \(\alpha _j\in V_h\) such that \(u_j^{\prime } = [\alpha _j]_\sim\). Then the \(\mathcal {D}\)-states \(u_j=\delta (s, \alpha _j)\), for \(j=1, \dots ,k\), occur in \(V_h\). Notice that \(u_1, \dots , u_k\) are pairwise distinct as well: if \(u_i\) were equal to \(u_j\) for \(i\ne j\), then from Lemma 4.17 we would have \(\alpha _i\sim \alpha _j\) and \(u_i^{\prime }=[\alpha _i]_\sim =[\alpha _j]_\sim =u_j^{\prime }\) would follow. Since \(V_h\) is entangled in \(\mathcal {D}\), there exists a monotone sequence \((\beta _i)_{i\in \mathbb {N}}\) in \(V_h\) reaching each \(u_j\) infinitely many times. Fix \(j\in \lbrace 1, \dots ,k\rbrace\). If \(\delta (s,\beta _i)=u_j\) then from \(\delta (s,\alpha _j)=u_j\) and \(\beta _i, \alpha _j\in V_h\) it follows \(\alpha _j\sim \beta _i\) again by Lemma 4.17, so that \(\delta ^{\prime }(s^{\prime },\beta _i)=[\beta _i]_\sim =[\alpha _j]_\sim =u_j^{\prime }\) in \(\mathcal {D}^{\prime }\). It follows that the sequence \((\beta _i)_{i\in \mathbb {N}}\) reaches \(u_j^{\prime }\) infinitely many times. Hence, \(V_h\) is entangled in \(\mathcal {D}^{\prime }\).

As for the second part of the Corollary, if \(u^{\prime }=[\alpha ]_\sim \in Q^{\prime }\) then \(I_{u^{\prime }}=[\alpha ]_\sim\) (see Equation (5) above). Let i (j, respectively) be the minimum (maximum) index h with \([\alpha ]_\sim \cap V_h\ne \emptyset\); then \([\alpha ]_\sim \subseteq V_i \cup \dots \cup V_j\). Fix \(h\in \lbrace i, \dots , j \rbrace\) and consider \(u=\delta (s,\alpha)\). Since there exist \(\alpha ^{\prime }, \alpha ^{\prime \prime } \in \text{Pref}(\mathcal {L(D)})\) such that \(\alpha ^{\prime } \sim \alpha \sim \alpha ^{\prime \prime }\), \(\alpha ^{\prime } \in V_i\) and \(\alpha ^{\prime \prime } \in V_j\), then, Lemma 4.17 implies \(\delta (s, \alpha ^{\prime }) = \delta (s, \alpha ^{\prime \prime }) = \delta (s, \alpha) = u\) and \(V_h\cap I_{u} \ne \emptyset\). Pick \(\beta \in V_h\cap I_{u}\). Then, the same lemma implies \(\beta \sim \alpha\) so that \(\beta \in I_{u^{\prime }}\) and \(V_h\cap I_{u^{\prime }} \ne \emptyset\) as well.\(□\)

Let us prove that \(\text{ent}(\mathcal {D}^{\prime })\le \text{ent}(\mathcal {D})\). Consider an entangled collection \(\lbrace [\alpha _1]_\sim , \dots , [\alpha _h]_\sim \rbrace\) of h states in \(\mathcal {D^{\prime }}\). Then, there is a monotone sequence \((\gamma _i)_{i\in \mathbb {N}}\) such that, for each \(j\in \lbrace 1,\dots ,h\rbrace\) we have \(\delta ^{\prime }(s^{\prime }, \gamma _i)= [\alpha _j]_\sim\) for infinitely many i’s. Let \(\mathcal {V}\) be a minimum-size finite e.c. decomposition of \(\mathcal {D}\). Since \(\mathcal {V}\) is a finite partition and all the elements of \(\mathcal {V}\) are convex, there exists \(V \in \mathcal {V}\) and \(n_0\) such that \(\gamma _i\in V\) for all \(i\ge n_0\). In particular, there are words \(\beta _1, \dots ,\beta _h\) in V such that \(\delta ^{\prime }(s^{\prime }, \beta _{k}) = [\alpha _k]_\sim\), for every \(k = 1, \dots , h\). Define \(u_k = \delta (s, \beta _{k})\) and notice that the states \(u_1, \dots , u_{h}\) are pairwise distinct. In fact, if \(u_r = u_s\) for \(r\ne s\), then by Lemma 4.17 we would have \(\beta _{r} \sim \beta _{s}\) and \([\alpha _r]_\sim =\delta ^{\prime }(s^{\prime }, \beta _{r})=[\beta _r]_\sim = [\beta _s]_\sim =\delta ^{\prime }(s^{\prime }, \beta _{s})= [\alpha _s]_\sim\), a contradiction. Moreover, \(\lbrace u_1, \dots , u_{h}\rbrace\) is an entangled set in \(\mathcal {D}\), because all these states occur in V (as witnessed by \(\beta _{ 1}, \dots , \beta _{ h}\)) and V is an element of an e.c. decomposition. Since this holds for any collection of entangled states in \(\mathcal {D}^{\prime }\), it follows that \(\text{ent}(\mathcal {D}^{\prime })\le \text{ent}(\mathcal {D})\).

Let us now prove that \(\text{ent}(\mathcal {D})\le \text{ent}(\mathcal {D}^{\prime })\). Let \(\lbrace u_1, \dots , u_{h} \rbrace\) be an entangled set of h states in \(\mathcal {D}\), witnessed by some monotone sequence \((\alpha _i)_{i \in \mathbb {N}}\). Every \(I_{u_k}\) is equal to a finite union of some \(I_{u^{\prime }}\)’s, with \(u^{\prime } \in Q^{\prime }\), and \((\alpha _i)_{i \in \mathbb {N}}\) goes through any \(u_k\) infinitely many times. Therefore, for all \(k=1, \dots h\) there exist \(u^{\prime }_k \in Q^{\prime }\) such that \(I_{u^{\prime }_k} \subseteq I_{u_k}\) and \((\alpha _i)_{i \in \mathbb {N}}\) goes through \(u^{\prime }_k\) infinitely many times. We conclude that \(u^{\prime }_1, \dots , u^{\prime }_h\) are pairwise distinct and \(\lbrace u^{\prime }_1, \dots , u^{\prime }_h \rbrace\) is an entangled set of states in \(\mathcal {D^{\prime }}\), which implies \(\text{ent}(\mathcal {D})\le \text{ent}(\mathcal {D}^{\prime })\).

Finally, we prove that \(\text{width}(\mathcal {D}^{\prime })= \text{ent}(\mathcal {D}^{\prime })\). If \(\mathcal {V}\) is an e.c. decomposition of \(\mathcal {D}\) of minimum size, then Corollary 4.18 implies that \(\mathcal {V}\) is an e.c. decomposition of \(\mathcal {D}^{\prime }\) satisfying the hypothesis of Lemma 4.12 so that \(\text{width}(\mathcal {D}^{\prime })= \text{ent}(\mathcal {D}^{\prime })\) follows from this lemma.\(□\)

By Lemma 4.9 we have \(\text{ent}(\mathcal {D}_{\mathcal {L}})= \text{ent}(\mathcal {L})\). Since \(\text{ent}(\mathcal {D})\le \text{width}(\mathcal {D})\) for all DFAs (Lemma 4.8), we obtain \(\text{ent}(\mathcal {L})\le \text{width}^D(\mathcal {L}),\) while from Theorem 4.19 we know that \(\text{width}(\mathcal {H}_{\mathcal {L}})= \text{ent}(\mathcal {D}_{\mathcal {L}})\). Hence, we have: \(\begin{equation*} \text{width}(\mathcal {H}_{\mathcal {L}})= \text{ent}(\mathcal {D}_{\mathcal {L}})= \text{ent}(\mathcal {L}) \le \text{width}^D(\mathcal {L})\le \text{width}(\mathcal {H}_{\mathcal {L}}) \end{equation*}\) and the conclusion follows.\(□\)

Let \(\mathcal {D}_{\mathcal {L}}\) be the minimum DFA for \(\mathcal {L}\). By definition, k states \(u_1,\dots ,u_k\) are entangled in \(\mathcal {D}_{\mathcal {L}}\) iff there exists a monotone sequence \((\alpha _j)_{j\in \mathbb {N}}\) such that, for each \(i=1, \dots , k\), we have \(\delta (s,\alpha _j)=u_i\) for infinitely many j’s. Moreover, since \(\mathcal {D}_{\mathcal {L}}\) is minimum, if \(i\ne i^{\prime }\) a word arriving in \(u_i\) and a word arriving in \(u_{i^{\prime }}\) belong to different \(\equiv _{\mathcal {L}}\)-classes. Hence, \(\text{ent}(\mathcal {D}_{\mathcal {L}})\gt p\) iff there exists a monotone sequence in \(\text{Pref} (\mathcal {L})\) which eventually reaches more than p classes of the Myhill-Nerode equivalence \(\equiv _{\mathcal {L}}\) infinitely often, and the corollary follows from the previous theorem.\(□\)

By Theorem 4.21 we have \(\text{width}^D(\mathcal {L}) =\text{ent}(\mathcal {D}_{\mathcal {L}})\). We begin by proving that, if the stated conditions hold true, then \(\text{ent}(\mathcal {D}_{\mathcal {L}}) \ge k\). Notice that for every integer i we have \(\mu _j \gamma ^i \in I_{u_j }\). Moreover, the \(\mu _j\)’s are pairwise distinct because the \(u_j\)’s are pairwise distinct, so without loss of generality we can assume \(\mu _1 \prec \dots \prec \mu _k\).

In both cases the sequence witnesses that \(\lbrace u_1, \dots , u_k \rbrace\) is an entangled set of distinct states, so \(\text{ent}(\mathcal {D}_{\mathcal {L}}) \ge k\).

Conversely, assume that \(\text{ent}(\mathcal {D}_{\mathcal {L}}) \ge k\). This means that there exist distinct states \(v_1, \dots , v_k\) and a monotone sequence \((\alpha _i)_{i \in \mathbb {N}}\) that reaches each of the \(v_j\)’s infinitely many times. Let us show that, up to taking subsequences, we can assume not only that \((\alpha _i)_{i \in \mathbb {N}}\) reaches each of the \(v_j\)’s infinitely many times, but it also satisfies additional properties.

Consider the k-tuples \((x^1_s, \dots , x^k_s)\), for \(s \in \lbrace 0, \dots , m\rbrace\) (corresponding to the states in column in Figure 6). There are \(m + 1 = |Q|^k + 1\) such k-tuples and therefore two of them must be equal. That is, there exist \(h, \ell\), with \(0 \le h \lt \ell \le m\), such that \((x^1_h, \dots , x^k_h)=(x^1_\ell , \dots , x^k_\ell)\). Hence, for all \(j \in \lbrace 1, \dots , k\rbrace\) there is a cycle \(x^j_h, x^j_{h+1}, \dots , x^j_\ell\), all these cycles are labeled by the same string \(\gamma ^{\prime }\), and we can write \(\theta = \phi \gamma ^{\prime } \psi\) for some \(\phi\) and \(\psi\).

Fig. 6.

View Figure

Let \(u_1, u_2, \dots , u_k\) be the pairwise distinct states \(x^1_h, x^2_h, \dots , x^k_h\) (they are distinct, because if \(u_i=u_j\) for some \(i\ne j\) we would have \(v_i=v_j\)). Hence, we have k pairwise distinct states and k equally labeled cycles. In order to fulfill the remaining conditions of the theorem, we proceed as follows. Considering the monotone sequence \((\alpha _i^{\prime }\phi)_{i \in \mathbb {N}}\), reaching each of the \(u_i\)’s infinitely many times, we may suppose without loss of generality (possibly eliminating a finite number of initial elements) that all \(\alpha _i^{\prime }\phi\)’s are co-lexicographically larger than \(\gamma ^{\prime }\) or they are all co-lexicographically smaller than \(\gamma ^{\prime }\).

If \(\gamma ^{\prime }\) is not a suffix of any \(\alpha _i^{\prime }\phi\) we can choose \(\gamma =\gamma ^{\prime }\) and, considering k words \(\mu _1, \dots , \mu _k\) of the sequence \((\alpha _i^{\prime }\phi)_{i \in \mathbb {N}}\) arriving in \(u_1, \dots , u_k\), respectively, we are done.

Otherwise, if \(\gamma ^{\prime }\) is a suffix of some \(\alpha _i^{\prime }\phi\), pick \(2k - 1\) strings \(\delta _1, \dots , \delta _{2k - 1}\) in the sequence \((\alpha _i^{\prime }\phi)_{i \in \mathbb {N}}\) such that \(\delta _k\) ends in \(u_k\), while \(\delta _i\) and \(\delta _{k + i}\) end in \(u_i\) for \(i = 1, \dots , k - 1\), and \(\begin{equation*} \delta _1 \prec \dots \prec \delta _k \prec \dots \prec \delta _{2k - 1}. \end{equation*}\)

Let r be an integer such that \(|(\gamma ^{\prime })^r| \gt |\delta _i|\) for every \(i = 1, \dots , 2k - 1\). Then \(\gamma = (\gamma ^{\prime })^r\) is the label of a cycle from \(u_i\), for every \(i = 1, \dots , k\), and \(\gamma\) is not a suffix of \(\delta _i\), for every \(i = 1, \dots , 2k - 1\). We distinguish two cases:

In both cases, we have either \(\mu _1 , \dots , \mu _k \prec \gamma\) or \(\gamma \prec \mu _1 , \dots , \mu _k\) and the conclusion follows.\(□\)

We will prove the lemma for \(h = 3\) (the extension to the general case is straightforward). Given \(\varphi \in \Sigma ^*\), we denote by \(\varphi (k)\) the k-th letter of \(\varphi\) from the right (if \(|\varphi | \lt k\) we write \(\varphi (k) = \varepsilon\), where \(\varepsilon\) is the empty string); therefore, if \(\varphi \ne \varepsilon\), then \(\varphi (1)\) is the last letter of \(\varphi\).

Let \(\nu _1\preceq \nu _2\preceq \nu _3\) be strings with \(\delta (s_i,\nu _i)=q_i\), for \(i=1,2,3\). Let \(d_{3,2}\) be the first position from the right where \(\nu _3\) and \(\nu _2\) differ (if \(\nu _3=\nu _2\), let \(d_{3,2}=|\nu _3|\)). Since \(\nu _2 \preceq \nu _3\), we have \(d_{3,2}\le |\nu _3|\). Defining \(d_{2,1}\) similarly, we have \(d_{2,1}\le |\nu _2|\).

We distinguish three cases.

Fig. 7.

View Figure

Case 3 is analogous to case 2 and will not be considered. In cases 1 and 2, \(|\nu _3|\ge d_{3,2}\) and \(|\nu _2| \ge d_{2,1}\ge d_{3,2}\), so that \(\nu _3, \nu _2\), and \(\nu _1\) end with the same (possibly empty) word \(\xi\) with \(|\xi |= d_{3,2}-1\). Summing up: \(\begin{equation*} \nu _1=\theta _1 \nu _1(d_{3,2}) \xi \preceq \nu _2=\theta _2 \nu _2(d_{3,2}) \xi \preceq \nu _3=\theta _3 \nu _3(d_{3,2})\xi \end{equation*}\) for some (possibly empty) strings \(\theta _1, \theta _2, \theta _3\).

Without loss of generality, we may assume that \(|\xi | \le | Q|^3\). Indeed, if \(|\xi |\gt | Q|^3\) then when we consider the triples of states visited while reading the last \(| \xi |\) letters in computations from \(s_i\) to \(q_i\) following \(\nu _i\), for \(i=1,2,3\), we should have met a repetition. If this were the case, we could erase a common factor from \(\xi\), obtaining a shorter word \(\xi _1\) such that \(\theta _1 \nu _1(d_{3,2})\xi _1\preceq \theta _2 \nu _2(d_{3,2}) \xi _1\preceq \theta _3\nu _3(d_{3,2}) \xi _1\), with the three new strings starting in \(s_1,s_2,s_3\) and ending in \(q_1,q_2,q_3\), respectively, respecting equalities and suffixes. Then we can repeat the argument until we reach a word not longer than \(|Q|^3\).

If \(d_{3,2}=d_{2,1}\), the order between the \(\nu _i^{\prime }s\) is settled in position \(d_{3,2}\). Let \(r_1, r_2, r_3\) be the states reached from \(s_1,s_2,s_3\) by reading \(\theta _1, \theta _2, \theta _3\), respectively. For \(i=1,2,3\), let \(\bar{\theta }_i\) be the label of a simple path from \(s_i\) to \(r_i\) and let \(\nu _i^{\prime }=\bar{\theta }_i \nu _i(d_{3,2})\xi\). Then \(\delta (s_i,\nu _i^{\prime })= q_i\), \(\nu _1^{\prime }\preceq \nu _2^{\prime }\preceq \nu _3^{\prime }\), \(|\nu _1^{\prime }|, |\nu _2^{\prime }|, |\nu _3^{\prime }|\le | Q| +| Q|^3\).

If \(d_{3,2}\lt d_{2,1}\) we have \(\nu _1(d_{3, 2}) = \nu _2 (d_{3, 2})\). Moreover, \(\theta _1\) and \(\theta _2\) end with the same word \(\xi ^{\prime }\) with \(|\xi ^{\prime }|= d_{2,1}-d_{3,2}-1\) (see Figure 7(a)), and we can write \(\begin{equation*} \theta _1=\theta _1^{\prime } \nu _1(d_{2,1})\xi ^{\prime } \preceq \theta _2=\theta ^{\prime }_2\nu _2(d_{2,1})\xi ^{\prime }. \end{equation*}\) Arguing as before we can assume, without loss of generality, that \(|\xi ^{\prime }| \le | Q|^2\). Moreover, we have \(\nu _1 (d_{2, 1}) \preceq \nu _2 (d_{2, 1})\) and, as before, we can assume that \(\theta ^{\prime }_1, \theta ^{\prime }_2\) and \(\theta _3\) label simple paths. Therefore, \(|\theta ^{\prime }_{1}|, |\theta ^{\prime }_{2}|, |\theta _{3} |\le | Q| - 1\). Hence, in this case we can find \(\nu ^{\prime }_1, \nu ^{\prime }_2, \nu ^{\prime }_3\) such that \(\delta (s_i,\nu _i^{\prime })= q_i\), \(\nu _1^{\prime }\preceq \nu _2^{\prime }\preceq \nu _3^{\prime }\), and \(|\nu _1^{\prime }|, |\nu _2^{\prime }|, |\nu _3^{\prime }|\le 1 + | Q| +| {Q}|^2 + | Q|^3\).

Finally, the construction implies that \(\nu _i= \nu _j\) iff \(\nu _i^{\prime } =\nu _j^{\prime }\), and \(\nu _i \dashv \nu _j\) iff \(\nu _i^{\prime } \dashv \nu _j^{\prime }\).\(□\)

\((\Leftarrow)\) Since condition 4. implies that \(\gamma\) is not a suffix of any of the \(\mu _j\), \(\text{width}^D(\mathcal {L}) \ge k\) follows from Theorem 4.23.

\((\Rightarrow)\) If \(\text{width}^D(\mathcal {L}) \ge k\), we use Theorem 4.23 and find words \(\mu _1^{\prime }, \dots , \mu _k^{\prime }, \gamma ^{\prime }\) and pairwise distinct states \(u_1, \dots , u_k \in Q\) such that for every \(j = 1, \dots , k\):

We only consider the case \(\gamma ^{\prime } \prec \mu _1^{\prime }, \dots , \mu _k^{\prime }\), since in the case \(\mu _1^{\prime }, \dots , \mu _k^{\prime }\prec \gamma ^{\prime }\) the proof is similar. Up to an index permutation, we may suppose without loss of generality that \(\gamma ^{\prime } \prec \mu _1^{\prime }\prec \dots \prec \mu _k^{\prime }\). Consider the \(2k\)-words \(\nu _i\) and states \(s_i,q_i\) defined, for \(i=1, \dots 2k\), as follows:

If we apply Lemma 4.25 to these \(2k\)-words, we obtain words \(\nu _1^{\prime }= \dots = \nu _k^{\prime } \prec \nu _{k+1}^{\prime }\prec \dots \prec \nu _{2k}^{\prime }\) such that, for all \(i=1,\dots ,k\):

Let r be the smallest integer such that \(|(\nu _1^{\prime })^r|\gt max\lbrace |\nu _{k+i}^{\prime } | \, |\, i \in \lbrace 1,\dots , k\rbrace \rbrace\). Let \(\mu _1=\nu _{k+1}^{\prime }, \dots , \mu _k=\nu _{2k}^{\prime } , \gamma = (\nu _1^{\prime })^r\). Since \(\nu _1^{\prime }\) is not a suffix of \(\mu _{i}\), for all \(i=1,\dots , k\), from \(\nu _1^{\prime } \prec \mu _{ 1}\prec \dots \prec \mu _{ k}\) it follows \(\gamma =(\nu _1^{\prime })^r \prec \mu _{ 1}\prec \dots \prec \mu _{ k}\), Moreover: \(\begin{equation*} |\mu _1|, \dots , |\mu _k| \lt |\gamma |\le max\lbrace |\nu _{k+i}^{\prime }| \, |\, i \in \lbrace 1,\dots , k\rbrace \rbrace +|\nu _1^{\prime }|\le 2\left(2k - 2 + \sum _{t = 1}^{2k } |Q|^t\right) \end{equation*}\) and the conclusion follows.\(□\)

We exhibit a dynamic programming algorithm based on Corollary 4.26, plugging in the value \(k = p+1\) and returning true if and only if \(\text{width}^D(\mathcal {L}) \ge k\) is false.

First, note that the alphabet’s size is never larger than the number of transitions: \(\sigma \le |\delta |\), and that \(|Q| \le |\delta |+1\) since we assume that each state can be reached from s. Up to minimizing \(\mathcal {D}\) (with Hopcroft’s algorithm, running in time \(O(|Q|\sigma \log |Q|) \le |\delta |^{O(1)}\)) we can assume that \(\mathcal {D}=\mathcal {D}_{\mathcal {L}}\) is the minimum DFA recognizing \(\mathcal {L}\). Let \(N^{\prime }= 2(2k - 2 + \sum _{t = 1}^{2k } |Q|^t)\) be the upper bound to the lengths of the strings \(\mu _i\) (\(1\le i \le k\)) and \(\gamma\) that need to be considered, and let \(N = N^{\prime }+1\) be the number of states in a path labeled by a string of length \(N^{\prime }\). Asymptotically, note that \(N \le |Q|^{O(k)} \le |\delta |^{O(k)}\). The high-level idea of the algorithm is as follows. First, in condition (3) of Corollary 4.26, we focus on finding paths \(\mu _j\)’s smaller than \(\gamma\), as the other case (all \(\mu _j\)’s larger than \(\gamma\)) can be solved with a symmetric strategy. Then:

Steps (1) and (2) could be naively solved by enumerating the strings \(\mu _1, \dots , \mu _k\), and \(\gamma\) and trying all possible combinations of states \(u_1, \dots , u_k\). Because of the string enumeration step, however, this strategy would be exponential in N, i.e., doubly-exponential in k. We show that a dynamic programming strategy is exponentially faster.

Step (1). This construction is identical to the one used in [3] for the Wheeler case \((p = 1)\). For completeness, we report it here. Let \(\pi _{u,\ell }\), with \(u\in Q\) and \(2 \le \ell \le N\), denote the predecessor of u such that the co-lexicographically smallest path of length (number of states) \(\ell\) connecting the source s to u passes through \(\pi _{u,\ell }\) as follows: \(s \rightsquigarrow \pi _{u,\ell } \rightarrow u\). The node \(\pi _{u,\ell }\) coincides with s if \(\ell =2\) and u is a successor of s; in this case, the path is simply \(s \rightarrow u\). If there is no path of length \(\ell\) connecting s with u, then we write \(\pi _{u,\ell } = \bot\). We show that the set \(\lbrace \pi _{u,\ell }\ :\ 2\le \ell \le N,\ u\in Q\rbrace\) stores in just polynomial space all co-lexicographically smallest paths of any fixed length \(2 \le \ell \le N\) from the source to any node u. We denote such a path — to be intended as a sequence \(u_1 \rightarrow \dots \rightarrow u_\ell\) of states — with \(\alpha _\ell (u)\). The node sequence \(\alpha _\ell (u)\) can be obtained recursively (in \(O(\ell)\) steps) as \(\alpha _\ell (u) = \alpha _{\ell -1}(\pi _{u,\ell }) \rightarrow u\), where \(\alpha _{1}(s) = s\) by convention. Note also that \(\alpha _\ell (u)\) does not fully specify the sequence of edges (and thus labels) connecting those \(\ell\) states, since two states may be connected by multiple (differently labeled) edges. However, the corresponding co-lexicographically smallest sequence \(\lambda ^-(\alpha _\ell (u))\) of \(\ell -1\) labels is uniquely defined as follows: \(\[ \left\lbrace \begin{array}{ll} \lambda ^-(\alpha _\ell (u)) = \mathrm{min}\lbrace a\in \Sigma \ |\ \delta (s,a)=u\rbrace & \mathrm{if}\ \ell =2 ,\\ \lambda ^-(\alpha _\ell (u)) = \lambda ^-(\alpha _{\ell -1}(\pi _{u,\ell }) \rightarrow u) = \lambda ^-(\alpha _{\ell -1}(\pi _{u,\ell })) \cdot \mathrm{min}\lbrace a\in \Sigma \ |\ \delta (\pi _{u,\ell },a) = u\rbrace & \mathrm{if}\ \ell \gt 2. \end{array}\right. \]\)

It is not hard to see that each \(\pi _{u,\ell }\) can be computed in \(|\delta |^{O(k)}\) time using dynamic programming. First, we set \(\pi _{u,2} = s\) for all successors u of s. Then, for \(\ell = 3, \dots , N\): \(\[ \pi _{u,\ell } = \underset{v\in \text{Pred}(u)}{\mathrm{argmin}} \Big (\lambda ^-(\alpha _{\ell -1}(v)) \cdot \mathrm{min}\lbrace a\in \Sigma \ |\ \delta (v,a)=u\rbrace \Big), \]\)

where \(\text{Pred}(u)\) is the set of all predecessors of u and the \(\mathrm{argmin}\) operator compares strings in co-lex order. In the equation above, if none of the \(\alpha _{\ell -1}(v)\) are well-defined (because there is no path of length \(\ell -1\) from s to v), then \(\pi _{u,\ell } = \bot\). Note that computing any particular \(\pi _{u,\ell }\) requires comparing co-lexicographically \(|\mathrm{Pred(u)}| \le |Q|\) strings of length at most \(\ell \le N \le |\delta |^{O(k)}\), which overall amounts to \(|\delta |^{O(k)}\) time. Since there are \(|Q|\times N = |\delta |^{O(k)}\) variables \(\pi _{u,\ell }\) and each can be computed in time \(|\delta |^{O(k)}\), overall Step (1) takes time \(|\delta |^{O(k)}\). This completes the description of Step (1).

Step (2). Fix a k-tuple \(u_1,\dots , u_k\) and a length \(2\le \ell \le N\). Our goal is now to show how to compute the co-lexicographically largest string \(\gamma\) of length \(\ell -1\) labeling k cycles of length (number of states) \(\ell\) originating (respectively, ending) from (respectively, in) all the states \(u_1,\dots , u_k\). Our final strategy will iterate over all such k-tuples of states (in time exponential in k) in order to find one satisfying the conditions of Corollary 4.26.

Our goal can again be solved by dynamic programming. Let \(u_1,\dots , u_k\) and \(u^{\prime }_1,\dots , u^{\prime }_k\) be two k-tuples of states, and let \(2\le \ell \le N\). Let moreover \(\pi _{u_1,\dots , u_k, u^{\prime }_1,\dots , u^{\prime }_k, \ell }\) be the k-tuple \(\langle u^{\prime \prime }_1,\dots , u^{\prime \prime }_k \rangle\) of states (if it exists) such that there exists a string \(\gamma\) of length \(\ell -1\) with the following properties:

If such a string \(\gamma\) does not exist, then we set \(\pi _{u_1,\dots , u_k, u^{\prime }_1,\dots , u^{\prime }_k, \ell } = \bot\).

Remember that we fix \(u_1,\dots , u_k\). For \(\ell = 2\) and each k-tuple \(u^{\prime }_1,\dots , u^{\prime }_k\), it is easy to compute \(\pi _{u_1,\dots , u_k, u^{\prime }_1,\dots , u^{\prime }_k, \ell }\): this k-tuple is \(\langle u_1,\dots , u_k \rangle\) (all paths have length 2) if and only if there exists \(c\in \Sigma\) such that \(u^{\prime }_i = \delta (u_i,c)\) for all \(1\le i\le k\) (otherwise it does not exist). Then, \(\gamma\) is formed by one character: the largest such c.

For \(\ell \gt 2\), the k-tuple \(\pi _{u_1,\dots , u_k, u^{\prime }_1,\dots , u^{\prime }_k, \ell }\) can be computed as follows. Assume we have computed those variables for all lengths \(\ell ^{\prime }\lt \ell\). Note that for each such \(\ell ^{\prime }\lt \ell\) and k-tuple \(u^{\prime \prime }_1, \dots , u^{\prime \prime }_k\), the variables \(\pi _{u_1,\dots , u_k, u^{\prime \prime }_1,\dots , u^{\prime \prime }_k, \ell ^{\prime }}\) identify k paths \(u_i \rightsquigarrow u^{\prime \prime }_i\) of length (number of nodes) \(\ell ^{\prime }\). Let us denote with \(\alpha _{\ell ^{\prime }}(u^{\prime \prime }_i)\) such paths, for \(1\le i\le k\).

Then, \(\pi _{u_1,\dots , u_k, u^{\prime }_1,\dots , u^{\prime }_k, \ell }\) is equal to \(\langle u^{\prime \prime }_1,\dots , u^{\prime \prime }_k \rangle\) maximizing co-lexicographically the string \(\gamma ^{\prime }\cdot c\) defined as follows:

If no \(c\in \Sigma\) satisfies condition (1), or condition (2) cannot be met, then \(\pi _{u_1,\dots , u_k, u^{\prime }_1,\dots , u^{\prime }_k, \ell } = \bot\).

Note that \(\pi _{u_1,\dots , u_k, u_1,\dots , u_k, \ell }\) allows us to identify (if it exists) the largest string \(\gamma\) of length \(\ell -1\) labeling k cycles originating and ending in each \(u_i\), for \(1\le i \le k\).

Each tuple \(\pi _{u_1,\dots , u_k, u^{\prime }_1,\dots , u^{\prime }_k, \ell }\) can be computed in \(|\delta |^{O(k)}\) time by dynamic programming (in order of increasing \(\ell\)), and there are \(|\delta |^{O(k)}\) such tuples to be computed (there are \(|Q|^{O(k)} \le |\delta |^{O(k)}\) ways of choosing \(u_1,\dots , u_k, u^{\prime }_1,\dots , u^{\prime }_k\), and \(N \le |\delta |^{O(k)}\)). Overall, also Step (2) can therefore be solved in \(|\delta |^{O(k)}\) time.

To sum up, we can check if the conditions of Corollary 4.26 hold as follows:

The conditions of Corollary 4.26 hold if and only if step 3 above succeeds for at least one k-tuple \(u_1, \dots , u_k\) and lengths \(\ell _i \lt \ell \le N\), for \(1\le i\le k\). Overall, the algorithm terminates in \(|\delta |^{O(k)} = |\delta |^{O(p)}\) time.\(□\)

By definition \((\sim ^{cv})^r\) is a right-invariant refinement. Moreover, \(\sim ^{cv}\) and \((\sim ^{cv})^r\) are \(\mathcal {P}\)-consistent because they are refinements of the \(\mathcal {P}\)-consistent equivalence relation \(\sim\). Let us prove that \((\sim ^{cv})^r\) is \(\mathcal {P}\)-convex. Assume that \(\alpha , \beta , \gamma \in \text{Pref} (\mathcal {L})\) are such that \(\alpha (\sim ^{cv})^r \gamma\), \(\alpha \prec \beta \prec \gamma\) and \(U_\alpha = U_\beta\). Being \((\sim ^{cv})^r\) a \(\mathcal {P}\)-consistent relation, we have \(U_\alpha = U_\beta =U_\gamma\). We must prove that \(\alpha (\sim ^{cv})^r \beta\). Fix \(\phi \in \Sigma ^*\) such that \(\alpha \phi \in \text{Pref} (\mathcal {L})\). We must prove that \(\alpha \phi \sim ^{cv} \beta \phi\). Now, \(\alpha (\sim ^{cv})^r \gamma\) implies \(\alpha \sim ^{cv} \gamma\). Since \(\alpha \prec \beta \prec \gamma\) and \(U_\alpha = U_\beta = U_\gamma\), then the \(\mathcal {P}\)-convexity of \(\sim ^{cv}\) implies \(\alpha \sim ^{cv} \beta\). In particular, \(\alpha \sim \beta\). Since \(\sim\) is right-invariant we have \(\alpha \phi \sim \beta \phi\), and from the \(\mathcal {P}\)-consistency of \(\sim\) we obtain \(U_{\alpha \phi } = U_{\beta \phi }\). Moreover, \(\alpha (\sim ^{cv})^r \gamma\) implies \(\alpha \phi (\sim ^{cv})^r \gamma \phi\) by right-invariance, so \(\alpha \phi \sim ^{cv} \gamma \phi\). By \(\mathcal {P}\)-convexity, from \(\alpha \phi \sim ^{cv} \gamma \phi\), \(U_{\alpha \phi } = U_{\beta \phi }\) and \(\alpha \phi \prec \beta \phi \prec \gamma \phi\) (since \(\alpha \prec \beta \prec \gamma)\) we conclude \(\alpha \phi \sim ^{cv} \beta \phi\).\(□\)

The equivalence relation \((\sim ^{cs})^r\) is \(\mathcal {P}\)-consistent (because it is a refinement of the \(\mathcal {P}\)-consistent equivalence relation \(\sim ^{cs}\)) and right-invariant (by definition it is a right-invariant refinement), so by Lemma 4.32 the equivalence relation \((((\sim ^{cs})^r)^{cv})^r\) is \(\mathcal {P}\)-consistent, \(\mathcal {P}\)-convex and right-invariant. Moreover, every \(\mathcal {P}\)-consistent, \(\mathcal {P}\)-convex and right-invariant equivalence relation refining \(\sim\) must also refine \((((\sim ^{cs})^r)^{cv})^r\), so \((((\sim ^{cs})^r)^{cv})^r\) is the coarsest \(\mathcal {P}\)-consistent, \(\mathcal {P}\)-convex and right-invariant equivalence relation refining \(\sim\).\(□\)

The relation \(\equiv _\mathcal {D}\) has index equal to \(|Q|\). It respects \(\text{Pref}(\mathcal {L})\) because if \(\alpha \equiv _\mathcal {D}\beta\) and \(\phi \in \Sigma ^*\) satisfies \(\alpha \phi \in \text{Pref}(\mathcal {L})\), then there exists \(\gamma\) with \(\alpha \phi \gamma \in \mathcal {L}\) so \(\delta (s, \alpha \phi \gamma)\in F\). Since \(\delta (s, \alpha)=\delta (s, \beta)\) we obtain \(\delta (s, \alpha \phi \gamma)=\delta (s, \beta \phi \gamma)\) and so \(\beta \phi \gamma \in \mathcal {L}\) and \(\beta \phi \in \text{Pref}(\mathcal {L})\) follows. Moreover, it is right-invariant because if \(\alpha \equiv _\mathcal {D}\beta\) and \(\phi \in \Sigma ^*\) is such that \(\alpha \phi \gamma \in \mathcal {L}\), then \(\beta \phi \in \text{Pref}(\mathcal {L})\) and from \(\delta (s, \alpha)=\delta (s, \beta)\) we obtain \(\delta (s, \alpha \phi)=\delta (s, \beta \phi)\).

For every \(\alpha \in \text{Pref} (\mathcal {L})\) we have \([\alpha ]_{\equiv _\mathcal {D}} = I_{\delta (s, \alpha)}\), which implies that \(\equiv _\mathcal {D}\) is \(\mathcal {P}\)-consistent. Moreover, \(\equiv _\mathcal {D}\) is \(\mathcal {P}\)-convex, that is, for every \(\alpha \in \text{Pref}(\mathcal {L})\) we have that \([\alpha ]_{\equiv _\mathcal {D}} = I_{\delta (s, \alpha)}\) is convex in \(U_\alpha\), because if \(u_1, \dots , u_k \in Q\) are such that \(U_\alpha = \bigcup _{i = 1}^k I_{u_i}\), then the \(u_i\)’s must be pairwise \(\le _\mathcal {D}\)-comparable, being in the same \(\le _\mathcal {D}\)-chain. Moreover, \(\equiv _\mathcal {D}\) refines \(\equiv _\mathcal {L}\), because \(\alpha \equiv _\mathcal {D} \beta\) implies that for every \(\phi \in \Sigma ^*\) we have \(\delta (s, \alpha \phi) = \delta (s, \beta \phi)\) and so \(\alpha \phi \in \mathcal {L}\) iff \(\beta \phi \in \mathcal {L}\). Since \(\equiv _\mathcal {L}^\mathcal {P}\) is the coarsest \(\mathcal {P}\)-consistent, \(\mathcal {P}\)-convex and right-invariant equivalence relation refining \(\equiv _\mathcal {L}\), and \(\equiv _\mathcal {D}\) is a \(\mathcal {P}\)-consistent, \(\mathcal {P}\)-convex and right-invariant equivalence relation refining \(\equiv _\mathcal {L}\), we conclude that \(\equiv _\mathcal {D}\) also refines \(\equiv _\mathcal {L}^\mathcal {P}\), which in particular implies that \(\mathcal {L}\) is the union of some \(\equiv _\mathcal {D}\)-classes. We know that \(\equiv _\mathcal {D}\) has finite index, so \(\equiv _\mathcal {L}^\mathcal {P}\) has finite index.\(□\)

Define the DFA \(\mathcal {D_\sim } = (Q_\sim , s_\sim , \delta _\sim , F_\sim)\) as follows.

Since \(\sim\) is right-invariant, it has finite index and \(\mathcal {L}\) is the union of some \(\sim\)-classes, then \(\mathcal {D_\sim }\) is a well-defined DFA and: (6) \(\begin{equation} \alpha \in [\beta ]_\sim \iff \delta _\sim (s_\sim , \alpha) = [\beta ]_\sim . \end{equation}\) which implies that for every \(\alpha \in \text{Pref} (\mathcal {L})\) it holds \(I_{[\alpha ]_\sim } = [\alpha ]_\sim\), and so \(\mathcal {L}(\mathcal {D_\sim }) = \mathcal {L}\).

For every \(i \in \lbrace 1, \dots , p \rbrace\), define: \(\begin{equation*} Q_i = \lbrace [\alpha ]_\sim \; |\; U_\alpha = U_i \rbrace . \end{equation*}\)

Notice that each \(Q_i\) is well-defined because \(\sim\) is \(\mathcal {P}\)-consistent, and each \(Q_i\) is a \(\le _{\mathcal {D_\sim }}\)-chain because \(\sim\) is \(\mathcal {P}\)-convex. It follows that \(\lbrace Q_i \; |\; 1\le i \le p\rbrace\) is a \(\le _{\mathcal {D_\sim }}\)-chain partition of \(Q_\sim\).

From Equation (6) we obtain: \(\begin{equation*} \begin{split} \text{Pref}(\mathcal {L(D_\sim)})^i &= \lbrace \alpha \in \text{Pref}(\mathcal {L(D_\sim)})\, |\, \delta _\sim (s_\sim , \alpha) \in Q_i \rbrace \\ &= \lbrace \alpha \in \text{Pref}(\mathcal {L(D_\sim)})\, |\, (\exists [\beta ]_\sim \in Q_i \, \alpha \in [\beta ]_\sim) \rbrace \\ &= \lbrace \alpha \in \text{Pref}(\mathcal {L(D_\sim)})\, |\, U_\alpha = U_i \rbrace = U_i. \end{split} \end{equation*}\)

In other words, \(\mathcal {D}_\sim\) witnesses that \(\mathcal {L}\) is recognized by a \(\mathcal {P}\)-sortable DFA. Moreover:

Finally, suppose \(\mathcal {B}\) is a \(\mathcal {P}\)-sortable DFA that recognizes \(\mathcal {L}\). Notice that by Lemma 4.35 we have that \(\equiv _{\mathcal {B}}\) is a \(\mathcal {P}\)-consistent, \(\mathcal {P}\)-convex, right-invariant equivalence relation on \(\text{Pref} (\mathcal {L})\) of finite index such that \(\mathcal {L}\) is the union of some \(\equiv _\mathcal {B}\)-classes, so \(\mathcal {D_{\equiv _\mathcal {B}}}\) is well-defined. Call \(Q_\mathcal {B}\) the set of states of \(\mathcal {B}\), and let \(\phi : Q_{\equiv _\mathcal {B}} \rightarrow Q_\mathcal {B}\) be the function sending \([\alpha ]_{\equiv _\mathcal {B}}\) into the state in \(Q_\mathcal {B}\) reached by reading \(\alpha\). Notice that \(\phi\) is well-defined because by the definition of \(\equiv _{\mathcal {B}}\) we obtain that all strings in \([\alpha ]_{\equiv _\mathcal {B}}\) reach the same state of \(\mathcal {B}\). It is easy to check that \(\phi\) determines an isomorphism between \(\mathcal {D_{\equiv _\mathcal {B}}}\) and \(\mathcal {B}\).\(□\)

\((1) \rightarrow (2)\) It follows from Lemma 4.35.

\((2) \rightarrow (3)\) The desired equivalence relation is simply \(\equiv _\mathcal {L}^\mathcal {P}\).

\((3) \rightarrow (1)\) It follows from Lemma 4.36.

Now, let us prove that the minimum DFA is \(\mathcal {D_{\equiv _\mathcal {L}^\mathcal {P}}}\) as defined in Lemma 4.36. First, \(\mathcal {D_{\equiv _\mathcal {L}^\mathcal {P}}}\) is well-defined because \(\equiv _\mathcal {L}^\mathcal {P}\) is \(\mathcal {P}\)-consistent, \(\mathcal {P}\)-convex and right-invariant by definition; moreover, it has finite index and \(\mathcal {L}\) is the union of some \(\equiv _\mathcal {L}^\mathcal {P}\)-equivalence classes by Lemma 4.35. Now, the number of states of \(\mathcal {D_{\equiv _\mathcal {L}^\mathcal {P}}}\) is equal to the index of \(\equiv _\mathcal {L}^\mathcal {P}\), or equivalently, of \(\equiv _{\mathcal {D_{\equiv _\mathcal {L}^\mathcal {P}}}}\). On the other hand, let \(\mathcal {B}\) be any \(\mathcal {P}\)-sortable DFA recognizing \(\mathcal {L}\) non-isomorphic to \(\mathcal {D_{\equiv _\mathcal {L}^\mathcal {P}}}\). Then \(\equiv _\mathcal {B}\) is a refinement of \(\equiv _\mathcal {L}^\mathcal {P}\) by Lemma 4.35, and it must be a strict refinement of \(\equiv _\mathcal {L}^\mathcal {P}\), otherwise \(\mathcal {D_{\equiv _\mathcal {L}^\mathcal {P}}}\) would be equal to \(\mathcal {D_{\equiv _\mathcal {B}}}\), which by Lemma 4.36 is isomorphic to \(\mathcal {B}\), a contradiction. We conclude that the index of \(\equiv _\mathcal {L}^\mathcal {P}\) is smaller than the index of \(\equiv _\mathcal {B}\), so again by Lemma 4.36 the number of states of \(\mathcal {D_{\equiv _\mathcal {L}^\mathcal {P}}}\) is smaller than the number of states of \(\mathcal {D_{\equiv _\mathcal {B}}}\) and so of \(\mathcal {B}\).\(□\)

We proceed by induction on \(| \alpha |\). If \(|\alpha | = 0\), then \(\alpha = \varepsilon ,\) and we are done. Now assume \(|\alpha | \ge 1\). We can write \(\alpha = \alpha ^{\prime } a\), with \(\alpha ^{\prime } \in \Sigma ^*\), \(a \in \Sigma\). Let \(u, v, z \in Q\) such that \(u \lt v \lt z\) and \(u, z \in U^{\prime }\). We must prove that \(v \in U^{\prime }\). By the inductive hypothesis, the set \(U^{\prime \prime }\) of all states in Q that can be reached from some state in U by following edges whose labels, when concatenated, yield \(\alpha ^{\prime }\), is a \(\le\)-convex set. In particular, there exist \(u^{\prime }, z^{\prime } \in U^{\prime \prime }\) such that \(u \in \delta (u^{\prime }, a)\) and \(z \in \delta (z^{\prime }, a)\). Since \(a \in \lambda (u) \cap \lambda (z)\) and \(u \lt v \lt z\), then \(\lambda (v) = \lbrace a\rbrace\) (otherwise by Axiom 1 we would obtain a contradiction), so there exists \(v^{\prime } \in Q\) such that \(v \in \delta (v^{\prime }, a)\). From \(u \lt v \lt z\) and Axiom 2 we obtain \(u^{\prime } \le v^{\prime } \le z^{\prime }\). Since \(u^{\prime }, z^{\prime } \in U^{\prime \prime }\) and \(U^{\prime \prime }\) is a \(\le\)-convex set, then \(v^{\prime } \in U^{\prime \prime }\), and so \(v \in U^{\prime }\).\(□\)

(1) Since V has width p, there exists an antichain A of cardinality p. It is easy to see that any subset \(I \subseteq A\) is a distinct \(\le\)-convex set. The bound \(2^p\) follows. (2) Consider a partial order formed by p mutually-incomparable total orders \(V_i\), all having \(n/p\) elements. Since any total order of cardinality \(n/p\) has \((n/p+1)(n/p)/2 + 1\) distinct convex sets and any combination of \(\le _{V_i}\)-convex sets forms a distinct \(\le\)-convex set, we obtain at least \(\begin{equation*} \prod _{i=1}^p ((n/p+1)(n/p)/2+1) \ge \prod _{i=1}^p n/p = (n/p)^p \end{equation*}\) distinct \(\le\)-convex sets.\(□\)

Let us prove the first statement.

\((\Rightarrow)\) Let \(c \in \Sigma\) such that \(c \in \lambda (u) \setminus \lbrace \#\rbrace\). Let \(u^{\prime } \in Q\) such that \(u \in \delta (u^{\prime }, c)\), and let \(\beta ^{\prime } \in I_{u^{\prime }}\). Then \(\beta ^{\prime } c \in I_u\), so from \(u \in S(\alpha ^{\prime } a)\) we obtain \(\beta ^{\prime } c \prec \alpha ^{\prime } a\), which implies \(c \preceq a\). Now assume that \(c = a\). Suppose for sake of contradiction that \(u^{\prime } \not\in S(\alpha ^{\prime })\). This means that there exists \(\gamma ^{\prime } \in I_{u^{\prime }}\) such that \(\alpha ^{\prime } \preceq \gamma ^{\prime }\). This implies \(\alpha ^{\prime } a \preceq \gamma ^{\prime } a\) and, since \(\gamma ^{\prime } a \in I_u\), we obtain \(u \not\in S(\alpha ^{\prime } a)\), a contradiction.

\((\Leftarrow)\) Let \(\beta \in I_u\). We must prove that \(\beta \prec \alpha ^{\prime } a\). If \(\beta = \varepsilon\) we are done, because \(\varepsilon \prec \alpha ^{\prime } a\). Now assume that \(\beta = \beta ^{\prime } b\). This means that there exists \(u^{\prime } \in Q\) such that \(u \in \delta (u^{\prime }, b)\) and \(\beta ^{\prime } \in I_{u^{\prime }}\). We know that \(b \preceq a\). If \(b \prec a\), then \(\beta \prec \alpha ^{\prime } a\) and we are done. If \(b = a\), then \(u^{\prime } \in S(\alpha ^{\prime })\), so \(\beta ^{\prime } \prec \alpha ^{\prime }\), which implies \(\beta \prec \alpha ^{\prime } a\).

The proof of the second statement is analogous (the only difference being that in \((\Leftarrow)\) it must necessarily be \(\beta \not= \varepsilon\), because \(a \preceq \text{min}_{\lambda (u)}\)).\(□\)

We only prove the first statement, the proof of the second statement being entirely analogous. We can assume \(l_i \lt |Q_i| - 1\), otherwise the conclusion is trivial. If \(a \prec \max (\lambda (Q_i[l_i + 1]))\) the conclusion is immediate by Axiom 1, so we can assume \(\max (\lambda (Q_i[l_i + 1])) \preceq a\). We know that \(Q_i[l_i + 1] \not\in S(\alpha ^{\prime } a)\), so by Lemma 5.11 there exists \(v^{\prime } \in Q\) such that \(Q_i[l_i + 1] \in \delta (v^{\prime }, a)\) and \(v^{\prime } \not\in S(\alpha ^{\prime })\). Suppose for sake of contradiction that \(Q_i[l_i + 1] \lt u\). By Axiom 2 we obtain \(v^{\prime } \le u^{\prime }\). From \(u^{\prime } \in S(\alpha ^{\prime })\) and Lemma 5.9 we conclude \(v^{\prime } \in S(\alpha ^{\prime })\), a contradiction.\(□\)

Again, we just prove the first statement since the proof of the second one is analogous.

Let \(z_i\) be the largest integer \(0 \le k \le |Q_i|\) such that (i) \(\mathtt {in}(Q_i[1,k],a) \le x\), and (ii) if \(k \ge 1\), then \(\max (\lambda (Q_i[k])) \preceq a\). We want to prove that \(l_i = z_i\).

\((\le)\) The conclusion is immediate if \(l_i = 0\), so we can assume \(l_i \ge 1\). It will suffice to prove that \(\mathtt {in}(Q_i[1,l_i],a) \le x\) and \(\max (\lambda (Q_i[l_i])) \preceq a\). This follows from Lemma 5.11 and the definition of x.

\((\ge)\) The conclusion is immediate if \(l_i = |Q_i|\), so we can assume \(l_i \lt |Q_i|\). We only have to prove that if \(l_i + 1 \le k \le |Q_i|\), then either \(\mathtt {in}(Q_i[1,k],a) \gt x\) or \(\max (\lambda (Q_i[k])) \succ a\). By Axiom 1, it will suffice to prove that we have \(\mathtt {in}(Q_i[1,l_i + 1],a) \gt x\) or \(\max (\lambda (Q_i[l_i + 1])) \succ a\). Assume that \(\max (\lambda (Q_i[l_i + 1])) \preceq a\). Since \(Q_i[l_i + 1] \not\in S(\alpha)\), by Lemma 5.11 there exists \(v^{\prime } \in Q\) such that \(Q_i[l_i + 1] \in \delta (v^{\prime }, a)\) and \(v^{\prime } \not\in S(\alpha ^{\prime })\). We will conclude that \(\mathtt {in}(Q_i[1,l_i + 1],a) \gt x\) if we show that for every \(j = 1, \dots , p\), if \(u^{\prime } \in Q_j\) and \(u \in Q_i\) are such that \(u^{\prime } \in Q_j[1, l^{\prime }_j]\) (and so \(u^{\prime } \in S(\alpha ^{\prime })\)) and \(u \in \delta (u^{\prime }, a)\), then it must be \(u \in Q_i[1, l_i + 1]\). This follows from Lemma 5.12.\(□\)

First, let us prove that for every \(k = 1, \dots , n\), we can retrieve the labels — with multiplicities — of all edges entering \(v_k\). By scanning \(\mathtt {CHAIN}\) we can retrieve the integers \(k_1\) and \(k_2\) such that the states in chain \(Q_i\) are \(v_{k_1}, v_{k_1 + 1}, \dots , v_{k_2 - 1}, v_{k_2}\). By scanning \(\mathtt {OUT}\), which stores the label and the destinational chain of each edge, we can retrieve how many edges enter chain \(Q_i\), and we can retrieve the labels - with multiplicities - of all edges entering chain \(Q_i\). By considering the substring of \(\mathtt {IN\_DEG}\) between the \((k_1 - 1)\)-th one and the \(k_2\)-th one we can retrieve the in-degrees of all states in chain \(Q_i\). Since we know the labels - with multiplicities - of all edges entering chain \(Q_i\) and the in-degrees of all states in chain \(Q_i\), by Axiom 1 we can retrieve the labels - with multiplicities - of all edges entering each node in chain \(Q_i\): order the multiset of incoming edge labels, scan the nodes in \(Q_i\) in order, and assign the labels to each node in \(Q_i\) in agreement with their in-degrees.

Let us prove that for every \(i = 1, \dots , p\) we can retrieve the integers \(l_i\) and \(r_i\) such that \(S_i(\alpha) = Q_i[1, l_i]\) and \(L_i(\alpha) = Q_i[r_i,|Q_i|]\). We proceed by induction on \(|\alpha |\). If \(|\alpha | = 0\), then \(\alpha = \varepsilon\), so for every \(i = 1, \dots , p\) we have \(l_i = 0\), for every \(i = 2, \dots , p\) we have \(r_i = 1\), and \(r_1 = 2\). Now, assume \(|\alpha | \gt 0\). We can write \(\alpha = \alpha ^{\prime } a\), with \(\alpha ^{\prime } \in \Sigma ^*\) and \(a \in \Sigma\). By the inductive hypothesis, for \(j = 1, \dots , p\) we know the integers \(l^{\prime }_j\) and \(r^{\prime }_j\) such that \(S_j(\alpha ^{\prime }) = Q_j[1, l^{\prime }_j]\) and \(L_j(\alpha ^{\prime }) = Q_j[r^{\prime }_j,|Q_j|]\). Notice that by using \(\mathtt {OUT\_DEG}\) and \(\mathtt {OUT}\) we can compute \(\mathtt {out}(Q_j[1,l^{\prime }_j], i, a)\) for every \(j = 1, \dots , p\) (see Definition 5.13). Since we know the labels - with multiplicities - of all edges entering each state, we can also compute \(\mathtt {in}(Q_i[c, d],a)\) and \(\lambda (Q_i[k]))\) for every \(i = 1, \dots , p\), \(1 \le c \le d \le |Q_i|\), and \(1 \le k \le |Q_i|\). By Lemma 5.14 we conclude that we can compute \(l_i\) and \(r_i\) for every \(i = 1, \dots , p\), and we are done.

Now, let us prove that for every \(\alpha \in \Sigma ^*\) we can retrieve the set \(\lbrace i \in \lbrace 1, \dots , n\rbrace \; | \; \alpha \in I_{v_i} \rbrace\). We proceed by induction on \(|\alpha |\). If \(|\alpha | = 0\), then \(\alpha = \varepsilon\) and \(\lbrace i \in \lbrace 1, \dots , n\rbrace \; | \; \varepsilon \in I_{v_i} \rbrace = \lbrace 1 \rbrace\). Now, assume \(|\alpha | \gt 0\). We can write \(\alpha = \alpha ^{\prime } a\), with \(\alpha ^{\prime } \in \Sigma ^*\) and \(a \in \Sigma\). By the inductive hypothesis, we know \(\lbrace i \in \lbrace 1, \dots , n\rbrace \; | \; \alpha ^{\prime } \in I_{v_i} \rbrace\). For every \(i = 1, \dots , p\) we decide whether \(I^i_\alpha \not= \emptyset\) by using \(\lbrace i \in \lbrace 1, \dots , n\rbrace \; | \; \alpha ^{\prime } \in I_{v_i} \rbrace\), \(\mathtt {OUT\_DEG}\) and \(\mathtt {OUT}\). If \(I_\alpha ^i \not= \emptyset\), then by Lemma 5.9 we know that \(I_\alpha ^i = Q_i [l_i + 1, r_i - 1]\), and we know how to determine \(l_i\) and \(r_i\). Hence, we can easily compute \(\lbrace i\in \lbrace 1, \dots , n\rbrace \; | \; \alpha \in I_{v_i} \rbrace\).\(□\)

Since \(\mathcal {N}\) and \(\mathcal {N}^{\prime }\) share the sequence \(\mathtt {CHAIN}\), it must be \(p = p^{\prime }\) and \(|Q_i| = |Q^{\prime }_i |\) for every i. Fix a string \(\alpha \in \Sigma ^*\). Then, by Lemma 5.15 we conclude that the set \(\lbrace i \in \lbrace 1, \dots , n\rbrace \; | \; \alpha \in I_{v_i} \rbrace\) is the same for both \(\mathcal {N}\) and \(\mathcal {N}^{\prime }\). Since \(\mathcal {N}\) and \(\mathcal {N}^{\prime }\) share also the sequence \(\mathtt {FINAL}\), we conclude that \(\alpha\) is accepted by \(\mathcal {N}\) if and only if it is accepted by \(\mathcal {N}^{\prime }\).\(□\)

Let \(v_1, \dots , v_n\) be the ordering of Q in Definition 5.6. We know that \(v_1\) is the initial state and for every \(j = 1, \dots , n\) we can decide whether \(v_j\) is final by using \(\mathtt {FINAL}\). Now, for every \(\alpha \in \Sigma ^*\), let \(C_\alpha = \lbrace i \in \lbrace 1, \dots , n\rbrace \; | \; \alpha \in I_{v_i} \rbrace\). Notice that we can compute \(C_\alpha\) for every \(\alpha \in \Sigma ^*\) by Lemma 5.15. Consider a list that contains pairs of the form \((\alpha , C_\alpha)\). Initially, the list contains only \((\varepsilon , \lbrace 1\rbrace)\). Remove recursively an element \((\alpha , C_\alpha)\) and for every \(a \in \Sigma\) add \((\alpha a, C_{\alpha a})\) to the list if and only if \(C_{\alpha a}\) is nonempty and it is not the second element of a pair which is or has already been in the list. This implies that after at most \(|\Sigma | \cdot 2^{n}\) steps the list is empty, and any non-empty \(C_\alpha\) has been the second element of some pair in the list. Then, we conclude that \(\mathcal {N}\) is distinguished by its paths if and only for every \(k = 1, \dots , n\) the set \(\lbrace v_k \rbrace\) has been the second element of some pair in the list. In particular, if \(\mathcal {N}\) is distinguished by its paths, then for every \(k = 1, \dots , n\) we know a string \(\alpha ^{\prime } \in \Sigma ^*\) such that \(C_{\alpha ^{\prime }} = \lbrace k\rbrace\). We are only left with showing that we can use \(\alpha ^{\prime }\) to retrieve all edges leaving \(v_k\). Fix a character \(a \in \Sigma\). Then, compute \(C_{\alpha ^{\prime } a}\) using again Lemma 5.15. Then, \(v_k\) has \(|C_{\alpha ^{\prime } a}|\) outgoing edges labeled a, whose indexes are given by \(C_{\alpha ^{\prime } a}\).\(□\)

The bound of \(\log (p\sigma) + O(1)\) bits per transition follows directly from Definition 5.6 and Theorem 5.19. Letting \(|\delta |\) denote the number of transitions and n denote the number of states, on DFAs the naive bound \(|\delta | \le n\sigma\) holds; this allows us to derive the bound of \(\sigma \log (p\sigma) + O(\sigma)\) bits per state on DFAs. On arbitrary NFAs, we can use the bound \(|\delta | \le 2p\sigma n\) implied by Lemma 2.6, yielding the bound of \(2p\sigma \log (p\sigma) + O(p\sigma)\) bits per state on NFAs.\(□\)

Since \(S(\alpha) \subseteq R(\alpha)\), we have \(d \ge c\). Now, notice that \(T_i (\alpha) \not= \emptyset\) if and only there exists an edge labeled a leaving a state in \(T (\alpha ^{\prime })\) and reaching chain \(Q_i\), if and only if \(d \gt c\). Hence, in the following we can assume \(T_i (\alpha) \not= \emptyset\). In particular, this implies \(l_i \lt |Q_i|\) and \(t_i \ge l_i + 1\). By Lemma 5.12 all edges labeled a, leaving a state in \(S(\alpha ^{\prime })\), and reaching chain \(Q_i\) must end in \(Q_i[1, l_i + 1]\). At the same time, since \(T_i (\alpha) \not= \emptyset\), the definition of \(t_i\) implies that there exists \(v^{\prime } \in T(\alpha ^{\prime })\) (and so \(v^{\prime } \in R(\alpha ^{\prime })\)) such that \(Q_i[t_i] \in \delta (v^{\prime }, a)\). Hence, the conclusion will follow if we prove that if \(u^{\prime }, u \in Q\) are such that \(u \in Q_i[1, t_i - 1]\) and \(u \in \delta (u^{\prime }, a)\), then \(u^{\prime } \in R(\alpha ^{\prime })\). Since \(u \lt Q_i[t_i]\), from Axiom 2 we obtain \(u^{\prime } \le v^{\prime }\) and since \(v^{\prime } \in R(\alpha ^{\prime })\), from Lemma 5.23 we conclude \(u^{\prime } \in R(\alpha ^{\prime })\).\(□\)

If \(\sigma \le n\), then we simply use the structure of Lemma 5.25. Otherwise (\(\sigma \gt n\)), let \(\Sigma ^{\prime } = \lbrace S[i]\ |\ 1\le i \le n\rbrace\) be the effective alphabet of S. We build a minimal perfect hash function \(h:\Sigma \rightarrow [0,|\Sigma ^{\prime }|-1]\) mapping (injectively) \(\Sigma ^{\prime }\) to the numbers in the range \([0,|\Sigma ^{\prime }|-1]\) and mapping arbitrarily \(\Sigma \setminus \Sigma ^{\prime }\) to the range \([0,|\Sigma ^{\prime }|-1]\). We store h using the structure described in [54]. This structure can be built in \(O(n)\) expected time, uses \(O(n)\) bits of space, and answers queries of the form \(h(x)\) in \(O(1)\) worst-case time. Note that \(|\Sigma ^{\prime }|\le n\), so we can build the structure of Lemma 5.25 starting from the string \(S^{\prime }\in [0,|\Sigma ^{\prime }| - 1]^n\) defined as \(S^{\prime }[i] = h(S[i])\). Then, rank and select operations on S can be answered as \(S.rank(i,c) = S^{\prime }.rank(i,h(c))\) and \(S.select(i,c) = S^{\prime }.select(i,h(c))\), provided that \(c\in \Sigma ^{\prime }\). Notice that the zero-order entropies of S and \(S^{\prime }\) coincide, since the character’s frequencies remain the same after applying h to the characters of S. We conclude that the overall space of the data structure is at most \(nH_0(S)(1+o(1)) + O(n)\) bits.\(□\)

Let \(n = |Q|\). In this proof, we assume that the states of Q have been sorted like in Definition 5.6: if \(\pi (v)\), for \(v\in Q\), is the unique integer such that \(v \in Q_{\pi (v)}\), then we consider the ordering \(v_1, \dots , v_n\) of Q such that for every \(1 \le i \lt j \le n\) it holds \(\pi (v_i) \lt \pi (v_j) \vee (\pi (v_i) = \pi (v_j) \wedge v_i \lt v_j)\). Moreover, we assume that \(s \in Q_1\) (again like in Definition 5.6), so \(s = v_1\). For every \(i = 1, \dots , p\), let \(e_i = |\lbrace (u,v,a)\ |\ \delta (u,a)=v, u\in Q, v\in Q_i, a\in \Sigma \rbrace |\) be the number of edges entering the ith chain, let \(\Sigma _i = (\bigcup _{u\in Q_i} \lambda (u)) \setminus \lbrace \# \rbrace\) be the set of characters labeling edges entering the ith chain, and let \(\sigma _i = |\Sigma _i|\).

We store the following data structures:

For an example, consider the automaton of Figure 9. All sequences except for bitvector \(\mathtt {IN^{\prime }}\) are reported in Example 5.7. To build bitvector \(\mathtt {IN^{\prime }}\), we first build the string \(\mathtt {IN}\) of all incoming labels of the sorted edges: \(\mathtt {IN} = aaabcabbbb\). Then, bitvector \(\mathtt {IN^{\prime }}\) marks with a bit “1” (i) the first character of each maximal unary substring in \(\mathtt {IN}\), and (ii) the characters of \(\mathtt {IN}\) labeling the first edge in each chain: \(\mathtt {IN^{\prime }} = 1001111000\).

By adding up the space of all components (note that the terms \(- \sum _{i=1}^p (e_i \log \sigma _i)\) in the dictionaries \(\Sigma _i\) and \(\sum _{i=1}^p (e_i \log \sigma _i)\) in sequence \(\mathtt {OUT}\) cancel out), we conclude that our data structures take at most \(e\log (p\sigma)(1+o(1)) + O(e)\) bits.

We proceed by showing how to solve queries 1 (string matching) and 2 (membership).

(1) Let us prove that, given a query string \(\alpha \in \Sigma ^m\), we can use our data structure to compute, in time \(O(m\cdot p^2 \cdot \log \log (p\sigma))\), the set \(T(\alpha)\) of all states reached by a \(\alpha\)-path on \(\mathcal {N}\), presented by p convex sets on the chains \(\lbrace Q_i\; | \; 1\le i \le p\rbrace\). By Lemma 5.24, it will suffice to show how to compute \(R(\alpha)\) and \(S(\alpha)\). We can recursively compute each \(R(\alpha)\) and \(S(\alpha)\) in time proportional to m by computing \(R(\alpha ^{\prime })\) and \(S(\alpha ^{\prime })\) for all prefixes \(\alpha ^{\prime }\) of \(\alpha\). Hence, we only have to show that we can update \(R (\alpha ^{\prime })\) and \(S(\alpha ^{\prime })\) with a new character, in \(O(p^2 \cdot \log \log (p\sigma))\) time. We start with the empty prefix \(\varepsilon\), whose corresponding sets are \(R (\varepsilon) = Q\) and \(S (\varepsilon) = \emptyset\). For the update we apply Lemmas 5.14 and 5.24. An inspection of the two lemmas reveals the we can update \(R (\alpha ^{\prime })\) and \(S(\alpha ^{\prime })\) with a new character by means of \(O(p^2)\) calls to the following queries.

As a consequence, we are left to show that we can solve each query in \(O(\log \log (p\sigma))\) time.

(op1). The states in \(Q_j[1,k]\) correspond to the convex set of all states (in the total order \(v_1, \dots , v_n\) of Definition 5.6) whose endpoints are \(v_l\) and \(v_r\), where \(l = \texttt {CHAIN}.select(j, 1)\) and \(r = l + k - 1\). Considering the order of edges used to define \(\mathtt {OUT}\), the set of all edges leaving a state in \(Q_j[1,k]\) forms a convex set in \(\mathtt {OUT}\) and in \(\mathtt {OUT\_DEG}\). Define:

Notice that x is equal to the number of edges leaving all states before \(v_l\), while y is the number of edges leaving all states up to \(v_r\) included. As a consequence, we have \(x \le y\). If \(x = y\), then there are no edges leaving states in \(Q_j[1,k]\) and we can immediately conclude \(\mathtt {out}(Q_j[1,k],i,a) = 0\). Assuming \(x \lt y\), \(x + 1\) and y are the endpoints of the convex set of all edges leaving states in \(Q_j[1,k]\). Hence, we are left with counting the number of such edges labeled a and reaching chain \(Q_i\). Notice that \(\mathtt {OUT}.rank(x, (i, a))\) is the number of all edges labeled a and reaching chain i whose start state comes before \(v_l\), whereas \(\mathtt {OUT}.rank(y, (i, a))\) is the number of all edges labeled a and reaching i whose start state comes before or is equal to \(v_r\). We can then conclude that \(\mathtt {out}(Q_j[1,k],i,a) = \mathtt {OUT}.rank(y, (i, a)) - \mathtt {OUT}.rank(x, (i, a))\).

(op2). First, we check whether \(a \in \Sigma _i\) by a membership query on the dictionary for \(\Sigma _i\). If \(a \not\in \Sigma _i\), we immediately conclude that the largest k with the desired properties is \(k = |Q_i|\). Assume \(a \in \Sigma _i\) and notice that the states in \(Q_i\) correspond to the convex set of all states whose endpoints are \(v_l\) and \(v_r\), where \(l = \texttt {CHAIN}.select(i, 1)\) and \(r = \texttt {CHAIN}.select(i + 1, 1) - 1\). Considering the order of edges used to define \(\mathtt {IN}\), the set of all edges entering a state in \(Q_j[1,k]\) forms a convex set in \(\mathtt {IN}\) and in \(\mathtt {IN\_DEG}\). Define

Notice that x is equal to the number of edges reaching all states before \(v_l\), while y is the number of edges reaching all states coming before or equal to \(v_r\). Since \(a \in \Sigma _i\), we have \(x \lt y\), and \(x + 1\) and y are the endpoints of the convex set of all edges reaching a state in \(Q_i\). The next step is to determine the smallest edge labeled a reaching a state in \(Q_i\). First, notice that the number of characters smaller than or equal to a in \(\Sigma _i\) can be retrieved, by Lemma 5.27, as \(\Sigma _i.rank (a)\). Notice that \(f = \mathtt {IN^{\prime }}.rank(x, 1)\) yields the number of 0-runs in \(\mathtt {IN^{\prime }}\) pertaining to chains before chain \(Q_i\) so that, since we know that \(a \in \Sigma _i\), then \(g = \mathtt {IN^{\prime }}.select(f + \Sigma _i.rank (a), 1)\) yields the smallest edge labeled a in the convex set of all edges reaching a state in \(Q_i\). We distinguish two cases for the parameter h of op2:

(op3). We simply use operation (op2) to compute the largest integer \(0\le k \le |Q_i|\) such that \(\mathtt {in}(Q_i[1,k],a) \le z - 1\). If \(k = |Q_i|\), then the desired integer does not exist, otherwise it is equal to \(k + 1\).

(op4). We first decide whether \(\Sigma _i.succ(a)\) is defined. If it is not defined, then the largest integer with the desired property is \(|Q_i|\). Now assume that \(\Sigma _i.succ(a)\) is defined. Then the largest integer with the desired property is simply the largest integer \(0 \le k \le |Q_i| - 1\) such that \(\mathtt {in}(Q_i[1,k],\Sigma _i.succ(a)) \le 0\), which can be computed using (op2).

(2) Let us prove that, given a query string \(\alpha \in \Sigma ^m\), we can use our data structure to compute \(I_\alpha\) in \(O(m\cdot p^2 \cdot \log \log (p\sigma))\) time, represented as p ranges on the chains in \(\lbrace Q_i\; | \; 1\le i \le p\rbrace\). We claim that it will suffice to run the same algorithm used in the previous point, starting with \(R = \lbrace v_1 \rbrace\) and \(S = \emptyset\). Indeed, consider the automaton \(\mathcal {N}^{\prime }\) obtained from \(\mathcal {N}\) by adding a new initial state \(v_0\) and adding exactly one edge from \(v_0\) to \(v_1\) (the old initial state) labeled with \(\#\), a character smaller than every character in the alphabet \(\Sigma\). Let \(\le ^{\prime }\) the co-lex order on \(\mathcal {N}^{\prime }\) obtained from \(\le\) by adding the pair \(\lbrace (v_0, v_1) \rbrace\), and consider the \(\le ^{\prime }\)-chain partition obtained from \(\lbrace Q_i\; | \; 1\le i \le p\rbrace\) by adding \(v_0\) to \(Q_1\). It is immediate to notice that for every \(k = 1, \dots , n\) and for every string \(\alpha \in \Sigma ^*\) we have that \(v_k \in I_\alpha\) on \(\mathcal {N}\) if and only if \(v_k \in T(\#\alpha)\) on \(\mathcal {N^{\prime }}\). Since \(v_0\) has no incoming edges and on \(\mathcal {N}^{\prime }\) we have \(R(\#) = \lbrace v_0, v_1 \rbrace\) and \(S (\#) = \lbrace v_0 \rbrace\), the conclusion follows. Now, given \(I_\alpha\), we can easily check whether \(\alpha \in \mathcal {L(N)}\). Indeed, for every \(i = 1, \dots , p\) we know the integers \(l_i\) and \(t_i\) such that \(I_\alpha ^i = Q_i[l_i + 1, t_i]\), and we decide whether some of these states are final by computing \(f = \texttt {CHAIN}.select (i, 1)\), and then checking whether \(\texttt {FINAL}.rank(f + l_i - 1, 1) - \texttt {FINAL}.rank(f + t_i - 1, 1)\) is larger than zero.\(□\)

Consider the set of all edges labeled with character a, leaving a state in \(Q_j\) and entering a state in \(Q_i\). Then, \(B_{j, i, a}\) is obtained by picking and sorting all start states of these edges, and \(C_{j, i, a}\) is obtained by picking and sorting all end states of these edges. The conclusion follows from Axiom 2.\(□\)

From \(\mathtt {CHAIN}\) and \(\mathtt {FINAL,}\) we can retrieve the chain of each state, and we can decide which states are final. We only have to show how to retrieve the set \(\lbrace (v_f,v_g,a)\ |\ v_g\in \delta (v_f,a),\ v_f,v_g\in Q, a\in \Sigma \rbrace\) of all NFA’s transitions. By Lemma 5.32, we only have to prove that for every \(1 \le i, j \le p,\) and for every \(a \in \Sigma\) we can retrieve \(B_{j, i, a}\) and \(C_{j, i, a}\). We will use ideas similar to those employed in the proof of Lemma 5.15.

Let us show how to retrieve \(B_{j, i, a}\) for every \(1 \le i, j \le p,\) and for every \(a \in \Sigma\). Fix i, j and a. From \(\mathtt {OUT\_DEG}\) we can retrieve the number of edges leaving each state in the jth chain. Then, the definition of \(\mathtt {OUT}\) implies that, for every state in the jth chain, we can retrieve the label and the destination chain of each edge leaving the state, so we can decide how many times a state in the jth chain occurs in \(B_{j, i, a}\).

Let us show how to retrieve \(C_{j, i, a}\) for every \(1 \le i, j \le p,\) and for every \(a \in \Sigma\). Fix i, j and a. From \(\mathtt {IN\_DEG}\) we can retrieve the number of edges entering each state in the ith chain. From \(\mathtt {OUT}\) we can retrieve all characters (with multiplicities) labeling some edge entering the ith-chain. As a consequence, Axiom 1 implies that, for every state in the ith chain, we can retrieve the label of each edge entering the state. Moreover, the definition of \(\mathtt {IN\_CHAIN}\) implies that, for every state in the ith cain, we can retrieve the start chain of each edge entering the state, so we can decide how many times a state in the ith chain occurs in \(C_{j, i, a}\).\(□\)

The bound of \(\log (p^2\sigma) + O(1)\) bits per transition follows easily from the definitions of aBWT (Definition 5.6) and \(\tt IN\_CHAIN\) (Definition 5.31). In order to bound this space as a function of the number of states, we use the bound \(|\delta | \le 2p\sigma n\) implied by Lemma 2.6, where \(|\delta |\) is the number of transitions.\(□\)