Detecting possibly frequent change-points: Wild Binary Segmentation 2 and steepest-drop model selection

Abstract

Many existing procedures for detecting multiple change-points in data sequences fail in frequent-change-point scenarios. This article proposes a new change-point detection methodology designed to work well in both infrequent and frequent change-point settings. It is made up of two ingredients: one is “Wild Binary Segmentation 2” (WBS2), a recursive algorithm for producing what we call a ‘complete’ solution path to the change-point detection problem, i.e. a sequence of estimated nested models containing $0, \ldots , T-1$ change-points, where T is the data length. The other ingredient is a new model selection procedure, referred to as “Steepest Drop to Low Levels” (SDLL). The SDLL criterion acts on the WBS2 solution path, and, unlike many existing model selection procedures for change-point problems, it is not penalty-based, and only uses thresholding as a certain discrete secondary check. The resulting WBS2.SDLL procedure, combining both ingredients, is shown to be consistent, and to significantly outperform the competition in the frequent change-point scenarios tested. WBS2.SDLL is fast, easy to code and does not require the choice of a window or span parameter.

Proofs

Proof of Theorem 3.1

Part (i). The statement is trivially true if $T = 1$. Suppose, inductively, that it is true for data lengths $1, \ldots , T-1$. For an input data sequence of length T, the first addition to the solution path breaks the domain of operation into two, of lengths $b_{m_0}$ and $T-b_{m_0}$. Therefore by the inductive hypothesis, the length of the solution path for the input data will be $1 + (b_{m-0} - 1) + (T-b_{m_0} - 1) = T-1$, which completes the proof of part (i).

Part (ii). The computation of the CUSUM statistic for all b in formula (2) is of computational order $O(e-s)$. Therefore the addition of elements to the solution path from all sub-domains within a single scale of operation is of computational order $O({\tilde{M}}T)$. If the scale has reached J, there is nothing to do and the procedure has stopped. Therefore the overall computational complexity before the sorting step is of order $O({\tilde{M}}JT)$. The sorting can be performed in computational time $O(T \log \,T)$, which never dominates $O({\tilde{M}}JT)$ as the smallest possible J is of computational order $O(\log \,T)$, which is straightforward to see from its definition. This completes the proof of part (ii).

Part (iii). We first set the probabilistic framework in which we analyse the behaviour of the WBS2 solution path algorithm. With the change-point locations denoted by $\eta _1, \ldots , \eta _N$ (with the additional notation $\eta _0 = 1$, $\eta _{N+1} = T+1$), we define the intervals

$$\begin{aligned} I_{i,j}^{k,l} = [\max (1, \eta _i + k), \min (T, \eta _j + l)], \end{aligned}$$

where $k,l \in \mathbb {Z} \cap [- C_1(\Delta )({\underline{f}}_T)^{-2}\log \,T, C_1(\Delta )({\underline{f}}_T)^{-2}\log \,T]$ for each $1 \le i+1 < j \le {N+1}$. Suppose that on each interval $I_{i,j}^{k,l}$, ${\tilde{M}}$ intervals $\{[s_m, e_m]\}_{m=1}^{{\tilde{M}}}$ have been drawn, with the start- and end-points having been drawn uniformly with replacement from $I_{i,j}^{k,l}$ (if ${\tilde{M}} > |I_{i,j}^{k,l}|(|I_{i,j}^{k,l}| - 1)/2$, then we understand $\{[s_m, e_m]\}_{m=1}^{{\tilde{M}}}$ to contain all possible subintervals of $I_{i,j}^{k,l}$). Note we do not reflect the (stochastic) dependence of $s_m, e_m$ on i, j, k, l so as not to over-complicate the notation. As in the proof of Theorem 3.2 in Fryzlewicz (2014), we define intervals ${{\mathcal {I}}}_r$ between change-points in such a way that their lengths are at least of order O(T), and they are separated from the change-points also by distances at least of order O(T). To fix ideas, define ${{\mathcal {I}}}_r = [\eta _{r-1} + \frac{1}{3}(\eta _r - \eta _{r-1}), \eta _{r-1} + \frac{2}{3}(\eta _r - \eta _{r-1})]$, $r = 1, \ldots , N+1$. For each interval $I_{i,j}^{k,l}$, we are interested in the following event

$$\begin{aligned} A_{i,j}^{k,l} = \{ \exists _{m_0\in \{1, \ldots , {\tilde{M}}\}} \,\, \exists _{r\in \{i+1,\ldots ,j-1\}}\,\, (s_{m_0}, e_{m_0}) \in {{\mathcal {I}}}_r \times {{\mathcal {I}}}_{r+1} \}. \end{aligned}$$

Note that

$$\begin{aligned} P\{(A_{i,j}^{k,l})^c\}\le & {} \prod _{m=1}^{{\tilde{M}}} P\left\{ (s_m, e_m) \not \in \bigcup _{r=i+1}^{j-1} {{\mathcal {I}}}_r \times {{\mathcal {I}}}_{r+1} \right\} \\\le & {} \prod _{m=1}^{{\tilde{M}}} \max _{r\in \{i+1,\ldots ,j-1\}} (1 - P\{(s_m, e_m) \in {{\mathcal {I}}}_r \times {{\mathcal {I}}}_{r+1}\} ) \le (1 - \delta ^2/9)^{{\tilde{M}}}, \end{aligned}$$

and hence

$$\begin{aligned} P\left( \bigcap _{i,j,k,l} A_{i,j}^{k,l} \right)\ge & {} 1 - \sum _{i,j,k,l} P\{(A_{i,j}^{k,l})^c\} \\\ge & {} 1 - \frac{1}{2}N(N+1) (2C_1(\Delta )({\underline{f}}_T)^{-2}\log \,T + 1)^2 (1 - \delta ^2/9)^{{\tilde{M}}}. \end{aligned}$$

Define further the following event

$$\begin{aligned} {{\mathcal {D}}}_{\Delta , T} = \left\{ \forall \,\, 1 \le s \le b < e \le T\qquad |{\tilde{\varepsilon }}_{s,e}^b| \le 2 \sigma \{ (1 + \Delta ) \log \, T\}^{1/2} \right\} , \end{aligned}$$

where ${\tilde{\varepsilon }}_{s,e}^b$ is defined as in formula (2) with $\varepsilon _t$ in place of $X_t$. It is the statement of Lemma A.1 in Fryzlewicz (2018) that ${\mathcal {B}}_{\Delta , T} \subseteq {{\mathcal {D}}}_{\Delta , T}$.

The following arguments apply on the set $\bigcap _{i,j,k,l} A_{i,j}^{k,l} \cap {{\mathcal {D}}}_{\Delta , T}$ for any fixed $\Delta > 0$. Proceeding exactly as in the proof of Theorem 3.2 in Fryzlewicz (2014), at the start of the procedure we have $s = 1$, $e = T$, and in view of the fact that we are on $A_{0,N+1}^{0,0} \cap {{\mathcal {D}}}_{\Delta , T}$, the procedure finds a $b_{m_0} \in [s_{m_0}, e_{m_0}] \subseteq [s,e]$ such that

• (a) there exists an $r \in \{1, 2, \ldots , N\}$ such that $|\eta _r - b_{m_0}| \le C_1(\Delta )({\underline{f}}_T)^{-2}\log \,T$, and

• (b) $|{\tilde{X}}_{s_{m_0}, e_{m_0}}^{b_{m_0}}| \gtrsim T^{1/2} {\underline{f}}_T$,

where the $\gtrsim$ symbol means “of the order of or larger”. Again by arguments identical to those in the proof of Theorem 3.2 in Fryzlewicz (2014), on each subsequent segment containing previously undetected change-points, we are on one of the sets $A_{i,j}^{k,l} \cap {{\mathcal {D}}}_{\Delta , T}$, and therefore the procedure again finds a $b_{m_0}$ satisfying properties (a), (b) above for a certain previously undetected change-point $\eta _r$, until all change-points have been identified in this way. Once all change-points have been identified, by Lemma A.5 of Fryzlewicz (2014), all maximisers $b_{m_0}$ of the absolute CUSUM statistics $|{\tilde{X}}_{s_m, e_m}^b|$ (over b) are such that $|{\tilde{X}}_{s_{m_0}, e_{m_0}}^{b_{m_0}}| \le C_2(\Delta ) \log ^{1/2}T$. As $\log ^{1/2}T = o(T^{1/2}{\underline{f}}_T)$, for T large enough, the sorting of the elements of $\tilde{\mathcal {P}}$ does not move any elements from the first N to the last $T-1-N$ or vice versa, which completes the proof of part (iii). $\square$

Proof of Theorem 3.2

With the notation as in the statement of the theorem and in the proof of Theorem 3.1, the following arguments apply on the set $\bigcap _{i,j,k,l} A_{i,j}^{k,l} \cap {{\mathcal {D}}}_{\Delta , T} \cap \Theta _{\theta , T}$ for any fixed $\Delta > 0$. Let T be large enough for the assertion of Theorem 3.1 to hold. Further, let ${\tilde{C}}$ be large enough for the following inequality to hold

$$\begin{aligned} {\tilde{\zeta }}_T = {\tilde{C}}{\hat{\sigma }}_T \{2 \log \,T\}^{1/2} > \max ( C_2(\Delta )\log ^{1/2}T, 2\sigma \{ (1+\Delta ) \log \,T \}^{1/2} ) \end{aligned}$$

(6)

(this is always possible on $\Theta _{\theta , T}$). If $N = 0$, then in view of inequality (6) and the fact that we are on ${{\mathcal {D}}}_{\Delta , T}$, from the definition of the SDLL algorithm we necessarily have ${\hat{N}} = 0$. If $N > 0$, then by part (iii) of Theorem 3.1, we must have $K + 1 \ge N$. If $|{\tilde{X}}_{s_{K+1}, e_{K+1}}^{b_{K+1}}| > {\tilde{\zeta }}_T$, then by inequality (6), we have $N = K + 1$ and the SDLL procedure correctly identifies ${\hat{N}} = K + 1 = N$. If $|{\tilde{X}}_{s_{K+1}, e_{K+1}}^{b_{K+1}}| < {\tilde{\zeta }}_T$, then we necessarily have $N < K+1$, and by part (iii) of Theorem 3.1, we have, for $Z_k = \log |{\tilde{X}}_{s_k, e_k}^{b_k}| - \log |{\tilde{X}}_{s_{k+1}, e_{k+1}}^{b_{k+1}}|$,

$$\begin{aligned} Z_N\sim & {} \log \,T,\\ Z_k\le & {} \log \left( \frac{(1+ \theta ) C_2(\Delta )}{ \beta {\tilde{C}} \sigma 2^{1/2}} \right) \quad \text {for}\quad k=N+1, \ldots , K\quad \text {if this range is non-empty}. \end{aligned}$$

Therefore, from the definition of the SDLL and by part (iii) of Theorem 3.1, for T large enough, ${\hat{N}} = N$ will be chosen. This completes the proof of the Theorem. $\square$