Exact Recovery of Stochastic Block Model by Ising Model

Abstract

In this paper, we study the phase transition property of an Ising model defined on a special random graph—the stochastic block model (SBM). Based on the Ising model, we propose a stochastic estimator to achieve the exact recovery for the SBM. The stochastic algorithm can be transformed into an optimization problem, which includes the special case of maximum likelihood and maximum modularity. Additionally, we give an unbiased convergent estimator for the model parameters of the SBM, which can be computed in constant time. Finally, we use metropolis sampling to realize the stochastic estimator and verify the phase transition phenomenon thfough experiments.

1. Introduction

In network analysis, community detection consists in inferring the group of vertices that are more densely connected in a graph [1]. It has been used in many domains, such as recommendation systems [2], task allocation in distributed computing [3], gene expressions [4], and so on. The stochastic block model (SBM) is one of the most commonly used statistical models for community detection problems [5,6]. It provides a benchmark artificial dataset to evaluate different community detection algorithms and inspires the design of many algorithms for community detection tasks. These algorithms, such as semi-definite relaxation, spectral clustering, and label propagation, not only have theoretical guarantees when applied to the SBM, but also perform well on datasets without the SBM assumption. The study of the theoretical guarantee of the SBM model can be divided between the problem of exact recovery and that of partial recovery. Exact recovery requires that the estimated community should be exactly the same as the underlining community structure of the SBM whereas partial recovery expects the ratio of misclassified nodes to be as small as possible. For both cases, the asymptotic behavior of the detection error is analyzed when the scale of the graph tends to infinity. There are already some well-known results for the exact recovery problem on the SBM. To name but a few, Abbe and Mossel established the exact recovery region for a special sparse SBM with two communities [7,8]. Later on, the result was extended to a general SBM with multiple communities [9].

Parameter inference in the SBM is often considered alongside the exact recovery problem. Previous inference methods require the joint estimation of node labels and model parameters [10], which have high complexity since the recovery and inference tasks are done simultaneously. In this article, we will decouple the inference and recovery problems, and propose an unbiased convergent estimator for SBM parameters when the number of communities is known. Once the estimator is obtained, the recovery condition can be checked to determine whether it is possible to recover the labels exactly. Additionally, the estimated parameter will guide the choice of parameters for our proposed stochastic algorithm.

In this article, the exact recovery of the SBM is analyzed by considering the Ising model, which is a probability distribution of node states [11]. We use the terms node states and node labels interchangeably throughout this paper, both of which refer to the membership of the underlining community. The Ising model was originally proposed in statistical mechanics to model the ferromagnetism phenomenon but has wide applications in neuroscience, information theory, and social networks. Among different variants of Ising models, the phase transition property is shared. Phase transition can be generally formulated when some information quantity changes sharply in a small neighborhood of parameters. Based on the random graph generated by an SBM with two underlining communities, the connection of the SBM and the Ising model was first studied by [12]. Our work will extend the existing result to the multiple community case, establish the phase transition property, and give the recovery error an upper bound. The error bounds decay in a polynomially fast rate in different phases. Then we will propose an alternative approach to estimate the labels by finding the Ising state with maximal probability. Compared with sampling from the Ising model directly, we will show that the optimization approach has a sharper error upper bound. Solving the optimization problem is a generalization of maximum likelihood and also has a connection with maximum modularity. Additionally, searching the state with maximal probability could also be done within all balanced partitions. We will show that this constrained search is equivalent to the graph minimum cut problem, and the detection error upper bound for the constrained maximization will also be given.

The exact solution to maximize the probability function or exact sampling from the Ising model is NP-hard. Many polynomial time algorithms have been proposed for approximation purposes. Among these algorithms, simulated annealing performs well and produces a solution that is very close to the true maximal value [13]. On the other hand, in the original Ising model, metropolis sequential sampling is used to generate samples for the Ising model [14]. Simulated annealing can be regarded as metropolis sampling with decreasing temperature. In this article, we will use the metropolis sampling technique to sample from the Ising model defined on the SBM. This approximation enables us to verify the phase transition property of our Ising model numerically.

This paper is organized as follows. Firstly, in Section 3 we introduce the SBM and give an estimator for the parameters of the SBM. Then, in Section 4, our specific Ising model is given and its phase transition property is obtained. Derived from the Ising model, in Section 5, the energy minimization method is introduced, and we establish its connection with maximum likelihood and modularity maximization algorithm. Furthermore, in Section 6, we realize the Ising model using the metropolis algorithm to generate samples. Numerical experiments and conclusion are given lastly to finish this paper.

Throughout this paper, the  community number is denoted by k; the random undirected graph G is written as $G(V,E)$ with vertex set V and edge set E; $V=\{1,\dots ,n\}=:\left[n\right]$; the label of each node is $X_i$, which is chosen from $W=\{1,\omega ,\dots ,{\omega }^{k-1}\}$, and we further require W to be a cyclic group with order k; $W^n$ is the n-ary Cartesian power of W; f is a permutation function on W and is extended to $W^n$ in an element-wise manner; $U^c$ is the complement set of U and $\left|U\right|$ is the cardinality of U; the set $S_k$ is used to represent all permutation functions on W and $S_k\left(\sigma \right):=\left\{f\left(\sigma \right)\right|f\in S_k\}$ for $\sigma \in W^n$; the indicator function $\delta (x,y)$ is defined as $\delta (x,y)=1$ when $x=y$, and $\delta (x,y)=0$ when $x\ne y$; $f\left(n\right)=O\left(g\right(n\left)\right)$ if there exists a constant $c>0$ such that $f\left(n\right)\le cg\left(n\right)$ for large n; $f\left(n\right)=o\left(g\right(n\left)\right)$ holds if ${lim}_{n\to \infty }\frac{f\left(n\right)}{g\left(n\right)}=0$; we define the distance of two vectors as: $dist(\sigma ,{\sigma }^{\prime })=\left|\{i\in \left[n\right]:{\sigma }_i\ne {\sigma }_i^{\prime }\}\right|\mathrm{for}\sigma ,{\sigma }^{\prime }\in W^n$ and the distance of a vector to a space $S\subseteq W^n$ as $dist(\sigma ,S):=min\{dist(\sigma ,{\sigma }^{\prime })|{\sigma }^{\prime }\in S\}$. For example, when $n=2$ and $k=2$, $\sigma =(1,\omega )\in W^2$; ${\omega }^0=1$; $\omega ·\omega ={\omega }^2=1$; let f be a mapping such that $f\left(1\right)=\omega$ and $f\left(\omega \right)=1$, then $f\in S_2$ and $f\left(\sigma \right)=(\omega ,1)$; $dist(\sigma ,f(\sigma \left)\right)=2$; $S_k\left(\sigma \right)=\{\sigma ,f\left(\sigma \right)\}$; and $S_k^c\left(\sigma \right)=\{(1,1),(\omega ,\omega )\}$.

2. Related Works

The classical Ising model is defined on a lattice and confined to two states $\{\pm 1\}$. This definition can be extended to a general graph and multiple-state case [15]. In [16], Liu considered the Ising model as defined on a graph generated by sparse SBM and his focus was to compute the log partition function, which was averaged over all random graphs. In [17], an Ising model with a repelling interaction was considered on a fixed graph structure, and the phase transition condition was established, which involves both the attracting and repelling parameters. Our Ising model derives from the work of [12], but we extend their results by considering the error upper bound and multiple-community case.

The exact recovery condition for the SBM can be derived as a special case of many generalized models, such as pairwise measurements [18], minimax rates [19], and side information [20]. The Ising model in this paper provides another way to extend the SBM model and derives the recovery condition. Additionally, the error upper bound for exact recovery of the two-community SBM by constrained maximum likelihood has been obtained in [7]. Compared with previous results, we establish a sharper upper bound for the multiple-community case in this paper.

The connection between maximum modularity and maximal likelihood was investigated in [21]. To get an optimal value of maximum modularity approximately, simulated annealing was exploited [22], which proceeds by using the partition approach, while the Metropolis sampling used in this paper is applied to estimate the node membership directly.

3. Stochastic Block Model and Parameter Estimation

In this paper, we consider a special symmetric stochastic block model (SSBM), which is defined as follows:

Definition 1 (SSBM with k communities).

Let $0\le q<p\le 1$, $V=\left[n\right]$ and $X=(X_1,\dots ,X_n)\in W^n$. X satisfies the constraint that $|\{v\in \left[n\right]:X_v=u\}|=\frac{n}{k}$ for $u\in W$. The random graph G is generated under $SSBM(n,k,p,q)$ if the following two conditions are satisfied.

• There is an edge of G between the vertices i and j with probability p if $X_i=X_j$ and with probability q if $X_i\ne X_j$.
• The existences of each edge are mutually independent.

To explain SSBM in more detail, we define the random variable $Z_{ij}:=\mathbb{1}[\{i,j\}\in E\left(G\right)]$, which is the indicator function of the existence of an edge between nodes i and j. Given the node labels X, $Z_{ij}$ follows Bernoulli distribution, whose expectation is given by:

$$\mathbb{E}\left[Z_{ij}\right]=\left\{\begin{array}{cc}p\hfill & \mathrm{if}{X}_i=X_j\hfill \\ q\hfill & \mathrm{if}{X}_i\ne X_j\hfill \end{array}\right.$$

(1)

Then the random graph G with n nodes is completely specified by $Z:=\{Z_{ij},1\le i<j\le n\}$ in which all $Z_{ij}$ are jointly independent. The probability distribution for SSBM can be written as:

$$\begin{array}{cc}\hfill & P_G\left(G\right):=P_G(Z=z|X)=p^{{\sum }_{X_i=X_j}{z}_{ij}}{q}^{{\sum }_{X_i\ne X_j}{z}_{ij}}\hfill \\ \hfill & ·{(1-p)}^{{\sum }_{X_i=X_j}(1-z_{ij})}{(1-q)}^{{\sum }_{X_i\ne X_j}(1-z_{ij})}\hfill \end{array}$$

(2)

We will use the notation ${\mathcal{G}}_n$ to represent the set containing all graphs with n nodes. By the normalization property, $P_G\left({\mathcal{G}}_n\right)={\sum }_{G\in {\mathcal{G}}_n}{P}_G\left(G\right)=1$.

In Definition 1, we have supposed that the node label X is fixed instead of a uniformly distributed random variable. Since the maximum posterior estimator is equivalent to the maximum likelihood when the prior is uniform, these two definitions are equivalent. Although the random variable definition is more commonly used in previous literature [6], fixing X makes our formal analysis more concise.

Given the SBM, the exact recovery problem can be formally defined as follows:

Definition 2 (Exact recovery in SBM).

Given X, the random graph G is drawn under $SSBM(n,k,p,q)$. We say that the exact recovery is solvable for $SSBM(n,k,p,q)$ if there exists an algorithm that takes G as input and outputs $\widehat{X}$ such that:

$$P_a\left(\widehat{X}\right):=P(\widehat{X}\in S_k\left(X\right))\to 1\mathrm{as}n\to \infty$$

In the above definition, the notation $P_a\left(\widehat{X}\right)$ is called the probability of accuracy for estimator $\widehat{X}$. Let $P_e\left(\widehat{X}\right)=1-P_a\left(\widehat{X}\right)$ represent the probability of error. Definition 2 can also be formulated as $P_e\left(\widehat{X}\right)\to 0$ as $n\to \infty$. The notation $\widehat{X}\in S_k\left(X\right)$ means that we can only expect a recovery up to a global permutation of the ground truth label vector X. This is common in unsupervised learning as no anchor exists to assign labels to different communities. Additionally, given a graph G, the algorithm can be either deterministic or stochastic. Generally speaking, the probability of $\widehat{X}\in S_k\left(X\right)$ should be understood as ${\sum }_{G\in {\mathcal{G}}_n}{P}_G\left(G\right)P_{\widehat{X}|G}(\widehat{X}\in S_k\left(X\right))$, which reduced to $P_G(\widehat{X}\in S_k\left(X\right))$ for the deterministic algorithm.

For constants $p,q$, which are irrelevant with the graph size n, we can always find algorithms to recover X such that the detection error decreases exponentially fast as n increases; that is to say, the task with a dense graph is relatively easy to handle. Within this paper, we consider a sparse case when $p=\frac{alogn}{n},q=\frac{blogn}{n}$. This case corresponds to the sparsest graph when exact recovery of the SBM is possible. And under this condition, a well known result [9] states that exact recovery is possible if and only if:

$$\sqrt{a}-\sqrt{b}>\sqrt{k}$$

(3)

Before diving into the exact recovery problem, we first consider the inference problem for SBM. Suppose k is known, and we want to estimate $a,b$ from the graph G. We offer a simple method by counting the number of edges $T_1$ and the number of triangles $T_2$ of G, and the estimators $\widehat{a},\widehat{b}$ are obtained by solving the following equation systems:

$$\begin{array}{cc}\hfill \frac{x+(k-1)y}{2k}& =\frac{T_1}{nlogn}\hfill \end{array}$$

(4)

$$\begin{array}{cc}\hfill \frac{1}{k^2}\left(\frac{x^3}{6}+\frac{k-1}{2}x{y}^2+(k-1)(k-2)\frac{y^3}{6}\right)& =\frac{T_2}{{log}^{3}n}\hfill \end{array}$$

(5)

The theoretical guarantee for the solution is given by the following theorem:

Theorem 1.

When n is large enough, the equation system of Equations (4) and (5) has the unique solution $(\widehat{a},\widehat{b})$, which are unbiased consistent estimators of $(a,b)$. That is, $\mathbb{E}\left[\widehat{a}\right]=a,\mathbb{E}\left[\widehat{b}\right]=b$, and $\widehat{a}$ and $\widehat{b}$ converge to $a,b$ in probability, respectively.

Given a graph generated by the SBM, we can use Theorem 1 to obtain the estimated $a,b$ and determine whether exact recovery of label X is possible by Equation (3). Additionally, Theorem 1 provides good estimation of $a,b$ to initialize their parameters of some recovery algorithm like maximum likelihood or our proposed Metropolis sampling in Section 6.

4. Ising Model for Community Detection

In the previous section, we have defined SBM and its exact recovery problem. While SBM is regarded as obtaining the graph observation G from node label X, the Ising model provides a way to generate estimators of X from G by a stochastic procedure. The definition of such an Ising model is given as follows:

Definition 3 (Ising model with k states).

Given a graph G sampled from $SSBM(n,k,\frac{alogn}{n},\frac{blogn}{n})$, the Ising model with parameters $\gamma ,\beta >0$ is a probability distribution of the state vector $\sigma \in W^n$ whose probability mass function is

$$\begin{array}{c}\hfill P_{\sigma |G}(\sigma =\overline{\sigma })=\frac{exp(-\beta H\left(\overline{\sigma }\right))}{Z_G(\alpha ,\beta )}\end{array}$$

(6)

where

$$H\left(\overline{\sigma }\right)=\gamma \frac{logn}{n}\sum _{\{i,j\}\notin E\left(G\right)}\delta ({\overline{\sigma }}_i,{\overline{\sigma }}_j)-\sum _{\{i,j\}\in E\left(G\right)}\delta ({\overline{\sigma }}_i,{\overline{\sigma }}_j)$$

(7)

The subscript in $P_{\sigma |G}$ indicates that the distribution depends on G, and  $Z_G(\alpha ,\beta )$ is the normalizing constant for this distribution.

In physics, $\beta$ refers to the inverse temperature and $Z_G(\gamma ,\beta )$ is called the partition function. The Hamiltonian energy $H\left(\overline{\sigma }\right)$ consists of two terms: the repelling interaction between nodes without edge connection and the attracting interaction between nodes with edge connection. The parameter $\gamma$ indicates the ratio of the strength of these two interactions. The term $\frac{logn}{n}$ is added to balance the two interactions because there are only $O\left(\frac{logn}{n}\right)$ connecting edges for each node. The probability of each state is proportional to $exp(-\beta H\left(\overline{\sigma }\right))$, and the state with the largest probability corresponds to that with the lowest energy.

The classical definition of the Ising model is specified by $H\left(\sigma \right)=-{\sum }_{(i,j)\in E\left(G\right)}{\sigma }_i·{\sigma }_j$ for ${\sigma }_i=\pm 1$. There are two main differences between Definition 3 and the classical one. Firstly, we add a repelling term between nodes without an edge connection. This makes these nodes have a larger probability to take different labels. Secondly, we allow the state at each node to take k values from W instead of the two values $\pm 1$. When $\gamma =0$ and $k=2$, Definition 3 is reduced to the classical definition of the Ising model up to a scaling factor.

Definition 3 gives a stochastic estimator ${\widehat{X}}^{*}$ for X: ${\widehat{X}}^{*}$ is one sample generated from the Ising model, which is denoted as ${\widehat{X}}^{*}\sim \mathrm{Ising}(\gamma ,\beta )$. The exact recovery error probability for ${\widehat{X}}^{*}$ can be written as $P_e\left({\widehat{X}}^{*}\right):={\sum }_{G\in {\mathcal{G}}_n}{P}_G\left(G\right)P_{\sigma |G}\left(S_k^c\left(X\right)\right)$. From this expression we can see that the error probability is determined by two parameters $(\gamma ,\beta )$. When these parameters take proper values, $P_e\left({\widehat{X}}^{*}\right)\to 0$, and the exact recovery of the SBM is achievable. On the contrary, $P_e\left({\widehat{X}}^{*}\right)\to 1$ if $(\gamma ,\beta )$ takes other values. These two cases are summarized in the following theorem:

Theorem 2.

Define the function $g\left(\beta \right),\tilde{g}\left(\beta \right)$ as follows:

$$g\left(\beta \right)=\frac{b{e}^{\beta }+a{e}^{-\beta }}{k}-\frac{a+b}{k}+1$$

(8)

and:

$$\tilde{g}\left(\beta \right)=\left\{\begin{array}{cc}g\left(\beta \right)\hfill & \beta \le \overline{\beta }=\frac{1}{2}log\frac{a}{b}\hfill \\ g\left(\overline{\beta }\right)=1-\frac{{(\sqrt{a}-\sqrt{b})}^2}{k}\hfill & \beta >\overline{\beta }\hfill \end{array}\right.$$

(9)

where $\overline{\beta }=arg{min}_{\beta >0}g\left(\beta \right)$. Let ${\beta }^{*}$ be defined as:

$${\beta }^{*}=log\left(\frac{a+b-k-\sqrt{{(a+b-k)}^2-4ab)}}{2b}\right)$$

(10)

which is the solution to the equation $g\left(\beta \right)=0$ and ${\beta }^{*}<\overline{\beta }$. Then depending on how $(\gamma ,\beta )$ take values, for any given $ϵ>0$ and ${\widehat{X}}^{*}\sim \mathrm{Ising}(\gamma ,\beta )$, when n is sufficiently large, we have:

• If $\gamma >b$ and $\beta >{\beta }^{*}$, $P_e\left({\widehat{X}}^{*}\right)\le n^{\tilde{g}\left(\beta \right)/2+ϵ}$;
• If $\gamma >b$ and $\beta <{\beta }^{*}$, $P_a\left({\widehat{X}}^{*}\right)\le (1+o\left(1\right))max\{n^{g\left(\overline{\beta }\right)},n^{-g\left(\beta \right)+ϵ}\}$;
• If $\gamma <b$, $P_a\left({\widehat{X}}^{*}\right)\le exp(-Cn)$ for any $C>0$.

By simple calculus, $\tilde{g}\left(\beta \right)<0$ for $\beta >{\beta }^{*}$ and $g\left(\beta \right)>0$ for $\beta <{\beta }^{*}$. $g\left(\overline{\beta }\right)<0$ follows from Equation (3). The illustration of $g\left(\beta \right),\tilde{g}\left(\beta \right)$ is shown in Figure 1a. Therefore, for sufficiently small $ϵ$ and as $n\to \infty$, the upper bounds in Theorem 2 all converge to 0 at least in polynomial speed. Therefore, Theorem 2 establishes the sharp phase transition property of the Ising model, which is illustrated in Figure 1b.

Theorem 2 can also be understood from the marginal distribution for $\sigma :P_{\sigma }(\sigma =\overline{\sigma })={\sum }_{G\in {\mathcal{G}}_n}{P}_G\left(G\right)P_{\sigma |G}(\sigma =\overline{\sigma })$. Let $D(\sigma ,{\sigma }^{\prime })$ be the event when $\sigma$ is closest to ${\sigma }^{\prime }$ among all its permutations. That is,

$$D(\sigma ,{\sigma }^{\prime }):=\{\sigma =arg\underset{f\in S_k}{min}dist(f\left(\sigma \right),{\sigma }^{\prime })\}$$

(11)

Then Theorem 2 can be stated with respect to the marginal distribution $P_{\sigma }$:

Corollary 1.

Suppose $\gamma >b$, depending on how β takes values:

• When $\beta >{\beta }^{*}$, $P_{\sigma }(\sigma =X|D(\sigma ,X))=1-o\left(1\right)$;
• When $\beta <{\beta }^{*}$, $P_{\sigma }(\sigma =X|D(\sigma ,X))=o\left(1\right)$.

Below we outline the proof ideas of Theorem 2. The insight is obtained from the analysis of the one-flip energy difference. This useful result is summarized in the following lemma:

Lemma 1.

Suppose ${\overline{\sigma }}^{\prime }$ differs from $\overline{\sigma }$ only at position r by ${\overline{\sigma }}_r^{\prime }={\omega }^s·{\overline{\sigma }}_r$. Then the change of energy is:

$$\begin{array}{cc}\hfill H\left({\overline{\sigma }}^{\prime }\right)-H\left(\overline{\sigma }\right)& =(1+\gamma \frac{logn}{n})\sum _{i\in N_r\left(G\right)}{J}_s({\overline{\sigma }}_r,{\overline{\sigma }}_i)\hfill \\ \hfill & +\gamma \frac{logn}{n}(m({\omega }^s·{\overline{\sigma }}_r)-m\left({\overline{\sigma }}_r\right)+1)\hfill \end{array}$$

(12)

where $m\left({\omega }^j\right):=\left|\right\{i\in \left[n\right]|{\overline{\sigma }}_i={\omega }^j\left|\right\}$, $N_r\left(G\right):=\left\{j\right|(r,j)\in E\left(G\right)\}$ and $J_s(x,y)=\delta (x,y)-\delta ({\omega }_s·x,y)$.

Lemma 1 gives an explicit way to compare the probability of two neighboring states by the following equality:

$$\frac{P_{\sigma |G}(\sigma ={\overline{\sigma }}^{\prime })}{P_{\sigma |G}(\sigma =\overline{\sigma })}=exp(-\beta (H\left({\overline{\sigma }}^{\prime }\right)-H\left(\overline{\sigma }\right)))$$

(13)

Additionally, since the graph is sparse and every node has $O(logn)$ neighbors, from Equation (12) the computational cost (time complexity) for the energy difference is also $O(logn)$.

When $H\left({\overline{\sigma }}^{\prime }\right)>H\left(\overline{\sigma }\right)$, we can expect $P_{\sigma |G}(\sigma ={\overline{\sigma }}^{\prime })$ is far less than $P_{\sigma |G}(\sigma =\overline{\sigma })$. Roughly speaking, if  ${\sum }_{dist({\sigma }^{\prime },X)=1}exp(-\beta (H\left({\overline{\sigma }}^{\prime }\right)-H\left(X\right)))$ converges to zero, we can expect the probability of all other states differing from $S_k\left(X\right)$ converges to zero. On the contrary, if ${\sum }_{dist({\sigma }^{\prime },X)=1}exp(-\beta (H\left({\overline{\sigma }}^{\prime }\right)-H\left(X\right)))$ tends to infinity, then $P_{\sigma }\left(S_k\left(X\right)\right)$ converges to zero. This illustrates the idea behind the proof of Theorem 2. The rigorous proof can be found in Section 8.

5. Community Detection via Energy Minimization

Since ${\beta }^{*}$ is irrelevant with n, when $\gamma >b$, we can choose a sufficiently large $\beta$ such that $\beta >{\beta }^{*}$, then by Theorem 2, $\sigma \in S_k\left(X\right)$ almost surely, which implies that $P_{\sigma |G}(\sigma =X)$ has the largest probability for almost all graphs G sampled from the SBM. Therefore, instead of sampling from the Ising model, we can directly maximize the conditional probability to find the state with the largest probability. Equivalently, we can proceed by minimizing the energy term in Equation (7):

$${\widehat{X}}^{\prime }:=arg\underset{\overline{\sigma }\in W^n}{min}H\left(\overline{\sigma }\right)$$

(14)

In (14), we allow $\overline{\sigma }$ to take values from $W^n$. Since we know X has equal size $|\{v\in \left[n\right]:X_v=u\}|=\frac{n}{k}$ for each label u, another formulation is to restrict the search space to $W^{*}:=\{\sigma \in W^n\left|\right|\{v\in \left[n\right]:{\sigma }_v={\omega }^s\}|=\frac{n}{k},s=0,\dots ,k-1\}$. When $\sigma \in W^{*}$, minimizing $H\left(\sigma \right)$ is equivalent to:

$${\widehat{X}}^{″}:=arg\underset{\sigma \in W^{*}}{min}\sum _{\{i,j\}\notin E\left(G\right)}\delta ({\sigma }_i,{\sigma }_j)$$

(15)

where the minimal value is the minimum cut between different detected communities.

When ${\widehat{X}}^{″}\ne X$, we must have $dist({\widehat{X}}^{″},X)\ge 2$ to satisfy the constraint ${\widehat{X}}^{″}\in W^{*}$. Additionally, the estimator of ${\widehat{X}}^{″}$ is parameter-free whereas ${\widehat{X}}^{\prime }$ depends on $\gamma$. The extra parameter $\gamma$ in the expression of ${\widehat{X}}^{\prime }$ can be regarded as a kind of Lagrange multiplier for this integer programming problem. Thus, the optimization problem for ${\widehat{X}}^{\prime }$ is the relaxation of that for ${\widehat{X}}^{″}$ by introducing a penalized term and enlarging the searched space from $W^{*}$ to $W^n$.

When $\beta >\overline{\beta }$, $\tilde{g}\left(\beta \right)$ becomes a constant value. Therefore, we can get $n^{g\left(\overline{\beta }\right)/2}$ as the tightest error upper bound for the Ising estimator ${\widehat{X}}^{*}$ from Theorem 2. For the estimator ${\widehat{X}}^{\prime }$ and ${\widehat{X}}^{″}$, we can obtain a sharper error upper bound, which is summarized in the following theorem:

Theorem 3.

When $\sqrt{a}-\sqrt{b}>\sqrt{k}$, for sufficiently large n,

• If $\gamma >b$, $P_G({\widehat{X}}^{\prime }\notin S_k\left(X\right))\le (k-1+o\left(1\right))n^{g\left(\overline{\beta }\right)}$;
• $P_G({\widehat{X}}^{\prime \prime }\notin S_k\left(X\right))\le ({(k-1)}^2+o\left(1\right))n^{2g\left(\overline{\beta }\right)}$.

As $g\left(\overline{\beta }\right)<0$, $n^{2g\left(\overline{\beta }\right)}<n^{g\left(\overline{\beta }\right)}<n^{g\left(\overline{\beta }\right)/2}$, Theorem 3 implies that $P_e\left({\widehat{X}}^{″}\right)$ has the sharpest upper bound among the three estimators. This can be intuitively understood as the result of smaller search space. The proof technique of Theorem 3 is to consider the probability of events $H\left(X\right)>H\left(\overline{\sigma }\right)$ for $dist(\sigma ,X)\ge 1$. Then by union bound, these error probabilities can be summed up. We note that a loose bound $n^{g\left(\overline{\beta }\right)/4}$ was obtained in [7] for the estimator ${\widehat{X}}^{″}$ when $k=2$. For a general case, since $\tilde{g}\left(\beta \right)=1-\frac{{(\sqrt{a}-\sqrt{b})}^2}{k}$, Theorem 3 implies that exact recovery is possible using ${\widehat{X}}^{\prime }$ as long as $\sqrt{a}-\sqrt{b}>\sqrt{k}$ is satisfied.

Estimator ${\widehat{X}}^{\prime }$ has one parameter, $\gamma$. When $\gamma$ takes different values, ${\widehat{X}}^{\prime }$ is equivalent with maximum likelihood or maximum modularity in the asymptotic case. The following analysis shows their relationship intuitively.

The maximum likelihood estimator is obtained by maximizing the log-likelihood function. From (2), this function can be written as:

$$log{P}_G\left(Z\right|X=\sigma )=-log\frac{a}{b}·H\left(\sigma \right)+C$$

where the parameter $\gamma$ in $H\left(\sigma \right)$ satisfies $\gamma \frac{logn}{n}=\frac{1}{log(a/b)}(log(1-\frac{alogn}{n})-log(1-\frac{blogn}{n}))$ and C is a constant irrelevant with $\sigma$. When n is sufficiently large, we have $\gamma \to {\gamma }_{ML}:=\frac{a-b}{log(a/b)}$. That is, the maximum likelihood estimator is equivalent to ${\widehat{X}}^{\prime }$ when $\gamma ={\gamma }_{ML}$ asymptotically.

The maximum modularity estimator is obtained by maximizing the modularity of a graph [23], which is defined by:

$$\begin{array}{cc}\hfill Q& =\frac{1}{2\left|E\right|}\sum _{ij}(A_{ij}-\frac{d_i{d}_j}{2\left|E\right|})\delta (C_i,C_j)\hfill \end{array}$$

(16)

For the i-th node, $d_i$ is its degree and $C_i$ is its community belonging. A is the adjacency matrix. Up to a scaling factor, the modularity Q can be re-written using the label vector $\sigma$ as:

$$\begin{array}{cc}\hfill Q\left(\sigma \right)=& -\sum _{\{i,j\}\notin E\left(G\right)}\frac{d_i{d}_j}{2\left|E\right|}\delta ({\sigma }_i,{\sigma }_j)\hfill \\ \hfill & +\sum _{\{i,j\}\in E\left(G\right)}(1-\frac{d_i{d}_j}{2\left|E\right|})\delta ({\sigma }_i,{\sigma }_j)\hfill \end{array}$$

(17)

From (17), we can see that $Q\left(\sigma \right)\to -H\left(\sigma \right)$ with $\gamma ={\gamma }_{MQ}=\frac{a+b}{2}$ as $n\to \infty$. Indeed, we have $d_i\sim \frac{(a+b)logn}{2},\left|E\right|\sim \frac{1}{2}n{d}_i$. Therefore, we have $\frac{d_i{d}_j}{2\left|E\right|}\to {\gamma }_{MQ}\frac{logn}{n}$. That is, the maximum modularity estimator is equivalent with ${\widehat{X}}^{\prime }$ when $\gamma ={\gamma }_{MQ}$ asymptotically.

Using $a>b$ and the inequality $x-1>logx>2\frac{x-1}{x+1}$ for $x>1$ we can verify that ${\gamma }_{MQ}>{\gamma }_{ML}>b$. That is, both the maximum likelihood and the maximum modularity estimator satisfy the exact recovery conditions $\gamma >b$ in Theorem 3.

6. Community Detection Based on Metropolis Sampling

From Theorem 2, if we could sample from the Ising model, then with large probability, the sample is aligned with X. However, exact sampling is difficult when n is very large since the cardinality of the state space increases in the rate of $k^n$. Therefore, some approximation is necessary, and the most common way to generate an Ising sample is using Metropolis sampling [14]. Empirically speaking, starting from a random state, the Metropolis algorithm updates the state by randomly selecting one position to flip its state at each iteration step. Then after some initial burning time, the generated samples can be regarded as sampling from the Ising model.

The theoretical guarantee of Metropolis sampling is based on the Markov chain. Under some general conditions, Metropolis samples converge to the steady state of the Markov chain, and the steady state follows the probability distribution to be approximated. For the Ising model, there are many previous works which have shown the convergence of Metropolis sampling [24].

For our specific Ising model and energy term in Equation (7), the pseudo code of our algorithm is summarized in Algorithm 1. This algorithm requires that the number of the communities k is known and the strength ratio parameter $\gamma$ is given. We should choose $\gamma >b$ where b is estimated by $\widehat{b}$ in Theorem 1. The iteration time N should also be specified in advance.

Algorithm 1 Metropolis sampling algorithm for SBM.

Inputs: the graph G, inverse temperature $\beta$, the strength ratio parameter $\gamma$ Output: $\widehat{X}=\overline{\sigma }$ 1: random initialize $\overline{\sigma }\in W^n$ 2: for$i=1,2,\dots ,N$do 3:  propose a new state ${\overline{\sigma }}^{\prime }$ according to Lemma 1 where $s,r$ are randomly chosen 4:  compute $\Delta H(r,s)=H\left({\overline{\sigma }}^{\prime }\right)-H\left(\overline{\sigma }\right)$ using (12) 5:  if $\Delta H(r,s)<0$ then 6:   ${\sigma }_r\leftarrow w^s·{\sigma }_r$ 7:  else 8:   with probability $exp(-\beta \Delta H(r,s\left)\right)$ such that ${\sigma }_r\leftarrow w^s·{\sigma }_r$ 9:  end if 10: end for

The computation of $\Delta H(r,s)$ needs $O(logn)$ time from Lemma 1. For some special Ising model, it needs to take $N=O(nlogn)$ to generate the sample for good approximation [25]. For our model, it is unknown whether $O(nlogn)$ is sufficient, and we empirically chose $N=O\left(n^2\right)$ in numerical experiments. Then the time complexity of Algorithm 1 is $O(n^{2}logn)$.

In the remaining part of this section, we present experiments conducted to verify our theoretical results. Firstly, we considered several combinations of $(a,b,k)$ and obtained the estimator $(\widehat{a},\widehat{b})$ by Theorem 1. Using the empirical mean squared error (MSE) $\frac{1}{m}{\sum }_{i=1}^m{(\widehat{a}-a)}^2+{(\widehat{b}-b)}^2$ as the criterion and choosing $m=1000$, the result is shown in Figure 2a. As we can see, as n increases, the MSE decreases polynomially fast. Therefore, the convergence of $\widehat{a}\to a$ and $\widehat{b}\to b$ was verified.

Secondly, using Metropolis sampling, we conducted a moderate simulation to verify Theorem 2 for the case $\gamma >b$. We chose $n=9000,k=2$, and the empirical accuracy was computed by $P_e=\frac{1}{m_1{m}_2}{\sum }_{i=1}^{m_1}{\sum }_{j=1}^{m_2}\mathbb{1}[{\widehat{X}}^{*}=\pm X]$. In this formula, $m_1$ is the number of times the random graph was generated by the SBM, whereas $m_2$ is the number of times consecutive samples were generated by Algorithm 1 for a given graph. We chose $m_1=2100,m_2=6000$, which is fairly large and can achieve a good approximation of $P_e\left({\widehat{X}}^{*}\right)$ by the law of large numbers. The result is shown in Figure 2b.

The vertical red line ($\beta ={\beta }^{*}=0.198$), computed from (10), represents the phase transition threshold. The point $(0.199,\frac{1}{2})$ in the figure can be regarded as the empirical phase transition threshold, whose first coordinate is close to ${\beta }^{*}$. The green line $(\beta ,n^{g\left(\beta \right)/2})$ is the theoretical lower bound of accuracy for $\beta >{\beta }^{*}$, and the purple line $(\beta ,n^{-g\left(\beta \right)})$ is the theoretical upper bound of accuracy for $\beta <{\beta }^{*}$. It can be expected that as n becomes larger, the empirical accuracy curve (blue line in the figure) will approach the step function, which jumps from 0 to 1 at $\beta ={\beta }^{*}$.

7. Conclusions

In this paper, we presented one convergent estimator (in Theorem 1) to infer the parameters of the SBM and analyzed three label estimators to detect communities of the SBM. We gave the exact recovery error upper bound for all label estimators (in Theorems 2 and 3) and studied their relationships. By introducing the Ising model, our work makes a new path to study the exact recovery problem for the SBM. More theoretical and empirical work will be done in the future, such as convergence analyses on modularity (in Equation (17)), the necessary iteration time (in Algorithm 1) for Metropolis sampling, and so on.

8. Proof of Main Theorems

8.1. Proof of Theorem 1

Lemma 2.

Consider an Erdős–Rényi random graph G with n nodes, in which edges are placed independently with probability p [26]. Suppose $p=\frac{alogn}{n}$, the number of edges is denoted by $\left|E\right|$ while the number of triangles is T. Then $\frac{\left|E\right|}{nlogn}\to \frac{a}{2}$ and $\frac{T}{{log}^{3}n}\to \frac{a^3}{6}$ in probability.

Proof. 

Let $X_{ij}$ represent a Bernoulli random variable with parameter p. Then $\left|E\right|={\sum }_{i,j}{X}_{ij}$, $X_{ij}$ are i.i.d. $\mathbb{E}\left[T\left(G\right)\right]=\frac{n(n-1)}{2}p=\frac{(n-1)logn}{2}a$ and $Var\left[\right|E\left|\right]=\frac{n(n-1)}{2}p(1-p)<a\frac{(n-1)logn}{2}$. Then by Chebyshev’s inequality,

$$\begin{array}{cc}\hfill P\left(\right|\frac{\left|E\right|}{nlogn}-\frac{a}{2}\frac{n-1}{n}|>ϵ)& \le \frac{Var\left[\right|E|/(nlogn\left)\right]}{{ϵ}^2}\hfill \\ \hfill & <\frac{a(n-1)}{2n^2{ϵ}^{2}logn}\hfill \end{array}$$

For a given $ϵ$, when n is sufficiently large,

$$\begin{array}{cc}\hfill P\left(\right|\frac{\left|E\right|}{nlogn}-\frac{a}{2}|>ϵ)& <P\left(\right|\frac{\left|E\right|}{nlogn}-\frac{a}{2}\frac{n-1}{n}|>2ϵ)\hfill \\ \hfill & \le \frac{n-1}{8n^2{ϵ}^{2}logn}\hfill \end{array}$$

Therefore, by the definition of convergence in probability, we have $\frac{\left|E\right|}{nlogn}\to \frac{a}{2}$ as $n\to \infty$.

Let $X_{ijk}$ represents a Bernoulli random variable with parameter $p^3$. Then $T={\sum }_{i,j,k}{X}_{ijk}$. It is easy to compute that $\mathbb{E}\left[T\right]=\left(\genfrac{}{}{0pt}{}{n}{3}\right)p^3$. Since $X_{ijk}$ are not independent, the variance of T needs careful calculation. From [27] we know that:

$$\begin{array}{cc}\hfill Var\left[T\right]& =\left(\genfrac{}{}{0pt}{}{n}{3}\right)p^3+12\left(\genfrac{}{}{0pt}{}{n}{4}\right)p^5+30\left(\genfrac{}{}{0pt}{}{n}{5}\right)p^6+20\left(\genfrac{}{}{0pt}{}{n}{6}\right)p^6\hfill \\ \hfill & -{\left(\genfrac{}{}{0pt}{}{n}{3}\right)}^2{p}^6=O\left({log}^{3}n\right)\hfill \end{array}$$

Therefore by Chebyshev’s inequality,

$$\begin{array}{cc}\hfill P\left(\right|\frac{T}{{log}^{3}n}-\frac{a^3}{6}\frac{(n-1)(n-2)}{n^2}|>ϵ)& \le \frac{Var[T/{log}^{3}n]}{{ϵ}^2}\hfill \\ \hfill & =\frac{1}{{ϵ}^2}O\left(\frac{1}{{log}^{3}n}\right)\hfill \end{array}$$

Hence, $\frac{T}{{log}^{3}n}\to \frac{a^3}{6}$. □

The convergence of $\left|E\right|$ in the Erdős–Rényi graph can be extended directly to the SBM since the existence of each edge is independent. However, for T, it is a little tricky since the existences of each triangle are mutually dependent. The following two lemmas give the formula for the variance of inter-community triangles in the SBM.

Lemma 3.

Consider a two-community SBM $(2n,p,q)$ and count the number of triangles T, which has a node in $S_1$ and an edge in $S_2$. Then the variance of T is:

$$\begin{array}{cc}\hfill Var\left[T\right]& =\frac{n^2(n-1)}{2}{q}^{2}p+n^2(n-1)(n-2)p^2{q}^3\hfill \\ \hfill & +\frac{n^2{(n-1)}^2}{2}{q}^{4}p-\frac{n^2(n-1)(3n-4)}{2}{q}^4{p}^2\hfill \end{array}$$

(18)

Lemma 4.

Consider a three-community SBM $(3n,p,q)$ and count the number of triangles T, which has a node in $S_1$, one node in $S_2$, and one node in $S_3$. Then the variance of T is:

$$Var\left[T\right]=n^3{q}^3+3n^3(n-1)q^4+3n^3{(n-1)}^2{q}^5-n^3(3n^2-3n+1)q^6$$

The proof of the above two lemmas uses some counting techniques and is similar to that in [27], and we omit it here.

Lemma 5.

For a SBM $(n,k,p,q)$ where $p=\frac{alogn}{n},q=\frac{blogn}{n}$. The number of triangles is T. Then $\frac{T}{{(logn)}^3}$ converges to $\frac{1}{k^2}(\frac{a^3}{6}+\frac{k-1}{2}a{b}^2+(k-1)(k-2)\frac{b^3}{6})$ in probability as $n\to \infty$.

Proof. 

We split T into three parts: the first is the number of triangles within community i, $T_i$. There are k terms of $T_i$. The second is the number of triangles that have one node in community i and one edge in community j, $T_{ij}$. There are $k(k-1)$ terms of $T_{ij}$. The third is the number of triangles that have one node in community i, one node in community j and one node in community k.

We only need to show that:

$$\begin{array}{cc}\hfill \frac{T_i}{{log}^{3}n}& \to \frac{{(a/k)}^3}{6}\hfill \end{array}$$

(19)

$$\begin{array}{cc}\hfill \frac{T_{ij}}{{log}^{3}n}& \to \frac{1}{2}(a/k){(b/k)}^2\hfill \end{array}$$

(20)

$$\begin{array}{cc}\hfill \frac{T_{ijk}}{{log}^{3}n}& \to {(b/k)}^3\hfill \end{array}$$

(21)

The convergence of $\frac{T_i}{{log}^{3}n}$ comes from Lemma 2. For $T_{ij}$ we use the conclusion from Lemma 3. We replace n with $n/k$, $p=a\frac{logn}{n}$, and $q=b\frac{logn}{n}$ in Equation (18). $Var\left[T_{ij}\right]\sim \frac{a{b}^2}{2k^3}{log}^{3}n$. Since the expectation of $\frac{T_{ij}}{{log}^{3}n}$ is $(n/k)\left(\genfrac{}{}{0pt}{}{n/k}{2}\right)p{q}^2/\left({log}^{3}n\right)=\frac{n-1}{2n}\frac{a{b}^2}{k^3}$, by Chebyshev’s inequality we can show that:

$$\begin{array}{cc}\hfill P\left(\right|\frac{T_{ij}}{{log}^{3}n}-\frac{n-1}{2n}\frac{a{b}^2}{k^3}|>ϵ)& \le \frac{Var[T_{ij}/{log}^{3}n]}{{ϵ}^2}\hfill \\ \hfill & =\frac{1}{{ϵ}^2}O\left(\frac{1}{{log}^{3}n}\right)\hfill \end{array}$$

Therefore, $\frac{T_{ij}}{{log}^{3}n}$ converges to $\frac{1}{2}(a/k){(b/k)}^2$.

To prove $\frac{T_{ijk}}{{log}^{3}n}\to {(b/k)}^3$, from Lemma 4 we can get $Var\left[T_{ijk}\right]=O\left({log}^{5}n\right)$:

$$P\left(\right|\frac{T_{ijk}}{{log}^{3}n}-\frac{b^3}{k^3}|>ϵ)\le \frac{Var[T_{ijk}/{log}^{3}n]}{{ϵ}^2}=\frac{1}{{ϵ}^2}O\left(\frac{1}{logn}\right)$$

□

Proof of Theorem 1.

Let $e_1^{*}=\frac{a+(k-1)b}{2k}$, $k^2{e}_2^{*}=\frac{a^3}{6}+\frac{k-1}{2}a{b}^2+(k-1)(k-2)\frac{b^3}{6}$ and $e_1=\frac{T_1}{nlogn},e_2=\frac{T_2}{{log}^{3}n}$. From Lemma 2, $e_1\to e_1^{*}$. From Lemma 5, $e_2\to e_2^{*}$ as $n\to \infty$. Using $x=2k{e}_1-(k-1)y$, we can get:

$$g\left(y\right):=(k-1)(y^3-6e_1{y}^2+12e_1^{2}y)+6e_2-8k{e}_1^3=0$$

(22)

This equation has a unique real root since $g\left(y\right)$ is increasing on $\mathbb{R}$: $g^{\prime }\left(y\right)=3(k-1){(y-2e_1)}^2\ge 0$. Next we show that the root lies within $(0,2e_1)$.

$$\begin{array}{cc}\hfill & \underset{n\to \infty }{lim}g\left(0\right)=6e_2^{*}-8k{\left(e_1^{*}\right)}^3=-\frac{3}{k^2}(k-1)(k-2)a{b}^2\hfill \\ \hfill & -\frac{3(k-1)}{k^2}{a}^{2}b-\frac{k-1}{k^2}((k-2)-{(k-1)}^2)b^3<0\hfill \\ \hfill & \underset{n\to \infty }{lim}g\left(2e_1\right)=6e_2^{*}-8{\left(e_1^{*}\right)}^3=\frac{(k-1){(a-b)}^3}{k^3}>0\hfill \end{array}$$

Therefore, we can get a unique solution y within $(0,2e_1)$. Since $(a,b)$ is a solution for the equation array, the conclusion follows.

By taking expectation on both sizes of Equations (4) and (5) we can show $\mathbb{E}\left[\widehat{a}\right]=a,\mathbb{E}\left[\widehat{b}\right]=b$. By the continuous property of $g\left(y\right)$, $\widehat{b}\to b$ and $\widehat{a}\to a$ follows similarly. □

8.2. Proof of Theorem 2

Proof of Lemma 1.

First we rewrite the energy term in (7) as:

$$H\left(\overline{\sigma }\right)=\gamma \frac{logn}{n}\sum _{i<j}\delta ({\overline{\sigma }}_i,{\overline{\sigma }}_j)-(1+\gamma \frac{logn}{n})\sum _{\{i,j\}\in E\left(G\right)}\delta ({\overline{\sigma }}_i,{\overline{\sigma }}_j)$$

Then calculating the energy difference term by:

$$\begin{array}{cc}\hfill H\left({\overline{\sigma }}^{\prime }\right)-H\left(\overline{\sigma }\right)& =(1+\gamma \frac{logn}{n})\hfill \\ \hfill & ·\sum _{i\in N_r\left(G\right)}(\delta ({\overline{\sigma }}_r,{\overline{\sigma }}_i)-\delta ({\omega }^s·{\overline{\sigma }}_r,{\overline{\sigma }}_i))\hfill \\ \hfill & +\gamma \frac{logn}{n}\sum _{i\ne r}(\delta ({\omega }^s·{\overline{\sigma }}_r,{\overline{\sigma }}_i)-\delta ({\overline{\sigma }}_r,{\overline{\sigma }}_i))\hfill \\ \hfill & =(1+\gamma \frac{logn}{n})\sum _{i\in N_r\left(G\right)}{J}_s({\overline{\sigma }}_r,{\overline{\sigma }}_i)\hfill \\ \hfill & +\gamma \frac{logn}{n}\sum _{i=1}^n(\delta ({\omega }^s·{\overline{\sigma }}_r,{\overline{\sigma }}_i)-\delta ({\overline{\sigma }}_r,{\overline{\sigma }}_i)+1)\hfill \\ \hfill & =(1+\gamma \frac{logn}{n})\sum _{i\in N_r\left(G\right)}{J}_s({\overline{\sigma }}_r,{\overline{\sigma }}_i)\hfill \\ \hfill & +\gamma \frac{logn}{n}(m({\omega }^s·{\overline{\sigma }}_r)-m\left({\overline{\sigma }}_r\right)+1)\hfill \end{array}$$

□

Before diving into the technical proof of Theorem 2, we need to introduce some extra notations. When $\overline{\sigma }$ differs from X only at position r, taking $\overline{\sigma }=X$ in Lemma 1, we have:

$$H\left({\overline{\sigma }}^{\prime }\right)-H\left(\overline{\sigma }\right)=(1+\gamma \frac{logn}{n})(A_r^0-A_r^s)+\gamma \frac{logn}{n}$$

(23)

where $A_r^s$ is defined as $A_r^s=\left|\{j\in \left[n\right]\backslash \left\{r\right\}:\{j,r\}\in E\left(G\right),X_j={\omega }^s·X_r\}\right|$. Since the existence of each edge in G is independent, $A_r^s\sim \mathrm{Bernoulli}(\frac{n}{k},\frac{blogn}{n})$ for $s\ne 0$ and $A_0^s\sim \mathrm{Bernoulli}(\frac{n}{k}-1,\frac{alogn}{n})$.

For the general case, we can write:

$$H\left(\overline{\sigma }\right)-H\left(X\right)=(1+\gamma \frac{logn}{n})[A_{\overline{\sigma }}-B_{\overline{\sigma }}]+\gamma \frac{logn}{n}{N}_{\overline{\sigma }}$$

(24)

in which we use $A_{\overline{\sigma }}$ or $B_{\overline{\sigma }}$ to represent the binomial random variable with parameter $\frac{alogn}{n}$ or $\frac{blogn}{n}$, respectively, and $N_{\overline{\sigma }}$ is a deterministic positive number depending on $\overline{\sigma }$ but irrelevant with the graph structure. The following lemma gives the expression of $A_{\overline{\sigma }},B_{\overline{\sigma }}$ and $N_{\overline{\sigma }}$:

Lemma 6.

For SSBM $(n,k,p,q)$, we assume $\overline{\sigma }$ differs from the ground truth label vector X in the $\left|\mathcal{I}\right|:=dist(\overline{\sigma },X)$ coordinate. Let $I_{ij}=|\{r\in \left[n\right]|X_r=w^i,{\sigma }_r=w^j\}$ for $i\ne j$ and $I_{ii}=0$. We further denote the row sum as $I_i={\sum }_{j=0}^{k-1}{I}_{ij}$ and the column sum as $I_i^{\prime }={\sum }_{j=0}^{k-1}{I}_{ji}$. Then:

$$\begin{array}{cc}\hfill N_{\overline{\sigma }}& =\frac{1}{2}\sum _{i=0}^{k-1}{(I_i-I_i^{\prime })}^2\hfill \end{array}$$

(25)

$$\begin{array}{cc}\hfill B_{\overline{\sigma }}& \sim Bernoulli\left(\frac{n}{k}\right|\mathcal{I}|+\frac{1}{2}\sum _{i=0}^{k-1}(-2I_i^{\prime }{I}_i+I_i^{\prime 2}-\sum _{j=0}^{k-1}{I}_{ji}^2),q)\hfill \end{array}$$

(26)

$$\begin{array}{cc}\hfill A_{\overline{\sigma }}& \sim Bernoulli\left(\frac{n}{k}\right|\mathcal{I}|-\frac{1}{2}\sum _{i=0}^{k-1}(I_i^2+\sum _{j=0}^{k-1}{I}_{ij}^2),p)\hfill \end{array}$$

(27)

The proof of Lemma 6 is mainly composed of careful counting techniques, and we omit it here. When $\left|\mathcal{I}\right|$ is small compared to n, we have the following Lemma, which is an extension of Proposition 6 in [12].

Lemma 7.

For $t\in [\frac{1}{k}(b-a),0]$ and $\left|\mathcal{I}\right|\le n/\sqrt{logn}$

$$\begin{array}{cc}\hfill & P_G(B_{\overline{\sigma }}-A_{\overline{\sigma }}\ge t\left|\mathcal{I}\right|logn)\hfill \\ \hfill \le & exp\left(\left|\mathcal{I}\right|logn\left(f_{\beta }\left(t\right)-\beta t-1+O\left(\frac{1}{\sqrt{logn}}\right)\right)\right)\hfill \end{array}$$

(28)

where $f_{\beta }\left(t\right)={min}_{s\ge 0}(g\left(s\right)-st)+\beta t\le \tilde{g}\left(\beta \right)$.

Corresponding to the three cases of Theorem 2, we use three non-trivial lemmas to establish the properties of the Ising model.

Lemma 8.

Let $\gamma >b$. When $dist(\overline{\sigma },X)\ge \frac{n}{\sqrt{logn}}$ and $D(\overline{\sigma },X)$, the event $P_{\sigma |G}(\sigma =\overline{\sigma })>exp(-Cn)P_{\sigma |G}(\sigma =X)$ happens with a probability (with respect to SSBM) less than $exp(-\tau (\alpha ,\beta )n\sqrt{logn})$, where C is an arbitrary constant and $\tau (\alpha ,\beta )$ is a positive number.

Proof. 

We denote the event $P_{\sigma |G}(\sigma =\overline{\sigma })>exp(-Cn)P_{\sigma |G}(\sigma =X)$ as $\tilde{D}(\overline{\sigma },C)$. By Equation (24), $\tilde{D}(\overline{\sigma },C)$ is equivalent to:

$$(1+\frac{\gamma logn}{n})[B_{\overline{\sigma }}-A_{\overline{\sigma }}]>\frac{\gamma logn}{n}{N}_{\overline{\sigma }}-\frac{C}{\beta }n$$

(29)

We claim that $\overline{\sigma }$ must satisfy at least one of the following two conditions:

• $\exists i\ne j$ s.t. $\frac{1}{k(k-1)}\frac{n}{\sqrt{logn}}\le I_{ij}\le \frac{n}{k}-\frac{1}{k(k-1)}\frac{n}{\sqrt{logn}}$
• $\exists i\ne j$ s.t. $I_{ij}>\frac{n}{k}-\frac{1}{k(k-1)}\frac{n}{\sqrt{logn}}$ and $I_{ji}<\frac{1}{k(k-1)}\frac{n}{\sqrt{logn}}$

If neither of the above two condition holds, then from condition 1 we have $I_{ij}<\frac{1}{k(k-1)}\frac{n}{\sqrt{logn}}$ or $I_{ij}>\frac{n}{k}-\frac{1}{k(k-1)}\frac{n}{\sqrt{logn}}$ for any $0\le i,j\le k-1$. Since ${\sum }_{i,j}{I}_{ij}=\left|\mathcal{I}\right|\ge \frac{n}{\sqrt{logn}}$, there exists $i,j$ such that $I_{ij}>\frac{n}{k}-\frac{1}{k(k-1)}\frac{n}{\sqrt{logn}}$. Under such conditions, we also assume $I_{ji}>\frac{n}{k}-\frac{1}{k(k-1)}\frac{n}{\sqrt{logn}}$. Let $X^{\prime }$ be the vector that exchanges the value of $w^i$ with $w^j$ in X. We consider:

$$\begin{array}{cc}\hfill dist(\overline{\sigma },X^{\prime })& -dist(\overline{\sigma },X)=\left|\{r\in \left[n\right]|X_r=w^i,{\overline{\sigma }}_r\ne w^j\}\right|\hfill \\ \hfill & +\left|\right\{r\in \left[n\right]|X_r=w^j,{\overline{\sigma }}_r\ne w^i\}|\hfill \\ \hfill & -\left|\right\{r\in \left[n\right]|X_r=w^i,{\overline{\sigma }}_r\ne w^i\}|\hfill \\ \hfill & -\left|\right\{r\in \left[n\right]|X_r=w^j,{\overline{\sigma }}_r\ne w^j\}|\hfill \\ \hfill & =\frac{n}{k}-I_{ij}+\frac{n}{k}-I_{ji}-I_i-I_j\hfill \\ \hfill & <\frac{2}{k(k-1)}\frac{n}{\sqrt{logn}}-I_i-I_j<0\hfill \end{array}$$

(30)

which contracts with the fact that $\overline{\sigma }$ is nearest to X. Therefore, we should have $I_{ji}<\frac{1}{k(k-1)}\frac{n}{\sqrt{logn}}$. Now the $(i,j)$ pair satisfies condition 2, which contracts with the fact that $\overline{\sigma }$ satisfies neither of the two conditions.

Under condition 1, we can get a lower bound on $|A_{\overline{\sigma }}|$ from Equation (27). Let $I_{ij}^{\prime }=I_{ij}$ for $i\ne j$ and $I_{ii}^{\prime }=\frac{n}{k}-I_i$. Then we can simplify $|A_{\overline{\sigma }}|$ as:

$$\begin{array}{cc}\hfill |A_{\overline{\sigma }}|& =\frac{n}{k}\left|\mathcal{I}\right|-\frac{1}{2}\sum _{i=0}^{k-1}(I_i^2+\sum _{j=0}^{k-1}{I}_{ij}^2)\hfill \\ \hfill & =\frac{n^2}{2k}-\frac{1}{2}\sum _{i=0}^{k-1}\sum _{j=0}^{k-1}{I}_{ij}^{\prime 2}\hfill \end{array}$$

We further have ${\sum }_{i=0}^{k-1}{\sum }_{j=0}^{k-1}{I^{\prime }}_{ij}^2\le (k-1)\frac{n^2}{k^2}+{(\frac{n}{k}-I_{ij})}^2+I_{ij}^2$ where $I_{ij}$ satisfies condition 1. Therefore, ${\sum }_{i=0}^{k-1}{\sum }_{j=0}^{k-1}{I^{\prime }}_{ij}^2\le (k-1)\frac{n^2}{k^2}+{\left(\frac{1}{k(k-1)}\frac{n}{\sqrt{logn}}\right)}^2+{(\frac{n}{k}-\frac{1}{k(k-1)}\frac{n}{\sqrt{logn}})}^2=\frac{n^2}{k}-\frac{2n^2}{k^2(k-1)\sqrt{logn}}(1+o\left(1\right))$. As a result,

$$A_{\overline{\sigma }}\ge \frac{n^2}{k^2(k-1)\sqrt{logn}}(1+o\left(1\right))$$

(31)

Under condition 2, we can get a lower bound on $N_{\overline{\sigma }}$. Since $dist(\overline{\sigma },X^{\prime })-dist(\overline{\sigma },X)\ge 0$, from (30) we have $I_{ij}+I_{ji}+I_i+I_j\le \frac{2n}{k}$. Since $I_i\ge I_{ij}>\frac{n}{k}-\frac{1}{k(k-1)}\frac{n}{\sqrt{logn}}$, we have $I_j\le \frac{2}{k(k-1)}\frac{n}{\sqrt{logn}}$. Now consider $I_j^{\prime }-I_j\ge \frac{n}{k}-\frac{3}{k(k-1)}\frac{n}{\sqrt{logn}}$. From (25): $N_{\overline{\sigma }}\ge \frac{1}{2}{(\frac{n}{k}-\frac{3}{k(k-1)}\frac{n}{\sqrt{logn}})}^2=\frac{n^2}{2k^2}(1+o\left(1\right))$.

Now we use the Chernoff inequality to bound Equation (29); we can omit $\frac{\gamma logn}{n}$ on the left-hand side since it is far smaller than 1. Let $Z\sim \mathrm{Bernoulli}\left(\frac{alogn}{n}\right),Z^{\prime }\sim \mathrm{Bernoulli}\left(\frac{blogn}{n}\right)$, then:

$$\begin{array}{cc}\hfill & P_G\left(\tilde{D}(\overline{\sigma },C)\right)\le {\left(\mathbb{E}[exp\left(sZ\right)]\right)}^{|B_{\overline{\sigma }}|}{\left(\mathbb{E}[exp(-s{Z}^{\prime })]\right)}^{|A_{\overline{\sigma }}|}\hfill \\ \hfill & ·exp(-s(\frac{\gamma logn}{n}{N}_{\overline{\sigma }}-\frac{C}{\beta }n))\hfill \\ \hfill & \le exp\left(\right|B_{\overline{\sigma }}|\frac{blogn}{n}(e^s-1)+\left|A_{\overline{\sigma }}\right|\frac{alogn}{n}(e^{-s}-1)\hfill \\ \hfill & -s(\frac{\gamma logn}{n}{N}_{\overline{\sigma }}-\frac{C}{\beta }n))\hfill \end{array}$$

Using $|B_{\overline{\sigma }}|=N_{\overline{\sigma }}+\left|A_{\overline{\sigma }}\right|$ we can further simplify the exponential term as:

$$\frac{logn}{n}\left[\right|A_{\overline{\sigma }}|(b(e^s-1)+a(e^{-s}-1))+N_{\overline{\sigma }}(b(e^s-1)-\gamma s)]+s\frac{C}{\beta }n$$

Now we investigate the function $g_1\left(s\right)=b(e^s-1)+a(e^{-s}-1)$ and $g_2\left(s\right)=b(e^s-1)-\gamma s$. Both functions take zero values at $s=0$ and $g_1^{\prime }\left(s\right)=(b{e}^s-a{e}^{-s}),g_2^{\prime }\left(s\right)=b{e}^s-\gamma$. Therefore, $g_1^{\prime }\left(0\right)=b-a<0,g_2^{\prime }\left(0\right)=b-\gamma <0$ and we can choose $s^{*}>0$ such that $g_1\left(s^{*}\right)<0,g_2\left(s^{*}\right)<0$. To compensate the influence of the term $sCn/\beta$ we only need to make sure that the order of $\frac{logn}{n}min\left\{\right|A_{\overline{\sigma }}|,N_{\overline{\sigma }}\}$ is larger than n. This requirement is satisfied since either $|A_{\overline{\sigma }}|\ge \frac{n^2}{k^2(k-1)\sqrt{logn}}(1+o\left(1\right))$ or $N_{\overline{\sigma }}\ge \frac{n^2}{2k^2}(1+o\left(1\right))$. □

Lemma 9.

If $\gamma >b$, $\beta >{\beta }^{*}$, For $1\le r\le \frac{n}{\sqrt{logn}}$ and $\forall ϵ>0$, there is a set ${\mathcal{G}}^{\left(r\right)}$ such that:

$$P_G\left({\mathcal{G}}_n^{\left(r\right)}\right)\ge 1-n^{r(\tilde{g}\left(\beta \right)/2+ϵ)}$$

(32)

and for every $G\in {\mathcal{G}}_n^{\left(r\right)}$,

$$\frac{P_{\sigma |G}(dist(\sigma ,X)=r|D(\sigma ,X))}{P_{\sigma |G}(\sigma =X|D(\sigma ,X))}<n^{r\tilde{g}\left(\beta \right)/2}$$

(33)

For $r>\frac{n}{\sqrt{logn}}$, there is a set ${\mathcal{G}}^{\left(r\right)}$ such that:

$$P(G\in {\mathcal{G}}_n^{\left(r\right)})\ge 1-e^{-n}$$

(34)

and for every $G\in {\mathcal{G}}_n^{\left(r\right)}$,

$$\frac{P_{\sigma |G}(dist(\sigma ,X)=r|D(\sigma ,X))}{P_{\sigma |G}(\sigma =X|D(\sigma ,X))}<e^{-n}$$

(35)

Proof. 

We distinguish the discussion between two cases: $r\le \frac{n}{\sqrt{logn}}$ and $r>\frac{n}{\sqrt{logn}}$.

When $r\le \frac{n}{\sqrt{logn}}$, we can show that $dist(\sigma ,X)=r$ implies $D(\sigma ,X)$ by using the triangle inequality of dist. For $f\in S_k\backslash \left\{\mathrm{id}\right\}$, where $\mathrm{id}$ is the identity mapping, we have:

$$\frac{2n}{k}\le dist(f\left(X\right),X)\le dist(\sigma ,f\left(X\right))+dist(\sigma ,X)$$

Therefore, $dist(\sigma ,f\left(X\right))\ge \frac{2n}{k}-\frac{n}{\sqrt{logn}}\ge dist(\sigma ,X)$ and Equation (33) is equivalent with:

$$\frac{P_{\sigma |G}(dist(\sigma ,X)=r)}{P_{\sigma |G}(\sigma =X)}<n^{r\tilde{g}\left(\beta \right)/2}$$

(36)

The left-hand side can be written as:

$$\begin{array}{cc}\hfill \frac{P_{\sigma |G}(dist(\sigma ,X)=r)}{P_{\sigma |G}(\sigma =X)}& =\sum _{dist(\overline{\sigma },X)=r}exp(-\beta (H\left(\overline{\sigma }\right)-H\left(X\right)))\hfill \\ \hfill \mathrm{by}\left(24\right)& \le \sum _{dist(\overline{\sigma },X)=r}exp\left({\beta }_n(B_{\overline{\sigma }}-A_{\overline{\sigma }})\right)\hfill \end{array}$$

where ${\beta }_n=\beta (1+\gamma \frac{logn}{n})$.

Define ${\Xi }_n\left(r\right):={\sum }_{dist(\overline{\sigma },X)=r}exp\left({\beta }_n(B_{\overline{\sigma }}-A_{\overline{\sigma }})\right)$ and we only need to show that:

$$P_G({\Xi }_n\left(r\right)\ge n^{r\tilde{g}\left(\beta \right)/2})\le n^{r(\tilde{g}\left(\beta \right)/2+ϵ)}$$

(37)

Define the event ${\Lambda }_n(G,r):=\{B_{\overline{\sigma }}-A_{\overline{\sigma }}<0,\forall \overline{\sigma }s.t.dist(\overline{\sigma },X)=r\}$, and we proceed as follows:

$$\begin{array}{cc}\hfill P_G({\Xi }_n\left(r\right)\ge n^{r\tilde{g}\left(\beta \right)/2})& \le P_G\left({\Lambda }_n{(G,r)}^c\right)\hfill \\ \hfill & +P_G({\Xi }_n\left(r\right)\ge n^{r\tilde{g}\left(\beta \right)/2}|{\Lambda }_n(G,r))\hfill \end{array}$$

For the first term, since $|\left\{\overline{\sigma }\right|dist(\overline{\sigma },X)=r\}|={(k-1)}^r{n}^r$, by Lemma 7, $P_G\left({\Lambda }_n{(G,r)}^c\right)\le {(k-1)}^r{n}^{rg\left(\overline{\beta }\right)}\le n^{r(\tilde{g}\left(\beta \right)/2+ϵ/2)}$. For the second term, we use Markov inequality:

$$\begin{array}{c}\hfill P_G({\Xi }_n\left(r\right)\ge n^{r\tilde{g}\left(\beta \right)/2}|{\Lambda }_n(G,r))\le \mathbb{E}\left[{\Xi }_n\left(r\right)\right|{\Lambda }_n(G,r)]n^{-r\tilde{g}\left(\beta \right)/2}\end{array}$$

The conditional expectation can be estimated as follows:

$$\begin{array}{cc}\hfill & \mathbb{E}\left[{\Xi }_n\left(r\right)\right|{\Lambda }_n(G,r)]=\sum _{dist(\overline{\sigma },X)=r}\sum _{trlogn=-\infty }^{-1}\hfill \\ \hfill & P_G(B_{\overline{\sigma }}-A_{\overline{\sigma }}=trlogn)exp({\beta }_{n}trlogn)\hfill \\ \hfill & \le {(k-1)}^r{n}^{r+r{\beta }_n(b-a)/k}+\sum _{dist(\overline{\sigma },X)=r}\sum _{trlogn=r\frac{b-a}{k}logn}^{-1}\hfill \\ \hfill & P_G(B_{\overline{\sigma }}-A_{\overline{\sigma }}=trlogn)exp({\beta }_{n}trlogn)\hfill \end{array}$$

$r+r{\beta }_n(b-a)/k=f_{{\beta }_n}\left(\frac{b-a}{k}\right)<\tilde{g}\left({\beta }_n\right)$, therefore, ${(k-1)}^r{n}^{r+r{\beta }_n(b-a)/k}{n}^{-r\tilde{g}\left(\beta \right)/2}\le$$n^{r(\tilde{g}\left(\beta \right)/2+ϵ/2)}$. Using Lemma 7, we have:

$$\begin{array}{c}\hfill P_G(B_{\overline{\sigma }}-A_{\overline{\sigma }}=trlogn)exp({\beta }_{n}rtlogn)\le n^{r(f_{{\beta }_n}\left(t\right)-1+O\left(\frac{1}{\sqrt{logn}}\right))}\end{array}$$

Since ${\beta }_n\to \beta$, $\forall ϵ$, when n is sufficiently large we have $\tilde{g}\left({\beta }_n\right)\le \tilde{g}\left(\beta \right)+ϵ/2$. Therefore,

$$\begin{array}{cc}\hfill & \sum _{dist(\overline{\sigma },X)=r}\sum _{\begin{array}{c}trlogn=\\ r(b-a)/klogn\end{array}}^{-1}{P}_G(B_{\overline{\sigma }}-A_{\overline{\sigma }}=tlogn)exp({\beta }_{n}rtlogn)\hfill \\ \hfill & \le n^{r(\tilde{g}\left({\beta }_n\right)-\tilde{g}\left(\beta \right)/2)}\hfill \\ \hfill & \le n^{r(\tilde{g}\left(\beta \right)/2+ϵ/2)}O(logn){(k-1)}^r\hfill \end{array}$$

Combining the above equations, we have:

$$\begin{array}{cc}\hfill P_G({\Xi }_n\left(r\right)\ge n^{r\tilde{g}\left(\beta \right)/2})& \le n^{r(\tilde{g}\left(\beta \right)/2+ϵ/2)}O(logn){(k-1)}^r\hfill \\ \hfill & \le n^{r(\tilde{g}\left(\beta \right)/2+ϵ)}\hfill \end{array}$$

When $r>\frac{n}{\sqrt{logn}}$, using Lemma 8, we can choose a sufficiently large constant $C>1$ such that $k^{n}exp(-Cn)<e^{-n}$:

$$\begin{array}{cc}\hfill \frac{P_{\sigma |G}(dist(\sigma ,X)=r|D(\sigma ,X))}{P_{\sigma |G}(\sigma =X|D(\sigma ,X))}& =\sum _{\begin{array}{c}D(\sigma ,X)\\ dist(\sigma ,X)=r\end{array}}\frac{P_{\sigma |G}(\sigma =\overline{\sigma })}{P_{\sigma |X}(\sigma =X)}\hfill \\ \hfill & >exp(-n)\hfill \end{array}$$

happens with probability less than $e^{-n}$. Therefore, Equation (35) holds.□

If $\gamma >b$ and $\beta <{\beta }^{*}$, we have the following lemma:

Lemma 10.

If $\gamma >b$ and $\beta <{\beta }^{*}$, there is a set ${\mathcal{G}}_n^{\left(1\right)}$ such that $P_G\left({\mathcal{G}}_n^{\left(1\right)}\right)\ge 1-n^{g\left(\overline{\beta }\right)}$ and:

$$\begin{array}{cc}\hfill \mathbb{E}[\sum _{r=1}^{n}exp\left({\beta }_n(A_r^s-A_r^0)\right)|G\in {\mathcal{G}}_n^{\left(1\right)}]& =(1+o\left(1\right))n^{g\left({\beta }_n\right)}\hfill \end{array}$$

(38)

$$\begin{array}{cc}\hfill Var[\sum _{r=1}^{n}exp\left({\beta }_n(A_r^s-A_r^0)\right)|G\in {\mathcal{G}}_n^{\left(1\right)}]& \le (1+o\left(1\right))n^{g\left(2{\beta }_n\right)}\hfill \end{array}$$

(39)

Lemma 10 is an extension of Proposition 10 in [12] and can be proved using almost the same analysis. Thus we omit the proof of Lemma 10 here.

Lemma 11.

If $\gamma >b$ and $\beta <{\beta }^{*}$, there is a set ${\mathcal{G}}_n^{\prime }$ such that:

$$P_G\left({\mathcal{G}}_n^{\prime }\right)\ge 1-(1+o\left(1\right))max\{n^{g\left(\overline{\beta }\right)},n^{\tilde{g}\left(2{\beta }_n\right)-2g\left({\beta }_n\right)+ϵ}\}$$

(40)

and for every $G\in {\mathcal{G}}_n^{\prime }$,

$$\frac{P_{\sigma |G}(dist(\sigma ,X)=1|D(\sigma ,X))}{P_{\sigma |G}(\sigma =X|D(\sigma ,X))}\ge (1+o\left(1\right))n^{g\left({\beta }_n\right)}$$

(41)

Proof. 

The left-hand side of Equation (41) can be rewritten as:

$$\frac{P_{\sigma |G}(dist(\sigma ,X)=1)}{P_{\sigma |G}(\sigma =X)}=(1+o\left(1\right))\sum _{s=1}^{k-1}\sum _{r=1}^{n}exp\left({\beta }_n(A_r^s-A_r^0)\right)$$

(42)

Let ${\mathcal{G}}_n^{\left(1\right)}$ be defined in Lemma 10 and ${\mathcal{G}}_n^{\left(2\right)}:=\left\{\right|{\sum }_{r=1}^{n}exp\left({\beta }_n(A_r^s-A_r^0)\right)-(1+o\left(1\right))n^{g\left({\beta }_n\right)}|\le n^{g\left({\beta }_n\right)-ϵ/2}\}$.

Using Chebyshev’s inequality, we have:

$$P_G(G\notin {\mathcal{G}}_n^{\left(2\right)}|G\in {\mathcal{G}}_n^{\left(1\right)})\le n^{\tilde{g}\left(2{\beta }_n\right)-2g\left({\beta }_n\right)+ϵ}$$

Let ${\mathcal{G}}_n^{\prime }={\mathcal{G}}_n^{\left(1\right)}\cap {\mathcal{G}}_n^{\left(2\right)}$:

$$\begin{array}{cc}\hfill P_G(G\in {\mathcal{G}}_n^{\prime })& =P_G\left({\mathcal{G}}_n^{\left(1\right)}\right)P_G(G\in {\mathcal{G}}_n^{\left(2\right)}|G\in {\mathcal{G}}_n^{\left(1\right)})\hfill \\ \hfill & \ge (1-n^{\tilde{g}\left(2{\beta }_n\right)-2g\left({\beta }_n\right)+ϵ})(1-n^{g\left(\overline{\beta }\right)})\hfill \\ \hfill & =1-(1+o\left(1\right))max\{n^{g\left(\overline{\beta }\right)},n^{\tilde{g}\left(2{\beta }_n\right)-2g\left({\beta }_n\right)+ϵ}\}\hfill \end{array}$$

and for every $G\in {\mathcal{G}}_n^{\prime }$,

$$\sum _{r=1}^{n}exp\left({\beta }_n(A_r^s-A_r^0)\right)=(1+o\left(1\right))n^{g\left({\beta }_n\right)}$$

Therefore, from Equation (42) we have:

$$\frac{P_{\sigma |G}(dist(\sigma ,X)=1)}{P_{\sigma |G}(\sigma =X)}\ge (1+o\left(1\right))n^{g\left({\beta }_n\right)}$$

□

Let $\Lambda :=\{{\omega }^j·{\mathbf{1}}_n|j=0,\dots ,k-1\}$ where ${\mathbf{1}}_n$ is the all-ones vector with dimension n, and we have the following lemma:

Lemma 12.

Suppose $\gamma <b$ and $\overline{\sigma }$ satisfies $dist(\overline{\sigma },{\mathbf{1}}_n)\ge \frac{n}{\sqrt{logn}}$ and $D(\overline{\sigma },{\mathbf{1}}_n)$. Then the event $P_{\sigma |G}(\sigma =\overline{\sigma })>exp(-Cn)P_{\sigma |G}(\sigma ={\mathbf{1}}_n)$ happens with a probability (with respect to SSBM) less than $exp(-\tau (\alpha ,\beta )n\sqrt{logn})$ where C is an arbitrary constant, $\tau (\alpha ,\beta )$ is a positive number.

Proof. 

Let $n_r=\left|\{{\overline{\sigma }}_i=w^r|i\in \left[n\right]\}\right|$. Then $n_0\ge n_r$ for $r=1,\dots ,k-1$ since $arg{min}_{{\sigma }^{\prime }\in \Lambda }dist(\overline{\sigma },{\sigma }^{\prime })={\mathbf{1}}_n$. Without loss of generality, we suppose $n_0\ge n_1\dots \ge n_{k-1}$. Define $N_{\overline{\sigma }}=\frac{1}{2}(n(n-1)-{\sum }_{r=0}^{k-1}{n}_r(n_r-1))=\frac{1}{2}(n^2-{\sum }_{r=0}^{k-1}{n}_r^2)$. Denote the event $P_{\sigma |G}(\sigma =\overline{\sigma })>exp(-Cn)P_{\sigma |G}(\sigma ={\mathbf{1}}_n)$ as $D^{\prime }(\overline{\sigma },C)$, which can be transformed as:

$$\begin{array}{c}\hfill (1+\frac{\gamma logn}{n})\left(\sum _{{\overline{\sigma }}_i\ne {\overline{\sigma }}_j,X_i=X_j}{Z}_{ij}+\sum _{{\overline{\sigma }}_i\ne {\overline{\sigma }}_j,X_i\ne X_j}{Z}_{ij}\right)\\ \hfill \le \frac{\gamma logn}{n}{N}_{\overline{\sigma }}+\frac{C}{\beta }n\end{array}$$

(43)

Firstly we estimate the order of $N_{\overline{\sigma }}$, and obviously $N_{\overline{\sigma }}\le \frac{1}{2}{n}^2$. Using the conclusion in Appendix A of [18], we have:

$$\sum _{r=0}^{k-1}{n}_r^2\le \left\{\begin{array}{cc}n{n}_0\hfill & n_0\le \frac{n}{2}\hfill \\ n^2-2n_0(n-n_0)\hfill & n_0>\frac{n}{2}\hfill \end{array}\right.$$

(44)

By assumption of $dist(\overline{\sigma },{\mathbf{1}}_n)\ge \frac{n}{\sqrt{logn}}$, we have $n_0\le n-\frac{n}{\sqrt{logn}}$ and $n_0\ge \frac{n}{k}$ follows from $n_0\ge n_r$. When $n_0>\frac{n}{2}$, we have $N_{\overline{\sigma }}\ge n_0(n-n_0)\ge \frac{n^2}{\sqrt{logn}}(1+o\left(1\right))$. The second inequality is achieved if $n_0=n-\frac{n}{\sqrt{logn}}$. When $n_0<\frac{n}{2}$, $N_{\overline{\sigma }}\ge \frac{n^2-n{n}_0}{2}\ge \frac{n^2}{4}$ and the second inequality is achieved when $n_0=\frac{n}{2}$. Thus generally we have $\frac{n^2}{\sqrt{logn}}(1+o\left(1\right))\le N_{\overline{\sigma }}\le \frac{n^2}{2}$.

Since $\frac{C}{\beta }n=o\left(\frac{logn}{n}{N}_{\overline{\sigma }}\right)$ we can rewrite Equation (43) as:

$$\left(\sum _{\begin{array}{c}{\overline{\sigma }}_i\ne {\overline{\sigma }}_j\\ X_i=X_j\end{array}}-Z_{ij}+\sum _{\begin{array}{c}{\overline{\sigma }}_i\ne {\overline{\sigma }}_j\\ X_i\ne X_j\end{array}}-Z_{ij}\right)\ge -\gamma \frac{logn{N}_{\overline{\sigma }}}{n}(1+o\left(1\right))$$

(45)

Let $N_1={\sum }_{{\overline{\sigma }}_i\ne {\overline{\sigma }}_j,X_i=X_j}1$ and $N_2={\sum }_{{\overline{\sigma }}_i\ne {\overline{\sigma }}_j,X_i\ne X_j}1=N_{\overline{\sigma }}-N_1$.

Using the Chernoff inequality we have:

$$\begin{array}{cc}\hfill P_G\left(D^{\prime }(\overline{\sigma },C)\right)& \le {\left(\mathbb{E}[exp(-sZ)]\right)}^{N_1}{\left(\mathbb{E}[exp(-s{Z}^{\prime })]\right)}^{N_2}\hfill \\ \hfill & ·exp\left(\gamma \frac{logn{N}_{\overline{\sigma }}s}{n}(1+o\left(1\right))\right)\hfill \\ \hfill & =exp(\frac{logn}{n}(1+o\left(1\right))(e^{-s}-1)(a{N}_1+b{N}_2)\hfill \\ \hfill & +\gamma \frac{logn{N}_{\overline{\sigma }}s}{n}(1+o\left(1\right)))\hfill \end{array}$$

Since $s>0$ and $a>b$, we further have:

$$\begin{array}{cc}\hfill P_G\left(D^{\prime }(\overline{\sigma },C)\right)& \le exp\left(\frac{N_{\overline{\sigma }}logn}{n}(b(e^{-s}-1)+\gamma s+o\left(1\right))\right)\hfill \end{array}$$

Let $h_b\left(x\right)=x-b-xlog\frac{x}{b}$, which satisfies $h_b\left(x\right)<0$ for $0<x<b$, and take $s=-log\frac{\gamma }{b}>0$, using $N_{\overline{\sigma }}\ge \frac{n^2}{\sqrt{logn}}$ we have:

$$\begin{array}{cc}\hfill P_G\left(D^{\prime }(\overline{\sigma },C)\right)& \le exp\left(N_{\overline{\sigma }}\frac{logn}{n}{h}_b\left(\gamma \right)(1+o\left(1\right))\right)\hfill \\ \hfill & \le exp\left(h_b\left(\gamma \right)n\sqrt{logn}(1+o\left(1\right))\right)\hfill \end{array}$$

□

Proof of Theorem 2.

(1) Since $P_{\sigma }(\sigma \notin S_k\left(X\right))={\sum }_{f\in S_k}{P}_{\sigma }(\sigma \ne f\left(X\right)|D(\sigma ,f\left(X\right)))P_{\sigma }\left(D(\sigma ,f\left(X\right))\right)$, we only need to establish $P_{\sigma }(\sigma \ne X|D(\sigma ,X))\le n^{\tilde{g}\left(\beta \right)/2+ϵ}$. From Lemma 9, we can find ${\mathcal{G}}_n^{\left(r\right)}$ for $r=1,\dots ,n$. Let ${\mathcal{G}}_n^{\prime }={\cap }_{r=1}^n{\mathcal{G}}_n^{\left(r\right)}$ and choose $\frac{ϵ}{2}$; from Equations (32) and (34), we have:

$$\begin{array}{cc}\hfill P_G\left({\mathcal{G}}_n^{\prime c}\right)& =P_G\left({\cup }_{r=1}^n{\left({\mathcal{G}}_n^{\left(r\right)}\right)}^c\right)\hfill \\ \hfill & \le \sum _{r=1}^{n/\sqrt{logn}}{n}^{r(\tilde{g}\left(\beta \right)/2+ϵ/2)}+n{e}^{-n}\hfill \\ \hfill & \le \frac{1}{2}{n}^{\tilde{g}\left(\beta \right)/2+ϵ}\hfill \end{array}$$

where the last equality follows from the estimation of sum of geometric series. On the other hand, for every $G\in {\mathcal{G}}_n^{\prime }$, from Equations (33) and (35), we have:

$$\begin{array}{cc}\hfill \frac{P_{\sigma |G}(\sigma \ne X|D(\sigma ,X))}{1-P_{\sigma |G}(\sigma \ne X|D(\sigma ,X))}& =\frac{P_{\sigma |G}(\sigma \ne X|D(\sigma ,X))}{P_{\sigma |G}(\sigma =X|D(\sigma ,X))}\hfill \\ \hfill & <\sum _{r=1}^{n/\sqrt{logn}}{n}^{r\tilde{g}\left(\beta \right)/2}+n{e}^{-n}\hfill \end{array}$$

from which we can get the estimation $P_{\sigma |G}(\sigma \ne X|D(\sigma ,X))\le \frac{1}{2}{n}^{\tilde{g}\left(\beta \right)/2+ϵ}$. Finally,

$$\begin{array}{cc}\hfill P_{\sigma }(\sigma \ne X|D(\sigma ,X))& =\sum _{G\in {\mathcal{G}}_n^{\prime }}{P}_G\left(G\right)P_{\sigma |G}(\sigma \ne X|D(\sigma ,X))\hfill \\ \hfill +P_G\left({\mathcal{G}}_n^{\prime c}\right)& \le n^{\tilde{g}\left(\beta \right)/2+ϵ}.\hfill \end{array}$$

(2) When $\beta <{\beta }^{*}$, using Lemma 11, for every $G\in {\mathcal{G}}_n^{\prime }$ we can obtain:

$$\frac{1-P_{\sigma |G}(\sigma =X|D(\sigma ,X))}{P_{\sigma |G}(\sigma =X|D(\sigma ,X))}\ge (1+o\left(1\right))n^{g\left({\beta }_n\right)}$$

We then have:

$$P_{\sigma |G}(\sigma =X|D(\sigma ,X))\le (1+o\left(1\right))n^{-g\left({\beta }_n\right)}$$

Then:

$$\begin{array}{cc}\hfill & P_{\sigma }(\sigma =X|D(\sigma ,X))\le P\left({\mathcal{G}}_n^{\prime c}\right)\hfill \\ \hfill & +\sum _{G\in {\mathcal{G}}_n^{\prime }}{P}_G\left(G\right)P_{\sigma |G}(\sigma =X|D(\sigma ,X))\hfill \\ \hfill & \le (1+o\left(1\right))n^{-g\left({\beta }_n\right)}+(1+o\left(1\right))max\{n^{g\left(\overline{\beta }\right)},n^{\tilde{g}\left(2{\beta }_n\right)-2g\left({\beta }_n\right)+ϵ}\}\hfill \\ \hfill & \le (1+o\left(1\right))max\{n^{g\left(\overline{\beta }\right)},n^{-g\left(\beta \right)+ϵ}\}\hfill \end{array}$$

(3) When $\gamma <b$, for any $f\in S_k$, we have $dist(f\left(X\right),\Lambda )=\frac{(k-1)n}{k}>\frac{n}{\sqrt{logn}}$. Therefore, using Lemma 12, we can find a graph ${\mathcal{G}}_n^{\prime }$ such that $P_C\left({\mathcal{G}}_n^{\prime }\right)\le exp(-nC)$ and for any $G\notin {\mathcal{G}}_n^{\prime }$, $P_{\sigma |G}(\sigma =f\left(X\right))\le exp(-Cn)$. Therefore,

$$\begin{array}{c}\hfill P_a\left({\widehat{X}}^{*}\right)\le P_G\left({\mathcal{G}}_n^{\prime }\right)+k!exp(-Cn)=(1+k!)exp(-Cn)\end{array}$$

The conclusion of $P_a\left({\widehat{X}}^{*}\right)\le exp(-Cn)$ follows since C can take any positive value. □

8.3. Proof of Theorem 3

Lemma 13

(Lemma 8 of [7]). Let m be a positive number larger than n. When $Z_1,\dots ,Z_m$ are i.i.d. Bernoulli($\frac{blogn}{n}$) and $W_1,\dots ,W_m$ are i.i.d Bernoulli($\frac{alogn}{n}$), independent of $Z_1,\dots ,Z_m$, then:

$$P(\sum _{i=1}^m(Z_i-W_i)\ge 0)\le exp(-\frac{mlogn}{n}{(\sqrt{a}-\sqrt{b})}^2)$$

(46)

Proof of Theorem 3.

Let $P_F^{\left(r\right)}$ denote the probability when there is $\sigma$ satisfying $dist(\sigma ,X)=r$ and $H\left(\sigma \right)<H\left(X\right)$.

From Equation (23), when $\sigma$ differs from X only by one coordinate, from Lemma 13 the probability for $H\left(\sigma \right)<H\left(X\right)$ is bounded by $P_G(A_r^s-A_r^0>0)\le n^{-\frac{{(\sqrt{a}-\sqrt{b})}^2}{k}}$. Therefore, $P_F^{\left(1\right)}\le (k-1)n^{g\left(\overline{\beta }\right)}$. Using Lemma 7, we can get $P_F^{\left(r\right)}\le {(k-1)}^r{n}^{rg\left(\overline{\beta }\right)}$ for $r\le \frac{n}{\sqrt{logn}}$. For $r\ge \frac{n}{\sqrt{logn}}$, choosing $C=0$ in Lemma 8 we can get ${\sum }_{r\ge \frac{n}{\sqrt{logn}}}{P}_F^{\left(r\right)}\le e^{-n}$. That is, the dominant term is ${\sum }_{r\le \frac{n}{\sqrt{logn}}}{P}_F^{\left(r\right)}$ since the other part decreases exponentially fast. Therefore, the upper bound for error rate of ${\widehat{X}}^{\prime }$ is:

$$\begin{array}{cc}\hfill P_F=& \sum _{r=1}^n{P}_F^{\left(r\right)}\le (1+o\left(1\right))\sum _{r=1}^{\infty }{(k-1)}^r{n}^{rg\left(\overline{\beta }\right)}\hfill \\ \hfill \le & (1+o\left(1\right))\frac{(k-1)n^{g\left(\overline{\beta }\right)}}{1-(k-1)n^{g\left(\overline{\beta }\right)}}=(k-1+o\left(1\right))n^{g\left(\overline{\beta }\right)}\hfill \end{array}$$

When $\sigma \in W^{*}$, since $|\{r\in \left[n\right]|{\sigma }_r={\omega }^i\}|=I_i^{\prime }+\frac{n}{k}-I_i$, we have $I_i^{\prime }=I_i$. From Lemma 6, we can see $|B_{\overline{\sigma }}|=|A_{\overline{\sigma }}|$ and $N_{\overline{\sigma }}=0$. Then from Equation (24), $H\left(\overline{\sigma }\right)<H\left(X\right)$ is equivalent with $B_{\overline{\sigma }}>A_{\overline{\sigma }}$.

Therefore, when $dist(\overline{\sigma },X)\ge \frac{n}{\sqrt{logn}}$ and $D(\sigma ,X)$, from Equation (31), we have $A_{\overline{\sigma }}\ge \frac{n^2}{k^2(k-1)\sqrt{logn}}(1+o\left(1\right))$. We use Lemma 13 by letting $m=|A_{\overline{\sigma }}|$ when $m\ge \frac{n}{\sqrt{logn}}$; the error term is bounded by ${\sum }_{r\ge \frac{n}{\sqrt{logn}}}{P}_F^{\left(r\right)}\le {\sum }_{r\ge \frac{n}{\sqrt{logn}}}{(k-1)}^{r}exp(-\frac{{(\sqrt{a}-\sqrt{b})}^2}{k^2(k-1)}n\sqrt{logn}))\le exp(-n)$, which decreases exponentially fast. For $m<\frac{n}{\sqrt{logn}}$, we can use Lemma 7 directly by considering ${\sum }_{r=2}^{\frac{n}{\sqrt{logn}}}{P}_F^{\left(r\right)}$. The summation starts from $r=2$ since $\sigma \in W^{*}$. Therefore,

$$\begin{array}{cc}\hfill P_F=& \sum _{r=2}^n{P}_F^{\left(r\right)}\le (1+o\left(1\right))\sum _{r=2}^{\infty }{(k-1)}^r{n}^{rg\left(\overline{\beta }\right)}\hfill \\ \hfill \le & ({(k-1)}^2+o\left(1\right))n^{2g\left(\overline{\beta }\right)}.\hfill \end{array}$$

□