Transient Behavior of Gossip Opinion Dynamics with Community Structure

We study transient behavior of gossip opinion dynamics, in which agents randomly interact pairwise over a weighted graph with two communities. Edges within a community have identical weights different from edge weights between communities. We first derive an upper bound for the second moment of agent opinions. Using this bound, we obtain upper bounds for probability that a large proportion of agents have opinions close to average ones. The results imply a phase transition of transient behavior of the process: When edge weights within communities are larger than those between communities and those between regular and stubborn agents, most agents in the same community hold opinions close to the average opinion of that community with large probability, at an early stage of the process. However, if the difference between intra- and inter-community weights is small, most of the agents instead hold opinions close to everyone's average opinion at the early stage. In contrast, when the influence of stubborn agents is large, agent opinions settle quickly to steady state. We then conduct numerical experiments to validate the theoretical results. Different from traditional asymptotic analysis in most opinion dynamics literature, the paper characterizes the influences of stubborn agents and community structure on the initial phase of the opinion evolution.


Introduction
Opinion dynamics is the study of how personal opinions change through interactions in social networks.Analysis of convergence and stability of opinion dynamics has gained considerable attention in recent decades [1], but less research has focused on transient behavior of the opinion formation process.Social networks often have topologies where subgroups of nodes are densely connected internally but loosely connected with others (that is, community structure [2]).Such structure is observed to be able to influence opinion dynamics [3,4].It is often difficult to determine whether a realistic social network reaches steady state or not, and whether its communities evolve homogeneously at the early stage.So it is necessary to investigate how the opinion formation process, especially during its initial phase, corresponds to its community structure.Such results can also provide insight into design of community detection algorithms based on state observations [5,6] and design of model reduction tools for large-scale networks [7].

Related Work
Individual opinion can be modeled by a continuous variable taking values in an interval of real numbers, or a discrete variable in a finite set [1,8].There are at least three types of continuous-opinion models explaining how interpersonal interactions shape opinion profiles of social networks, namely, models of assimilative, homophily, and negative social influences [1,9].Evidences for all three types have been found in recent empirical studies [10,11,12].A crucial example of the first class of models is the DeGroot model [13], in which agents update according to the average of their neighbors' opinions.The Friedkin-Johnsen (FJ) model [14] generalizes the DeGroot model and allows long-term disagreement, rather than consensus, by assuming that agents are consistently affected by their initial opinions.The Hegselmann-Krause (HK) model [15] and the Deffuant-Weisbuch (DW) model [16] are representatives of the second class of models.In these models, agents stay away from those who hold different beliefs, and hence the agents tend to form clusters in the end.Negative influences presence in the third class of models can increase opinion difference between individuals, and make the group end in polarization [1,17].
Most studies of opinion dynamics have focused on asymptotic behavior, attempting to answer why opinion disagreement occurs in the long run even though social interactions tend to reduce opinion difference [9,18].In contrast, transient behavior has attracted less attention.As the availability of large-scale datasets from the internet increases, there is a growing need to understand how the process behaves over a finite time interval [19,20,21].Important behavioral dynamics, such as election and online discussion, often have finite duration, and their prediction based on transient evolution is of great interest [10,22].Extensive amount of information produced by social media nowadays may change public opinions only temporarily [23], making it hard for the dynamics process to reach steady state.Asymptotic analysis thus may not be sufficient for understanding such scenarios.Finally, real large-scale networked dynamic processes may converge slowly [19,24], but stay close to a certain state for a long time [25,26].In practice, to distinguish between two types of states can be challenging and requires knowledge of transient system behavior.The authors in [19] propose a framework to analyze the transient stage of discrete-state Markov chains that model opinion dynamics.The paper [25] studies quasistationary distributions of a contact process, and [27] analyzes the transient opinion profiles of a voter model.The authors in [26] provide criteria for detecting transient clusters in a generalized HK model that normally reaches a consensus asymptotically.A recent paper [28] studies how opinion difference evolves over finite time intervals for a stochastic bounded confidence model with two agents and the model over a bistar graph.
The study of community structure can be traced back to [29], in which a community is defined as a complete subgraph in a network.One of the modern definitions for communities is modularity, which characterizes the nonrandomness of a group partition [30].The stochastic block model (SBM) [31], generating random graphs with communities, is another popular framework in the literature.Researchers have studied how community structure of a network influences opinion evolution, based on several models such as the DW model [32,33], the Taylor model [34], and the Sznajd model [35].The paper [36] studies the DeGroot model with stubborn agents over a weighted graph, and shows that the steady state of agents in the same community concentrates around the state of some stubborn agent.This paper considers transient behavior of a gossip model with stubborn agents.This model, in which agents randomly interact in pairs, is a stochastic counterpart of the DeGroot model.The model captures the random nature of interpersonal influence and exhibits various behavior.Consensus of the gossip model has been studied in [37,38].The authors in [39] show that the existence of stubborn agents may explain fluctuation of social opinions, and also that, if the network is highly fluid, then the expected steady state of regular (non-stubborn) agents is close.In contrast, it is shown in [36] that polarization can emerge, if the model evolves over a weighted graph with two stubborn agents.Studying transient behavior of the gossip model can provide insight into analysis of more complex models, because the linear averaging rule is a key building block of most opinion models.

Contribution
In this paper, we study transient behavior of the gossip model over a weighted graph with two communities.It is assumed that edges within communities have identical weights different from edge weights between communities.We first obtain an upper bound for the second moment of agent states (Lemma 4).Using this bound, we are able to provide probability bounds for agent states concentrating around average opinions (Corollary 1), expected average opinions (Theorem 1), and average initial opinions (Theorem 2).The results reveal a phase transition phenomenon [40,41,42]: When edge weights within communities are larger than those between communities and those between regular and stubborn agents, most agents in the same community have states with small deviation from the average opinion of that community with large probability, at the early stage of the process.But if the difference between intra-and inter-community weights is not so large, most agents in the network have states concentrating around everyone's average opinion with small error and large probability at the early stage (Corollary 2).In contrast, if weights between regular and stubborn agents are larger than those between regular agents, agent states have a distribution close to their stationary one, right after the beginning of the process (Theorem 3 and Corollary 3).
These results indicate that the gossip model has entirely different transient behavior, under different link strength parameters.It is known that the expected steady state depends on the positions of stubborn agents [39], and the model reaches a consensus if there are no stubborn agents [37,38].The obtained results indicate that agents may form transient clusters if the influence of stubborn agents is relatively small.These transient clusters may not be the same as the stationary ones, because they only depend on initial regular states and edge weights between the regular.In addition, the results demonstrate how transient behavior of the model corresponds to community structure of the network, by showing that the difference between intra-and inter-community weights has to be large enough to ensure the existence of such a correspondence and by showing how the relative magnitude of the weights influences the duration of the transient clusters.
The obtained results can be directly applied to community detection based on state observations [5,6].Suppose that a network is unknown but several snapshots of an opinion dynamic are available.Then our results ensure that partitioning agent states can recover the community labels of agents, if the states are collected in certain transient time interval and intra-community influence is large.The results can also provide insight into predicting and distinguishing transient behavior of an opinion formation process in practice [19,22].For example, when the size of a network and the difference between intraand inter-community interaction strength are large, the duration of such transient interval is expected to be large as well.Given an estimate of the runtime of a process, we may determine whether the current clusters are steady or not.Finally, exploiting properties of transient clusters can help improve model reduction at the initial phase of dynamics over large-scale networks with community structure [7].If agents in the same community have states close to each other at the early stage of the process, then we may track the opinion formation process with much less parameters than the original system, by replacing topological data with community labels.
In the early work [43], we study how the expectation of agent states concentrates around average initial states, which follows from Lemma 3 of the current paper.Here we directly investigate how agent states evolve by analyzing the second moment and thus provide more detailed characterization of transient behavior.

Outline
In Section 2 we introduce the considered model and problem.Section 3 provides theoretical results.Numerical experiments are presented in Section 4. Section 5 concludes the paper.Some proofs are given in the appendices.
Notation.Denote the n-dimensional Euclidean space by R n , the set of n × m real matrices by R n×m , the set of nonnegative integers by N, and N + = N \ {0}.Denote the natural logarithm by log x, x ∈ R. Let 1 n be the allone vector with dimension n, e (n) i be the n-dimensional unit vector with i-th entry being one, I n be the n × n identity matrix, and 0 m,n be the m × n all-zero matrix.Denote the Euclidean norm of a vector by • .For a vector x ∈ R n , denote its i-th entry by x i , and for a matrix A ∈ R n×n , denote its (i, j)-th entry by a ij or [A] ij .The cardinality of a set S is denoted by |S|.The function I [property] is the indicator function equal to one if the property in the bracket holds, and equal to zero otherwise.Denote the expectation of a random vector X by E{X}.For two sequences of real numbers, f (n) and ) and f (n) = Ω(g(n)) hold.For x, y ∈ R, denote x ∨ y := max{x, y} and x ∧ y := min{x, y}.

Problem Formulation
The gossip model with stubborn agents is a random process evolving over an undirected graph G = (V, E, A), where V is the node set with |V| = n ≥ 2, E is the edge set, and A = [a ij ] ∈ R n×n is the weighted adjacency matrix.The graph G has no self-loops (i.e., a ii = 0, ∀i ∈ V).V contains two types of agents, regular and stubborn, denoted by V r and V s , respectively (so V = V r ∪ V s and V r ∩ V s = ∅).In this paper we assume that the regular agents form two disjoint communities V r1 and V r2 , and denote n j=i+1 a ij is the sum of all edge weights.Hence 1 T W 1/2 = 1.At time t, edge {i, j} is selected with probability w ij independently of previous updates, and agents update as follows, where Q(t) ∈ R nr×nr , R(t) ∈ R nr×ns , and In other words, only regular agents in {i, j} update their states to the average of i and j's previous states, and all other agents do not update.
We study how community structure and stubborn agents influence transient behavior of agent states X(t).The community structure will be defined in Section 3.1.By transient behavior we mean a property of X(t) that holds over a finite time interval, as opposed to asymptotic behavior that holds as time t → ∞.As mentioned in Section 1.1, agents in the same community tend to have similar states, but how well and how long these clusters form still require rigorous analysis.We characterize the transient clusters of X(t) based on three types of references: (1) agents' average states within communities and everyone's average state at time t, (2) expected average states at time t, and (3) average initial states.As a comparison, we also study the time when the distribution of X(t) is close to the stationary distribution.To sum up, the considered problem is as follows.
Problem.Given the initial regular states X(0), the stubborn states z s , the community structure C, and the weighted adjacency matrix A, provide bounds for the deviation of X(t) from the three types of average states over finite time intervals, and bounds for the time when the distribution of X(t) is close to the stationary distribution.
Probability bounds for the deviation of X(t) from average states at time t are given in Corollary 1, which is a consequence of second-moment analysis (Lemma 4).
The deviation from expected average states is analyzed in Theorem 1.This result is stronger than the previous one because the references are deterministic.Theorem 2 shows how X(t) concentrates around average initial states.Theorem 3 gives a lower bound of the time when X(t) is close to steady state.

Theoretical Analysis
We study transient behavior of gossip model with two communities.The analysis provides crucial insight into understanding transient behavior of the model under general conditions.Main results are presented in Section 3.1 and a discussion is given in Section 3.2.

Main Results
We assume that the regular agents form two disjoint communities V r1 and V r2 with equal size.For simplicity, sort the agents as follows: V r1 = {1, . . ., r 0 n/2}, V r2 = {1 + r 0 n/2, . . ., r 0 n}, and V s = {1 + r 0 n, . . ., n}, with r 0 ∈ (0, 1) such that r 0 n = n r is an even integer.The portion of stubborn agents is denoted by s 0 := 1 − r 0 .We introduce the following assumptions, illustrated in Fig. 1, for the weighted adjacency matrix A of graph G.

Assumption 1 (Network topology)
(iii) There exists a nonnegative number l (s) depending on n such that 1≤j≤s0n l Remark 1 Assumption 1 (i) indicates that the graph on regular agents is a weighted graph such that edges between agents in the same community have the same weight l (r) s whereas those between communities have weight l (r) d .In other words, the influence strength between agents depends on their latent attributes.Such a weighted graph can be treated as the expected adjacency matrix of an SBM, in which nodes are assigned with community labels, and an edge exists with probability depending on the community labels of that edge's endpoints.We introduce this simplified assumption to highlight the phase transition phenomenon in the transient phase of the dynamics, and proofs under this assumption are still nontrivial.It is possible to generalize the results to the SBM case by considering the concentration of adjacency matrices [44].Although assuming that interpersonal influence depends only on community labels is a simplified setting, this model has been found to be efficient also in empirical studies [10].Note that the adjacency matrix A has zeros entries in the diagonal from the assumption that the graph has no self-loops.Parameter l (s) given in Assumption 1 (iii) is the sum of edge weights between one regular agent and all stubborn agents, and thus represents the total influence of stubborn agents on this regular agent.We assume that this sum is the same for all regular agents for analysis simplicity.The results can be extended to the case where the weight sums have upper and lower bounds.
We further impose an assumption for the initial vector.

Assumption 2 (Initial condition)
The initial regular states X(0) and the stubborn states z s are deterministic, and satisfy that |X i (0)| ≤ c x and |z s j | ≤ c x , for all 1 ≤ i ≤ r 0 n and 1 ≤ j ≤ s 0 n, and some c x > 0.
From the definitions of Q(t) and R(t) in (1), it follows that where where It can be shown that Ã has a simple eigenvalue r 0 n(l ]/(2α), we present the following summary of properties of Q and orthogonal vectors, which will be used later.
Before presenting main theorems, we provide several properties of the gossip model.The first lemma concerns the explicit expression of the weight sum of the graph G.
Lemma 2 Under Assumption 1, the weight sum α = PROOF.The conclusion follows directly from the definition of α and Assumption 1. 2 The next lemma gives the expression of E{X(t)}, and shows how E{X(t)} evolves by decomposing it into three parts, which correspond to the eigenspaces of Q.
Lemma 3 Suppose that Assumption 1 holds.Then the expectation of X(t) satisfies that, for all t ∈ N, Then the result follows from (1)-(3). 2 We further introduce the following technical assumption.
Assumption 3 Denote l(s) for some constant c l > 0.

Remark 2
The assumption implies that both l(s) + (the maximum influence capability of a stubborn agent on all regular agents) and l(s) − (the maximum of absolute difference between the total influences of a stubborn agent on the two communities) are of the same order of l (s) .Otherwise, the influence of stubborn agents may not be homogeneous, which could be hard to analyze.Now we present a lemma that will be used in the proof of main theorems.Denote X η (t) := ηη T X(t), X ξ (t) := ξξ T X(t), Γ := r0n i=3 w (i) (w (i) ) T (thus, Γ T Γ = Γ 2 = Γ), and X Γ (t) := ΓX(t).From Lemma 1, ηη T + ξξ T + Γ = I r0n , and we have the decomposition X(t) = X η (t) + X ξ (t) + X Γ (t).Further, let X ⊥ (t) := ΓX(t), where Γ = ξξ T + Γ.We get another decomposition of X(t): X(t) = X η (t) + X ⊥ (t) and X η (t) T X ⊥ (t) = 0. Note that X η (t) + X ξ (t) and X η (t) represent the average states in each community and everyone's average state, respectively (the i-th entry of So the two decompositions reveal dynamics of average states.The following lemma gives two upper bounds for the second moment of X(t) under the two decompositions.
Lemma 4 Suppose that Assumptions 1-3 hold.Denote where PROOF.The main idea of the proof is to separately bound the terms An immediate consequence of the preceding analysis is that the difference between the agent states and average states at each time step can be bounded.For two vectors X, Y ∈ R r0n and ε ∈ (0, 1), we denote the set of agents i such that |X i − Y i | > εc x (i.e., the difference between X i and Y i is large) by Then the proof of Lemma 4 ensures the following result.
Corollary 1 Suppose that Assumptions 1-3 hold.Then it holds for t ∈ N, n ≥ 4/r 0 , and ε, δ ∈ (0, 1) that Remark 4 The first result (7) bounds the probability of at least δr 0 n agents having states at least εc x away from the average states in their communities.This probability bound decreases first due to the decay of (1 − λ 3 ) t and the relatively small value of λ i t, and then increases with t to a constant bound (whether the constant bound is trivial depends on the ratios λ i /λ 3 , i = 1, 2, which we will discuss in detail in Corollary 2).Similarly, the second result (8) shows that there can be a time interval, over which many agents have states close to everyone's average state with high probability.
PROOF.To prove the results, note that by the Markov inequality, for X, Y ∈ R r0n and ε ∈ (0, 1), Now note that X(t)−X η (t)−X ξ (t) = X Γ (t) and X(t)− X η (t) = X ⊥ (t), so (7) and ( 8) follow from the upper bound of E{ X Γ (t) 2 } and E{ X ⊥ (t) 2 } given in the proof of Lemma 4. 2 The preceding results indicate that we can use average states as references to describe how agent states evolve in finite time.Further analysis based on Lemma 4 can yield stronger results.That is, we can use the expected average states E{X η (t) + X ξ (t)} and E{X η (t)} as references.

Remark 5
The results provide bounds for the probability of agent states concentrating around expected average states.In particular, consider the case where stubbornagent influence is small (i.e., λ 1 is small).If the influence strength within communities is much larger than that between communities (hence λ 2 /λ 3 is small), then it follows from (10) that most agent states concentrate around average states within communities and thus form transient clusters with high probability, at the early phase of the process.If the stubborn-agent influence is small (λ 1 /(λ 2 ∧ λ 3 ) is small), but the influence strength within and between communities is similar (λ 2 and λ 3 have similar values), then (11) yields that most agent states concentrate around everyone's average state with high probability.See Corollary 2 for detailed discussion on how link strength parameters influence the concentration.

Remark 6
In this remark we compare the obtained results with existing transient behavior analysis.The authors of [26] study a generalized HK model and define a transient cluster as a subgroup of agents whose opinion range decreases faster than the distance of the subgroup from other agents.Here we characterize agent states based on their distance from the average states.Such a framework includes situations where clusters approach each other, but defining such references requires specifying subgroups beforehand.The paper [28] provides bounds for opinion difference in finite time for a stochastic bounded confidence model.These bounds coincide with the asymptotic behavior.In contrast, here we study transient behavior of the gossip model that is different from the asymptotic behavior.For the gossip model without stubborn agents, [38] shows that X(t) − X η (t) 2 is close to its expectation at the early stage of the process, which provides insight into why both average states and expected average states can be used as references.We will study such concentration in the future.
It is possible to derive bounds similar to Theorem 1 but with slightly different constants for S(X(t), X η (0)+ X ξ (0), ε) and S(X(t), X η (0), ε), that is, the deviation of agent states from average intial states.The next theorem specifies the time intervals over which agent states concentrate around average initial states.Note that parallel results for the expected average states also hold.

Remark 7
The first part of the theorem states that, if λ 3 is large enough, then we can uniformly upper-bound the probability that the minority of regular agents have states different from the initial average opinion of their community over a time interval.The second part indicates that it is possible to uniformly upper-bound the probability of a small portion of regular agents with states away from everyone's average initial state over a time interval, if λ 1 is much smaller than λ 2 and λ 3 .We note that similar results hold for the expected average states.To obtain a nontrivial bound, δ (the portion of regular agents whose states do not concentrate) and γ (the probability that the concentration uniformly happens) need to be small.But ε (the error of the concentration) does not need.For example, it is sufficient to set ε to be less than half of the distance between average states of the two communities, so that the two transient clusters can be distinguished.For a fixed network, setting smaller ε and δ leads to a large bound of γ, because stronger concentration occurs with less probability.
PROOF.The conclusion is obtained by following the same argument of Theorem 1 and then analyzing the obtained upperbounds.See Appendix C. 2 So far we have studied concentration around average states based on conditions of λ i , The following corollary, which is a consequence of Theorem 2, explicitly shows how such phenomena depend on link strength parameters l  d , and l (s) , ( 14) holds for all t ∈ (t 1 , t 1 ) with 0 ), then for large enough n depending on l (r) d and l (s) , ( 16) holds for all t ∈ (t 2 , t 2 ) with PROOF.We follow the notation in Theorem 2. From definition of λ i , it follows that the assumptions of Theorem 2 (i) are equivalent to The condition l d , and l (s) .Hence the conclusion follows from the expression of t 1 and t 1 given in Theorem 2. The proof of (ii) is similar.It suffices to note that (15) is equivalent to The condition l (r) 0 ) ensures that (19) holds with large enough n that depends on l (r) d and l (s) . 2

Remark 8
The first part of the corollary indicates that, if the intra-community weights are of higher order than the inter-community weights and the average weight between regular and stubborn agents, then most agent states concentrate around the initial average opinion of the corresponding community with high probability over a finite time interval.The length of this interval depends on relative strength within communities to that between communities and between regular and stubborn agents.The assumption l (r) d ) of the second part means that intra-community weights are less than or slightly larger than inter-community weights.If stubborn influence is also small, then most agent states are close to everyone's initial average opinion with large probability at the early stage of the process.Note that l (r) d ), so the corollary establishes a sharp phase transition.
The preceding results study the case where the influence of stubborn agents is small.In the following, we investigate the case with large stubborn influence.It is known that gossip model (1) converges to a unique stationary distribution π (that is, if X(0) has distribution π, then X(t) has the same distribution for all t ∈ N + ), if there is at least one stubborn agent and the network is connected [6,39].To characterize the distance between transient and stationary distributions of agent states, we introduce the Wasserstein metric between two measures µ and ν, where J is the set of pairs of random vectors (X, Y ) such that the marginal distribution of X is µ and the marginal distribution of Y is ν.The following theorem provides the time interval over which the distribution of X(t) is close to π.
Theorem 3 Suppose that Assumptions 1-2 hold.Then, for ε ∈ (0, 1), it holds that PROOF.The conclusion is obtained from a coupling argument.See Appendix B. 2 Theorem 3 provides a general characterization for behavior of the gossip model in the long run.The following corollary focuses on the case where the influence of stubborn agents is large.
Remark 9 Corollary 3 indicates that, if the influence of stubborn agents is large, then the distribution of X(t) can be close to the stationary distribution of the gossip model at the early stage of the process.

Discussion and Extension
In this subsection, we first summarize obtained transient behavior under different parameter settings, then discuss the extensions of the results.
In the previous subsection, we obtained several probability bounds for agent states concentrating around average states (Corollary 1), expected average states (Theorem 1), and average initial states (Theorem 2).Note that these bounds are the same except for some constants.So the explicit dependence of transient behavior on link strength parameters l (r) s , l (r) d , and l (s) , given in Corollary 2, still holds for the cases of average states and expected average states.Table 1 summarizes the findings.The results indicate a phase transition phenomenon: the model behaves differently at the early stage of the process under different parameter settings.
We note that the obtained bounds are not tight.As shown in Section 4, expected average states are good references for transients.The concentration occurs for small networks.In contrast, concentration around average initial states requires much larger n.The key idea of studying transient behavior in this paper is to find references such as the expected average states and then to bound the deviation of agent states from such references.Following this framework, it is possible to generalize the obtained results to multiple-community cases.With the help of matrix perturbation theory, it is also possible to deal with the case where communities do not have equal size or the total influences of stubborn agents on regular agents are different (i.e., Assumption 1 (iii) does not hold).Studying the gossip model over an SBM needs analysis of the deviation of the random graph from its expected graph based on matrix concentration inequalities (e.g.[44]).

Numerical Simulation
In this section, we conduct numerical simulations to validate the theoretical results obtained in Section 3.
In the first set of experiments shown in Fig. 2, we demonstrate transient behavior of the gossip model.Set the size of the network n = 60, the bound of agent states c x = 1, and the portion of regular agents r 0 = 0.9.Generate the initial value X i (0) independently from uniform distribution on (0, 1) for all i ∈ V r1 , X j (0) independently from uniform distribution on (−1, 0) for all j ∈ V r2 , and set stubborn-agent states First, we set the edge weight between agents in the same community to be l (r) s = (log 2.5 n)/n, greater than the edge weight between communities l (r) d = (log n)/n.The edge weights between regular and stubborn agents are as follows: for all i ∈ V r1 , l (s) ij = (log n)/n for 1 ≤ j ≤ s 0 n/2, and a ij = 0 for 1 + s 0 n/2 ≤ j ≤ s 0 n; for all i ∈ V r2 , l (s) ij = 0 for 1 ≤ j ≤ s 0 n/2, and a ij = (log n)/n for 1 + s 0 n/2 ≤ j ≤ s 0 n.Hence l (s) = (s 0 log n)/2.This setting intuitively means that the first half of stubborn agents influence only V r1 , whereas the second half influence only V r2 .We use this setting for all numerical (stubborn influence is small, intracommunity influence is moderate or small) Concentrate around everyone's average state/expected average state/average initial state over the time interval (Θ(r0n), Θ(l = (log n)/n, and l (s) = (s0 log 3 n)/2.Most agents move quickly towards stubborn agents during the initial phase, and stay polarized.experiments.Fig. 2(a) shows that agents form two transient clusters corresponding to their community labels and centered around the expected average states within communities.Asymptotic behavior of the system can be different from what shown in Fig. 2(a).As an example, set l (r) s = (log 2.5 n)/n, l (r) d = (log n)/n, and l (s) = (s 0 log 2 n)/2.As in Fig. 2(b), agents in the same community also form transient clusters, but they get close in the end.Next, we set l (r) s = (log 2.5 n)/n, l (r) d = (log 2.4 n)/n, and l (s) = (s 0 log n)/2.In this case, the influence of stubborn agents is small, and inter-community influence strength is similar to intra-community strength.Hence all agents form a single transient cluster and concentrate around everyone's average state, as shown in Fig. 2(c).However, they cannot reach a consensus and keep fluctuating in the long run, because of the presence of stubborn agents.Finally, set l d , and ε are the same as (a).
d , and ε are the same as (a) and n = 500.
d , and ε are the same as (b).Fig. 3.The probability pt of at least δ portion of agents with states at least εcx away from average states at time t.In all experiments, l (s) = (s0 log n)/2.The expected average states are considered in (a-b), whereas the average initial states in (c-e). .Here S ε t (k) is the set S(X(t), E{X η (t) + X ξ (t)}, ε) at the k-th run, that is, the set of agents with states at least εc x away from the expected average state of their communities at time t.The concentration error ε and the portion of non-concentrating agents δ are defined in Theorem 1. Fig. 3(a) shows that p t first decreases with time and then fluctuates around a constant, and it decreases with δ.These findings validate the bound (10).Although the bound (10) is not tight (e.g., the ratio λ 2 /(λ 3 ε 2 δ) ≈ 18.7 > 1), the experiment shows that the concentration still occurs for small networks.Next, we set l (r) d = (log 2.4 n)/n for the case of moderate intracommunity influence.Fig. 3(b) illustrates that the probability varies similarly, validating the bound (11).Now we examine the concentration around average initial states.Fig. 3(c) shows that the concentration around average initial states within communities happens with less probability than that around expected average states shown in Fig. 3(a).Larger networks ensure more concentration, as shown in Fig. 3(d) with n = 500.The concentration around everyone's average initial state, given in Fig. 3(e), is similar to the expected state case.
Finally, for concentration around average initial states, we conduct two experiments to demonstrate how the phase transition occurs when edge weights vary.In the first simulation, we set l

Conclusion
In this paper, we investigated transient behavior of the gossip model with two communities.By analyzing the second moment of agent states, we found a phase transition phenomenon: When edge weights within communities are large and weights between regular and stubborn agents are small, most agents in the same community have states close to the average opinion of that community at the early stage of the process.When the difference between intra-and inter-community weights is small or the inter-community weights are larger than the intra-community ones, most agents in the network have states close to everyone's average opinion.In contrast, if weights between regular and stubborn agents are large, the distribution of agent states is close to the stationary distribution.Future work includes to study the gossip model over general graphs, to investigate transient behavior of other models, and to link theoretical findings to empirical data.

Acknowledgment
This work was supported by the Knut & Alice Wallenberg Foundation and the Swedish Research Council.

Appendix
A Proof of Lemma 4 PROOF OF (5).We divide the proof of (5) into four steps.
Step 1 provides an upper and a lower bound for E{ X η (t) 2 }.Bounds of E{ X ξ (t) 2 } are given in step 2. The lower bounds of E{ X η (t) 2 } and E{ X ξ (t) 2 } are used in step 3, obtaining an upper bound for E{ X Γ (t) 2 }.Step 4 puts everything together and completes the proof.
Step 1 (Upper and lower bounds of E{ X η (t) 2 }).Note that In the last equation, X(t) and z s are taken outside the conditional expectations because of measurability, and the conditional expectations degenerate into expectations because [Q(t), R(t)] is independent of X(t).Let E r be the collection of edges whose end points are regular agents, and let E s be the set of edges connecting regular and stubborn agents.Hence, E = E r ∪ E s .
From the definition of Q(t), we have that, at time t, Q(t) T ηη T Q(t) = ηη T when an edge in E r is selected, and where e i := e 2r 0 nl (s) , where D(s) := 1≤v≤s0n l(s) v e s v (e s v ) T .Hence, (A.1) is Taking expectation yields the following upper and lower bounds for E{ X η (t) 2 } when n ≥ 1/r 0 , Furthermore, by induction and from (A.2), Assumptions 2 and 3, and Lemma 5 at the end of this section, where zs := M z s /l (s) and ζ 1 := η T Rz s /λ 1 = η T M z s /l (s) .
For X ξ (t), it holds that where the last equation is obtained in the same way as (A.1).For Q(t) T ξξ T Q(t), we have that where the last equation is obtained from {u,v}∈Er Cu =Cv Recall that e s i = e (s0n) i . Similary to Step 1, we have that where we denote ξ := Hence when n ≥ 4/r 0 , the following bounds hold Furthermore, by induction and from (A.6), Assumptions 2 and 3, and Lemma 5 at the end of this section, where ζ 2 := ξ T Rz s /λ 1 = ξ T M z s /l (s) .
where the first inequality follows from the definition of the Wasserstein metric, the second inequality from Assumption 2, and the last inequality is obtained by noticing the following fact

Fig. 1 .
Fig. 1.Illustration of Assumption 1.The graph on the left demonstrates the underlying network with two communities (dots and circles) and one stubborn agent (the circle with a cross).Solid lines represent weighted edges.The weights are indicated by line thickness.Edge weights within communities are larger than between communities (l (r) s > l (r) d ).The edge weights between regular agents and the stubborn agent are the same.The rest of the graphs show random interactions between agents, represented by dashed lines.Agents interact more often if they have an edge with a larger weight.
that(17) and(18) hold for large enough n that depends on l The case where l (r) s = (log 2.5 n)/n, l (r) d = (log n)/n, and l (s) = (s0 log n)/2.Most agent states in the same community are close to the expected average state of that community (black dashed lines).
The case where l (r) s = (log 2.5 n)/n, l (r) d = (log n)/n, and l (s) = (s0 log 2 n)/2.Agents behave similar to Fig.2(a), but move towards the origin in the long run.
The case where l (r) s = (log 2.5 n)/n, l (r) d = (log 2.4 n)/n, and l (s) = (s0 log n)/2.Most states are close to everyone's expected average state (the black dashed line), and keep fluctuating.
The case where l (r) s = (log n)/n, l

Fig. 2 .
Fig. 2. Behavior of the gossip model.The figure illustrates the evolution of regular-agent states under four sets of parameters.The color of a trajectory represents the community label of that agent (red represents Vr1, and blue represents Vr2).
n)/n, and l (s) = (s 0 log 3 n)/2.Fig.2(d) illustrates the case where the influence of stubborn agents is large.The figure shows that agent states move quickly towards the positions of stubborn agents.Next we study the probability of concentration around average states, investigated in Theorems 1 and 2. We first study concentration around expected average Concentration around expected average states within communities, where n = 60, l (r) s = (log 2.5 n)/n, l (r) d = (log n)/n, and ε = 0.3.Concentration around everyone's expected average state, where n = 60, l (r) s = (log 2.5 n)/n, l (r) d = (log 2.4 n)/n, and ε = 0.2.Concentration around average initial states within communities, where Concentration around average initial states within communities, where l Concentration around everyone's average initial state, where n, l

Fig. 4 .
Fig. 4. Phase transition of transient behavior of the gossip model.In the experiments, we set l (r) s = (log β 1 n)/n, l (r) d = (log β 2 n)/n, and l (s) = log β 3 n, and compute dc = 2| i∈V r1 Xi(t) − i∈V r2 Xi(t)|/(r0n) with t = n log n .states.Set n = 60, l (r) s = (log 2.5 n)/n, l (r) d = (log n)/n, and l (s) = (s 0 log n)/2 (i.e., intra-community edge weights are large).The model is run for 200 times and the final time step is n(log n) 2 .We estimate the probability p t that the concentration fails by computing 1 200 log β2 n)/n = (log 5 n)/n and l (s) = log n, and consider l (r) s = (log β1 n)/n with β 1 = 1, . . ., 10.We run the gossip model for 50 times, for each value of β 1 and for n = 60, 500, 1000, 2000, and compute the differenced c = 2| i∈Vr1 X i (t) − i∈Vr2 X i (t)|/(r 0 n) with t = nlog n .The difference d c represents the difference between the averages of agent states in the two communities.The case d c = 1 means local concentration whereas d c = 0 represents global concentration.Fig. 4(a) presents the phase transition phenomenon (the effect becomes stronger as n grows).It shows that the local concentration appears when β 1 − β 2 > 0 (i.e., l (r) d = o(l (r) s )), as predicted by Corollary 2. On the other hand, Fig. 4(a) indicates that the global concentration appears when β 1 − β 2 < 0. In the second simulation, we set l (r) s = (log n)/n and l (r) d = (log β2 n)/n = (log 5 n)/n, and consider l (s) = log β3 n with β 3 = 1, . . ., 10.This simulation examines the phase transition in the influence of stubborn agents.Under this circumstance, d c = 2 means regular agents have states close to stubborn ones, whereas d c = 0 represents the global concentration.Fig. 4(b) illustrates that the transition occurs at nl (s) d = Θ(l (s) ) (i.e., β 3 = β 2 ), as predicted by Corollary 3.

w 2 B
(i) (w (i) ) T X(0)r0n i=3 w (i) (w (i) ) T M l (s) z s = X Γ (0) |ζ 3 |,where the first inequality follows from the Cauchy-Schwarz inequality, and the first equation is from the orthogonality of w (i) , 3 ≤ i ≤ r 0 n.Proof of Theorem 3 2. We call C the community structure of the graph.Regular agent i has state X i (t) ∈ R at time t ∈ N, and stubborn agent j has state z s j .Stacking these states, we denote the state vector of regular agents at time t by X(t) ∈ R nr and the state vector of stubborn agents by z s ∈ R ns , where n r := |V r | and n s := |V s |.The random interaction of the gossip model is captured by an interaction probability matrix 2}, and E{ X ⊥ (t) } 2 .The conclusions then follow from Lemma 1. See Appendix A for the details. 2 Remark 3 Lemma 4 provides two bounds for how the second moment of agent states evolve over time.Note that three types of terms appear in the bounds.The first type is an exponentially decreasing term (1 − λ i ) t , indicating the rate of the averaging update.The second one is a linearly increasing term 1 ∧ λ i t, showing cumulative influence of stubborn agents.Lastly, the ratios λ 2 t ∧ (λ 2 /λ 3 ) and λ 1 t∧[λ 1 /(λ 2 ∧λ 3 )] indicate the effect of relative influence strength between the regular and the stubborn.The bounds become trivial for t = ω(1/λ 1 ).Hence they only describe transient behavior.