Elsevier

Theoretical Computer Science

Volume 930, 21 September 2022, Pages 1-23
Theoretical Computer Science

Finding Cheeger cuts in hypergraphs via heat equation

https://doi.org/10.1016/j.tcs.2022.07.006Get rights and content

Highlights

  • We consider the heat equation on hypergraphs, which is a differential equation exploiting the normalized Laplacian.

  • We show theoretically that the heat equation has a unique global solution.

  • We also show that we can extract a subset with conductance ϕG from the solution under a mild condition.

  • We can compute a differential approximation of heat and estimate the conductance obtained by sweeping this approximation.

Abstract

Cheeger's inequality states that a tightly connected subset can be extracted from a graph G using an eigenvector of the normalized Laplacian associated with G. More specifically, we can compute a vertex subset in G with conductance O(ϕG), where ϕG is the minimum conductance of G. It has recently been shown that Cheeger's inequality can be extended to hypergraphs. However, as the normalized Laplacian of a hypergraph is no longer a matrix, we can only approximate its eigenvectors; this causes a loss in the conductance of the obtained subset. To address this problem, we here consider the heat equation on hypergraphs, which is a differential equation exploiting the normalized Laplacian. We show that the heat equation has a unique global solution and that we can extract a subset with conductance ϕG from the solution under a mild condition. An analogous result also holds for directed graphs.

Introduction

The goal of spectral clustering of graphs is to extract tightly connected communities from a given weighted graph G=(V,E,w), where w:ER+ is a weight function, using eigenvectors of matrices associated with G. One of the most fundamental results in this area is Cheeger's inequality, which relates the second-smallest eigenvalue of the normalized Laplacian of G and the conductance of G. Here, the (random-walk) normalized Laplacian of G is defined as LG=IAGDG1, where AGRV×V and DGRV×V are the (weighted) adjacency matrix and the (weighted) degree matrix, respectively, of G, that is, DG is a diagonal matrix with the (v,v)-th element for vV being the (weighted) degree dG(v):=eE|vew(e) of v. Note that all eigenvalues of LG are non-negative and the smallest eigenvalue is always zero, as LG(DG1)=0, where 1 is the all-one vector and 0 is the zero vector. The conductance of a set SV is defined asϕG(S):=eG(S)w(e)min{volG(S),volG(VS)}, where G(S) is the set of edges between S and VS, and volG(S):=vSdG(v) is the volume of S. Intuitively, smaller ϕG(S) corresponds to more tightly connected S. The conductance of G is the minimum conductance of a set in G; that is, ϕG:=minSVϕG(S). Then, Cheeger's inequality [1], [2] states thatλG2ϕG2λG, where λGR+ is the second-smallest eigenvalue of LG. The second inequality of (1) is algorithmic in the sense that we can compute a set SV with conductance of at most 2λG=O(ϕG), which is called a Cheeger cut, in polynomial time from an eigenvector corresponding to λG. Moreover, Cheeger's inequality is tight in the sense that computing a set with conductance o(ϕG) is NP-hard [3], assuming the small set expansion hypothesis (SSEH) [4].

Several attempts to extend Cheeger's inequality to hypergraphs have been made. To explain the known results, we first extend the concepts of conductance and the normalized Laplacian to hypergraphs. Let G=(V,E,w) be a weighted hypergraph, where w:ER+ is a weight function. The (weighted) degree of a vertex vV is dG(v):=eE|vew(e). For a vertex set SV, the conductance of S is defined asϕG(S):=eG(S)w(e)min{volG(S),volG(VS)}, where G(S) is the set of hyperedges intersecting both S and VS, and volG(S) has the same definition as that for ordinary graphs. The conductance of G is defined as ϕG:=minSVϕG(S).

The normalized Laplacian LG:RV2RV1 of a hypergraph G [5], [6] is multi-valued and no longer linear (see Section 2 for a detailed definition). This notion of Laplacian is a realization for hypergraphs of more general notion introduced in [6], which is motivated to generalize graph Laplacians to submodular transformations, i.e., vector valued submodular function as natural as possible. In the simplest setting that the hypergraph G is unweighted and d-regular, that is, every vertex has degree d, and the elements of the given vector xRV are pairwise distinct, the LG acts as follows: We create an undirected graph Gx on V from G by adding for each hyperedge eE an undirected edge uv, where uargminuex(u) and vargmaxvex(v), then return LGxx. Because the choices of u and v are not unique in general, LG may be multivalued.

When LG(v)λv holds for λR and v0, we can state that λ and v are an eigenvalue and an eigenvector, respectively, of LG. As with the graph case, all eigenvalues of LG are non-negative and the first eigenvalue is zero as LG(DG1)=0 holds. Moreover, the second-smallest eigenvalue λGR+ exists. Cheeger's inequality for hypergraphs [5], [6] states thatλG2ϕG2λG. Again, the second inequality is algorithmic: If we can compute an eigenvector corresponding to λG, we can obtain a Cheeger cut; that is, a set SV with ϕG(S)=O(ϕG), in polynomial time. Unlike the undirected graph case, however, only an O(logn)-approximation algorithm is available for computing λG [6]. Further, it is known that the tight approximation ratio is O(logr) under the SSEH [5], where r is the maximum cardinality of hyperedges. We remark that it is also known that computing a set with conductance o(ϕHlogr/r) is NP-hard under the SSEH for hypergraphs [5], [7]. Hence, the following natural question arises: Can we compute a Cheeger cut without computing λG and applying Cheeger's inequality on the corresponding eigenvector?

To answer this question, we consider the following differential equation called the heat equation [5]:dρtdtLG(ρt)andρ0=s, where sRV is an initial vector. Intuitively, we gradually diffuse values (or heat) on vertices along hyperedges so that the maximum and minimum values in each hyperedge become closer. We can show that (HE) always has a unique global solution for t02 using the theory of monotone operators and evolution equations [8], [9] (see Section 3 for the details). Let ρtsRV be the solution at time t0. In particular, ρ0s=s holds. In addition, if vVs(v)=1, we can show that vVρts(v)=1 holds for any t0, and that ρts converges to πRV as t when G is connected, where π(v):=dG(v)/vol(V) (see [5, Theorem 3.4]). Throughout this paper, we assume that hypergraph G is connected.

We define πvRV as the vector for which πv(v)=1 and πv(u)=0 for uv. For a vector xRV, let sweep(x) denote the set of all sweep sets with respect to DG1x; that is, sets of the form either {vV|x(v)/dG(v)τ} or {vV|x(v)/dG(v)τ}, for some τR. We want to show that the conductance of the sweep set of a vector obtained from the heat equation is small. To this end, for T0, we introduce a key quantity in our analysis:gv(T)=ddtlogρtπvπDG12|t=T, where xDG12:=xD1x, which quantifies how fast the heat converges to the limit, that is, π. We can show that gv(T) is twice the Rayleigh quotient of DG1/2(ρTπvπ) with respect to the normalized Laplacian xDG1/2LG(DG1/2(x)), where LG(x)=LG(DGx). This fact, combined with Cheeger's inequality for hypergraphs, implies that gv(T) captures the minimum conductance of a sweep set obtained from ρTπv.

Theorem 1

Let G=(V,E,w) be a weighted hypergraph. For any T>0, we havegv(T)(κTv)2 where κTv:=min{ϕG(S)|Ssweep(ρTπv)}.

We can show that this gv(T) is monotonically non-increasing and converges to an eigenvalue of the normalized Laplacian DG1/2LGDG1/2. If the gv(T) converges to twice of the minimum eigenvalue λGx of the Laplacian DGx1/2LGxDGx1/2 for a vector xRV, then by combining Theorem 1, Cheeger's inequality for undirected graphs, and the relation ϕGxϕG, we have the following corollary:

Corollary 2

We assume limTgv(T)=2λGx for a vector xRV. Then, for any ε>0,4ϕG+ε(κTv)2 holds for any sufficiently large T>0.

Hence, Corollary 2 implies that, when T is sufficiently large, under the assumption in the statement, we can obtain a set SV such that ϕG(S)=O(ϕG), thereby avoiding the problem of computing the second smallest eigenvalue λG of the hypergraph normalized Laplacian LG. Algorithm 1 gives a pseudocode of our algorithm.

We remark that the assumption in Corollary 2 is mild, because limTgv(T) 2λGx means that the heat ρTπvπ is in the orthogonal complement subspace of the eigenvector uGx corresponding to λGx. It is unlikely to occur in practice. It is not clear how to determine initial conditions for v that make the assumption hold. We leave it as a future work.

Although we cannot solve the differential equation (HE) exactly in polynomial time, we can efficiently simulate it by discretizing time using, e.g., the Euler method or the Runge-Kutta method. Indeed these methods have already been used in practice [10]. Alternatively, we can also use difference approximation, developed in the theory of monotone operators and evolution equations [9], to obtain the following:

Theorem 3

Let G=(V,E,w) be a weighted hypergraph and vV, and let T1 and λ(0,1). Then, we can compute a difference approximation {ρtλ}0tT of heat ρtπv such that ρtπvρtλDG1=O(λT) for every 0tT, in time polynomial in 1/λ, T, and eE|e|. Here, the precise definition of {ρtλ}0tT is in equation (16) in Section 7.

We can bound the conductance of the sweep set obtained from the difference approximation ρtλ of heat in terms of gv(t):

Theorem 4

Let G=(V,E,w) be a weighted hypergraph. Assume that the difference approximation ρtλ obtained by Theorem 3 satisfies ρtπvρtλDG1<δ. Then, for any 0<t<T,gv(t)(1+O(δ))(κtv,λ)2 holds, where κtv,λ:=min{ϕG(S)|Ssweep(ρtλ)}.

We briefly discuss directed graphs here, as we can show analogues of Theorem 1, Corollary 2, and Theorem 3 for directed graphs with almost the same proofs.

For a directed graph G=(V,E,w), the degree of a vertex vV is dG(v)=eE|vew(e) and the volume of a set SV is volG(S)=vSdG(v). Note that we do not distinguish out-going and in-coming edges when calculating degrees. Then, the conductance of a set SV is defined asϕG(S):=min{eG+(S)w(e),eG(S)w(e)}min{volG(S),volG(VS)}, where G+(S) and G(S) are the sets of edges leaving and entering S, respectively. Then, the conductance of G is ϕG:=minSVϕG(S). Note that ϕG=0 when G is a directed acyclic graph.

Yoshida [10] introduced the notion of a Laplacian for directed graphs and derived Cheeger's inequality, which relates ϕG and the second-smallest eigenvalue λG of the normalized Laplacian of G. As with the hypergraph case, computing λG is problematic, but we can apply an analogue of Theorem 1 to obtain a set of small conductance without computing λG. In this paper, we focus on hypergraphs for simplicity of exposition. We remark that by using a different type of Laplacian for directed graphs, Chung [11] gave Cheeger's inequality for the Laplacian, and Andersen et al. [12] introduced an algorithm to compute Cheeger cuts for directed graphs using the personalized PageRank.

Chung [13] presented analogues of Theorem 1 and Corollary 2 for ordinary undirected graphs. Here, we review the proofs of these analogue, because our proofs of Theorem 1 and Corollary 2 extend them partially.

For the undirected graph case, we consider the following single-valued differential equation:dρtdt=LGρtandρ0=s. This differential equation has a unique global solution ρts=exp(tLG)s. We define a function fs:R+R asfs(t)=ρt/2sπDG12. When G is connected, ρts converges to π as t, irrespective of s; hence, fs measures the difference between ρt/2s and its unique stationary distribution π. For a set SV, we define πSRV as πS(v)=dG(v)/vol(S) if vS and πS(v)=0 otherwise. Then, we can obtain the inequalitiesexp(O(ϕG(S)t))fπS(t)exp(Ω((κtπS)2t)) for every SV, where κtπS is the minimum conductance of a sweep set with respect to the vector (ρtπS(v)/dG(v))vV. From the closed solution of ρts, we observe that ρt/2πS=vSdG(v)vol(S)ρt/2πv. Then, we haveexp(O(ϕG(S)t))fπS(t)=ρt/2πSπDG12(vSdG(v)vol(S)ρt/2πvπDG1)2maxvSρt/2πvπDG12=maxvSfπv(t)maxvSexp(Ω((κtπv)2t)). Taking the logarithm yields the desired result.

The main obstacle to extending the above argument to hypergraphs is that ρt does not have a closed-form solution as LG is no longer a linear operator and single-valued. To overcome this, we observe that there exists sequence t0=0<t1<t2< such that LG can be regarded as a linear operator Li in each interval [ti,ti+1). Here, Li is the normalized Laplacian of a graph constructed from the hypergraph G and the vector ρti. Then, we can show a counterpart of the second inequality of (3) for each fis:R+R defined as fis(Δ)=ρti+Δ/2sπDG12, which is sufficient for our analysis. (We will use another equivalent definition for fis for convenience. See Section 6 for details.)

Another obstacle is that the triangle inequality applied in the above argument is not true in general, because ρt/2πS is not necessarily equal to vSdG(v)vol(S)ρt/2πv for the hypergraph case. This fact makes it hard to obtain a counterpart of the first inequality of (3). To overcome this problem, by using the fact that the logarithmic derivative gv(t) is monotonically non-increasing and considering t>T>0, we obtain a non-trivial lower bound exp(O(gv(T)(tT))) of the square of norm ρtπvπDG12. Then, we can show that gv(T) goes to an eigenvalue of the normalized Laplacian LG as T increases. If gv(T) is close to the nonzero smallest eigenvalue λGx for a vector xRV, by using the Cheeger inequality (1) for graph Gx and the relation ϕGxϕG, we obtain a counterpart of the first inequality of (3).

As noted above, an analogue of Theorem 1 for ordinary graphs has been presented by Chung [13]. However, as the normalized Laplacian LG=IAGDG1 is a matrix for the graph case, that analysis is much simpler than that presented herein. Kloster and Gleich [14] have presented a deterministic algorithm that approximately simulates the heat equation for graphs. Hence, they extracted a tightly connected subset by considering a local part of the graph only.

The concept of the Laplacian for hypergraphs has been implicitly employed in semi-supervised learning on hypergraphs in the form xLG(x) [15], [16]. This concept was then formally presented by Chan et al. [5] at a later time. Subsequently, the Laplacian concept was further generalized to handle submodular transformations [17], [6]; this development encompasses Laplacians for graphs, hypergraphs [5], directed graphs [10], and directed hypergraphs [18]. In the present paper, we need a precise description of undirected graphs G˜i introduced below. To achieve this, we borrow some results in [18, Sections 3 and 4].

Finally, we note that another type of Laplacian for hypergraphs, which essentially replaces each hyperedge with a clique, has been used in the literature [19], [20]. We stress that their Laplacian differs from the Laplacian for hypergraphs studied in this work.

The remainder of this paper is organized as follows. In Section 2, we introduce the basic concepts used throughout this paper. We show that (HE) has a unique global solution in Section 3. In Section 4, we show some basic facts on the heat equation (HE). In Section 5, we show some details of proof of the fact that the compatibility between heat on the hypergraph and its contracted graph. In Section 6, we prove Theorem 1 and Corollary 2. Proofs of Theorem 3 and Theorem 4 are given in Section 7.

Section snippets

Preliminaries

For a vector xRV and a set SV, let x(S)=vSx(v). For a vector x,yRV and a positive semidefinite matrix ARV×V, we define x,yA=xAy and xA=x,xA=xAx.

Let G=(V,E,w) be a hypergraph. We omit the subscript G from notations such as AG when it is clear from the context. For a set SV, let 1SRV denote the characteristic vector of S, that is, 1S(v)=1 if vS and 1S(v)=0 otherwise. When S=V or S={v}, we simply write 1 and 1v, respectively. For a set SV, we define a vector πSRV as πS(v)=dG(v)

Existence and uniqueness of solution

We say that {ρt}t0 is a solution of (HE) if ρt is absolutely continuous with respect to t (hence ρt is differentiable at almost all t) and ρ0=s and satisfies ddtρtLG(ρt) for almost all t0. In this section, we show the existence and uniqueness of a solution to the heat equation (HE) using the theory of monotone operators. We refer the interested reader to the books by Brezis [22], Miyadera [9], and Showalter [23] for a detailed description of this topic.

We begin by introducing some

Properties of solutions to heat equation

We review some facts on the heat equation (HE). As we saw in Section 3, the heat equation (HE) always has a unique global solution. Also as we mentioned, when G is connected, ρt converges to π as t for any sRV with vVs(v)=1.

For our analysis, it is useful to regard the solution of the heat equation for hypergraph as a sequence of solutions of the heat equation on graphs. From this point of view, our Laplacian LG behaves like a piecewise linear operator in a sense. This fact enables us to

Proof of Theorem 10

In this section, we show the proof of Theorem 10. We recommend that readers should skip this section on first reading.

Proof of Theorem 10

By Corollary 9, for any initial vector s, there exists a unique solution ρts of (HE). Let μts=DG1ρts. By [18, §.3 and §.4], we can compute any higher right derivatives dnμtsdtn|t=0. Let (σ,) be the lexicographical ordered equivalence relation on V consistent with {dnμts/dtn|t=0}n.

For each eE, let Se, Ie, Seσ, and Ieσ be subsets introduced in §.2.1 and §.2.2. Let G=(V,E,w

Proofs of Theorem 1 and some corollaries

In this section, we prove Theorem 1 and some corollaries. We show them as follows:

  • We define a key function fi(Δ) and prove that fi(Δ) is same as the square of norm of difference between heat and the stationary distribution on the graph G˜i obtained by contracting hypergraph G (Section 6.1).

  • We prove that the logarithmic derivative of fi(Δ) can be described as a Rayleigh quotient of heat on G˜i and is monotonically non-decreasing (Section 6.2).

  • By proving the compatibility of norms on G and G˜i,

Computation and error analysis of difference approximation

In this section, we prove Theorem 3 and Theorem 4. In what follows, we fix a hypergraph G=(V,E,w), vV, T1, and λ(0,1).

We first review the construction of the difference approximation ρtλ given in [9, Section 5.3]. By the maximality of LG and the fact in [9, Lemma 5.1], for any λ>0 and x0RV, there exist {tkλ}, {xkλ}, and {ykλ} that satisfy the following:

  • (i)

    0=t0<t1<<tkλ<tk+1λ< with limktkλ=,

  • (ii)

    tkλtk1λλ(k=1,2,),

  • (iii)

    xkλxk1λ(tkλtk1λ)ykλDG1<λ(tkλtk1λ)(k=1,2,),

where ykλLG(xkλ) for k=1,

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Acknowledgements

This work was supported by JSPS KAKENHI Grant Numbers 18H05291, 19K20218, and 21K11763 and Grant for Basic Science Research Projects from The Sumitomo Foundation Grant Number 200484.

References (25)

  • I. Miyadera

    Nonlinear Semigroups, vol. 109

    (1992)
  • Y. Yoshida

    Nonlinear Laplacian for digraphs and its applications to network analysis

  • Cited by (7)

    View all citing articles on Scopus
    View full text