Finding Cheeger cuts in hypergraphs via heat equation
Introduction
The goal of spectral clustering of graphs is to extract tightly connected communities from a given weighted graph , where is a weight function, using eigenvectors of matrices associated with G. One of the most fundamental results in this area is Cheeger's inequality, which relates the second-smallest eigenvalue of the normalized Laplacian of G and the conductance of G. Here, the (random-walk) normalized Laplacian of G is defined as , where and are the (weighted) adjacency matrix and the (weighted) degree matrix, respectively, of G, that is, is a diagonal matrix with the -th element for being the (weighted) degree of v. Note that all eigenvalues of are non-negative and the smallest eigenvalue is always zero, as , where 1 is the all-one vector and 0 is the zero vector. The conductance of a set is defined as where is the set of edges between S and , and is the volume of S. Intuitively, smaller corresponds to more tightly connected S. The conductance of G is the minimum conductance of a set in G; that is, . Then, Cheeger's inequality [1], [2] states that where is the second-smallest eigenvalue of . The second inequality of (1) is algorithmic in the sense that we can compute a set with conductance of at most , which is called a Cheeger cut, in polynomial time from an eigenvector corresponding to . Moreover, Cheeger's inequality is tight in the sense that computing a set with conductance is NP-hard [3], assuming the small set expansion hypothesis (SSEH) [4].
Several attempts to extend Cheeger's inequality to hypergraphs have been made. To explain the known results, we first extend the concepts of conductance and the normalized Laplacian to hypergraphs. Let be a weighted hypergraph, where is a weight function. The (weighted) degree of a vertex is . For a vertex set , the conductance of S is defined as where is the set of hyperedges intersecting both S and , and has the same definition as that for ordinary graphs. The conductance of G is defined as .
The normalized Laplacian 1 of a hypergraph G [5], [6] is multi-valued and no longer linear (see Section 2 for a detailed definition). This notion of Laplacian is a realization for hypergraphs of more general notion introduced in [6], which is motivated to generalize graph Laplacians to submodular transformations, i.e., vector valued submodular function as natural as possible. In the simplest setting that the hypergraph G is unweighted and d-regular, that is, every vertex has degree d, and the elements of the given vector are pairwise distinct, the acts as follows: We create an undirected graph on V from G by adding for each hyperedge an undirected edge uv, where and , then return . Because the choices of u and v are not unique in general, may be multivalued.
When holds for and , we can state that λ and v are an eigenvalue and an eigenvector, respectively, of . As with the graph case, all eigenvalues of are non-negative and the first eigenvalue is zero as holds. Moreover, the second-smallest eigenvalue exists. Cheeger's inequality for hypergraphs [5], [6] states that Again, the second inequality is algorithmic: If we can compute an eigenvector corresponding to , we can obtain a Cheeger cut; that is, a set with , in polynomial time. Unlike the undirected graph case, however, only an -approximation algorithm is available for computing [6]. Further, it is known that the tight approximation ratio is under the SSEH [5], where r is the maximum cardinality of hyperedges. We remark that it is also known that computing a set with conductance is NP-hard under the SSEH for hypergraphs [5], [7]. Hence, the following natural question arises: Can we compute a Cheeger cut without computing and applying Cheeger's inequality on the corresponding eigenvector?
To answer this question, we consider the following differential equation called the heat equation [5]: where is an initial vector. Intuitively, we gradually diffuse values (or heat) on vertices along hyperedges so that the maximum and minimum values in each hyperedge become closer. We can show that (HE) always has a unique global solution for 2 using the theory of monotone operators and evolution equations [8], [9] (see Section 3 for the details). Let be the solution at time . In particular, holds. In addition, if , we can show that holds for any , and that converges to as when G is connected, where (see [5, Theorem 3.4]). Throughout this paper, we assume that hypergraph G is connected.
We define as the vector for which and for . For a vector , let denote the set of all sweep sets with respect to ; that is, sets of the form either or , for some . We want to show that the conductance of the sweep set of a vector obtained from the heat equation is small. To this end, for , we introduce a key quantity in our analysis: where , which quantifies how fast the heat converges to the limit, that is, π. We can show that is twice the Rayleigh quotient of with respect to the normalized Laplacian , where . This fact, combined with Cheeger's inequality for hypergraphs, implies that captures the minimum conductance of a sweep set obtained from .
Theorem 1 Let be a weighted hypergraph. For any , we have where .
We can show that this is monotonically non-increasing and converges to an eigenvalue of the normalized Laplacian . If the converges to twice of the minimum eigenvalue of the Laplacian for a vector , then by combining Theorem 1, Cheeger's inequality for undirected graphs, and the relation , we have the following corollary: Corollary 2 We assume for a vector . Then, for any , holds for any sufficiently large .
We remark that the assumption in Corollary 2 is mild, because means that the heat is in the orthogonal complement subspace of the eigenvector corresponding to . It is unlikely to occur in practice. It is not clear how to determine initial conditions for v that make the assumption hold. We leave it as a future work.
Although we cannot solve the differential equation (HE) exactly in polynomial time, we can efficiently simulate it by discretizing time using, e.g., the Euler method or the Runge-Kutta method. Indeed these methods have already been used in practice [10]. Alternatively, we can also use difference approximation, developed in the theory of monotone operators and evolution equations [9], to obtain the following: Theorem 3 Let be a weighted hypergraph and , and let and . Then, we can compute a difference approximation of heat such that for every , in time polynomial in , T, and . Here, the precise definition of is in equation (16) in Section 7.
We can bound the conductance of the sweep set obtained from the difference approximation of heat in terms of : Theorem 4 Let be a weighted hypergraph. Assume that the difference approximation obtained by Theorem 3 satisfies . Then, for any , holds, where .
We briefly discuss directed graphs here, as we can show analogues of Theorem 1, Corollary 2, and Theorem 3 for directed graphs with almost the same proofs.
For a directed graph , the degree of a vertex is and the volume of a set is . Note that we do not distinguish out-going and in-coming edges when calculating degrees. Then, the conductance of a set is defined as where and are the sets of edges leaving and entering S, respectively. Then, the conductance of G is . Note that when G is a directed acyclic graph.
Yoshida [10] introduced the notion of a Laplacian for directed graphs and derived Cheeger's inequality, which relates and the second-smallest eigenvalue of the normalized Laplacian of G. As with the hypergraph case, computing is problematic, but we can apply an analogue of Theorem 1 to obtain a set of small conductance without computing . In this paper, we focus on hypergraphs for simplicity of exposition. We remark that by using a different type of Laplacian for directed graphs, Chung [11] gave Cheeger's inequality for the Laplacian, and Andersen et al. [12] introduced an algorithm to compute Cheeger cuts for directed graphs using the personalized PageRank.
Chung [13] presented analogues of Theorem 1 and Corollary 2 for ordinary undirected graphs. Here, we review the proofs of these analogue, because our proofs of Theorem 1 and Corollary 2 extend them partially.
For the undirected graph case, we consider the following single-valued differential equation: This differential equation has a unique global solution . We define a function as When G is connected, converges to π as , irrespective of s; hence, measures the difference between and its unique stationary distribution π. For a set , we define as if and otherwise. Then, we can obtain the inequalities for every , where is the minimum conductance of a sweep set with respect to the vector . From the closed solution of , we observe that . Then, we have Taking the logarithm yields the desired result.
The main obstacle to extending the above argument to hypergraphs is that does not have a closed-form solution as is no longer a linear operator and single-valued. To overcome this, we observe that there exists sequence such that can be regarded as a linear operator in each interval . Here, is the normalized Laplacian of a graph constructed from the hypergraph G and the vector . Then, we can show a counterpart of the second inequality of (3) for each defined as , which is sufficient for our analysis. (We will use another equivalent definition for for convenience. See Section 6 for details.)
Another obstacle is that the triangle inequality applied in the above argument is not true in general, because is not necessarily equal to for the hypergraph case. This fact makes it hard to obtain a counterpart of the first inequality of (3). To overcome this problem, by using the fact that the logarithmic derivative is monotonically non-increasing and considering , we obtain a non-trivial lower bound of the square of norm . Then, we can show that goes to an eigenvalue of the normalized Laplacian as T increases. If is close to the nonzero smallest eigenvalue for a vector , by using the Cheeger inequality (1) for graph and the relation , we obtain a counterpart of the first inequality of (3).
As noted above, an analogue of Theorem 1 for ordinary graphs has been presented by Chung [13]. However, as the normalized Laplacian is a matrix for the graph case, that analysis is much simpler than that presented herein. Kloster and Gleich [14] have presented a deterministic algorithm that approximately simulates the heat equation for graphs. Hence, they extracted a tightly connected subset by considering a local part of the graph only.
The concept of the Laplacian for hypergraphs has been implicitly employed in semi-supervised learning on hypergraphs in the form [15], [16]. This concept was then formally presented by Chan et al. [5] at a later time. Subsequently, the Laplacian concept was further generalized to handle submodular transformations [17], [6]; this development encompasses Laplacians for graphs, hypergraphs [5], directed graphs [10], and directed hypergraphs [18]. In the present paper, we need a precise description of undirected graphs introduced below. To achieve this, we borrow some results in [18, Sections 3 and 4].
Finally, we note that another type of Laplacian for hypergraphs, which essentially replaces each hyperedge with a clique, has been used in the literature [19], [20]. We stress that their Laplacian differs from the Laplacian for hypergraphs studied in this work.
The remainder of this paper is organized as follows. In Section 2, we introduce the basic concepts used throughout this paper. We show that (HE) has a unique global solution in Section 3. In Section 4, we show some basic facts on the heat equation (HE). In Section 5, we show some details of proof of the fact that the compatibility between heat on the hypergraph and its contracted graph. In Section 6, we prove Theorem 1 and Corollary 2. Proofs of Theorem 3 and Theorem 4 are given in Section 7.
Section snippets
Preliminaries
For a vector and a set , let . For a vector and a positive semidefinite matrix , we define and .
Let be a hypergraph. We omit the subscript G from notations such as when it is clear from the context. For a set , let denote the characteristic vector of S, that is, if and otherwise. When or , we simply write 1 and , respectively. For a set , we define a vector as
Existence and uniqueness of solution
We say that is a solution of (HE) if is absolutely continuous with respect to t (hence is differentiable at almost all t) and and satisfies for almost all . In this section, we show the existence and uniqueness of a solution to the heat equation (HE) using the theory of monotone operators. We refer the interested reader to the books by Brezis [22], Miyadera [9], and Showalter [23] for a detailed description of this topic.
We begin by introducing some
Properties of solutions to heat equation
We review some facts on the heat equation (HE). As we saw in Section 3, the heat equation (HE) always has a unique global solution. Also as we mentioned, when G is connected, converges to π as for any with .
For our analysis, it is useful to regard the solution of the heat equation for hypergraph as a sequence of solutions of the heat equation on graphs. From this point of view, our Laplacian behaves like a piecewise linear operator in a sense. This fact enables us to
Proof of Theorem 10
In this section, we show the proof of Theorem 10. We recommend that readers should skip this section on first reading.
Proof of Theorem 10 By Corollary 9, for any initial vector s, there exists a unique solution of (HE). Let . By [18, §.3 and §.4], we can compute any higher right derivatives . Let be the lexicographical ordered equivalence relation on V consistent with . For each , let , , , and be subsets introduced in §.2.1 and §.2.2. Let
Proofs of Theorem 1 and some corollaries
In this section, we prove Theorem 1 and some corollaries. We show them as follows:
- •
We define a key function and prove that is same as the square of norm of difference between heat and the stationary distribution on the graph obtained by contracting hypergraph G (Section 6.1).
- •
We prove that the logarithmic derivative of can be described as a Rayleigh quotient of heat on and is monotonically non-decreasing (Section 6.2).
- •
By proving the compatibility of norms on G and ,
Computation and error analysis of difference approximation
In this section, we prove Theorem 3 and Theorem 4. In what follows, we fix a hypergraph , , , and .
We first review the construction of the difference approximation given in [9, Section 5.3]. By the maximality of and the fact in [9, Lemma 5.1], for any and , there exist , , and that satisfy the following:
- (i)
with ,
- (ii)
,
- (iii)
,
Declaration of Competing Interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
Acknowledgements
This work was supported by JSPS KAKENHI Grant Numbers 18H05291, 19K20218, and 21K11763 and Grant for Basic Science Research Projects from The Sumitomo Foundation Grant Number 200484.
References (25)
- et al.
, isoperimetric inequalities for graphs, and superconcentrators
J. Comb. Theory, Ser. B
(1985) - et al.
Diffusion operator and spectral analysis for directed hypergraph Laplacian
Theor. Comput. Sci.
(2019) - et al.
Polynomial-time algorithms for submodular Laplacian systems
Theor. Comput. Sci.
(2021) Eigenvalues and expanders
Combinatorica
(1986)- et al.
Reductions between expansion problems
- et al.
Graph expansion and the unique games conjecture
- et al.
Spectral properties of hypergraph Laplacian and approximation algorithms
J. ACM
(2018) Cheeger inequalities for submodular transformations
- et al.
Approximation algorithms for hypergraph small-set expansion and small-set vertex expansion
Theory Comput.
(2016) Nonlinear semi-groups in Hilbert space
J. Math. Soc. Jpn.
(1967)
Nonlinear Semigroups, vol. 109
Nonlinear Laplacian for digraphs and its applications to network analysis
Cited by (7)
Extended fractional-polynomial generalizations of diffusion and Fisher–KPP equations on directed networks
2023, Chaos, Solitons and FractalsCHENG’S MAXIMAL DIAMETER THEOREM FOR HYPERGRAPHS
2023, Tohoku Mathematical JournalNonlinear evolution equation associated with hypergraph Laplacian
2023, Mathematical Methods in the Applied Sciences