Hypercore decomposition for non-fragile hyperedges: concepts, algorithms, observations, and applications

Abstract

Hypergraphs are a powerful abstraction for modeling high-order relations, which are ubiquitous in many fields. A hypergraph consists of nodes and hyperedges (i.e., subsets of nodes); and there have been a number of attempts to extend the notion of ${\varvec{k}}$-cores, which proved useful with numerous applications for pairwise graphs, to hypergraphs. However, the previous extensions are based on an unrealistic assumption that hyperedges are fragile, i.e., a high-order relation becomes obsolete as soon as a single member leaves it.In this work, we propose a new substructure model, called ${\varvec{(k,t)}}$-hypercore, based on the assumption that high-order relations remain as long as at least t fraction of the members remains. Specifically, it is defined as the maximal subhypergraph where (1) every node is contained in at least ${\varvec{k}}$ hyperedges in it and (2) at least ${\varvec{t}}$ fraction of the nodes remain in every hyperedge. We first prove that, given ${\varvec{t}}$ (or ${\varvec{k}}$), finding the ${\varvec{(k,t)}}$-hypercore for every possible ${\varvec{k}}$ (or ${\varvec{t}}$) can be computed in time linear w.r.t the sum of the sizes of hyperedges. Then, we demonstrate that real-world hypergraphs from the same domain share similar ${\varvec{(k,t)}}$-hypercore structures, which capture different perspectives depending on ${\varvec{t}}$. Lastly, we show the successful applications of our model in identifying influential nodes, dense substructures, and vulnerability in hypergraphs.

Appendices

A. Proofs

1.1 A.1 Proof of proposition 1

Proof

Since H is finite, the number of subhypergraphs of H is also finite. Therefore, there exists one subhypergraph with maximal total size (which is possibly an empty hypergraph) where each node has degree at least k and at least t proportion of the constituent nodes remain in each hyperedge, completing the proof of existence. To show the uniqueness, suppose the opposite, and let $C^1 = (V^1, E^1)$ and $C^2 = (V^2, E^2)$ be two distinct (k, t)-hypercores of H. Then we consider the hypergraph $C' = (V', E')$ with $E' = \{e^1_i \cup e^2_i: i \in I_{E^1} \cup I_{E^2}\}$. Clearly, $C'$ is a subhypergraph of H with a larger total size that satisfies the node-degree and hyperedge-fraction conditions, which contradicts the maximality and completes the proof. $\square$

1.2 A.2 Proof of proposition 2

Proof

Suppose that $C_{k, t_2}(H)$ is not a subhypergraph of $C_{k, t_1}(H)$. Then we take the union $C_{k, t_2}(H) \cup C_{k, t_1}(H)$ and we obtain a hypergraph that is strictly larger than $C_{k, t_1}(H)$ and satisfies the conditions of $(k, t_1)$-hypercore, which contradicts with the maximality, completing the proof. The second statement can be proved similarly. $\square$

1.3 A.3 Proof of lemma 1

Proof

This equivalence is immediate by two facts. First, for each node $v \in V$, the degree of v in H is equal to the degree of v in $G_{bp}(H)$. Second, for each hyperedge $e \in E$, the number of nodes in e is equal to the degree of e in $G_{bp}(H)$. With the above two facts, this equivalence immediately follows. $\square$

1.4 A.4 Proof of lemma 2

Proof

By Definition  5, when $t = 0$, the definition of $C_{k; t = 0}(H)$ is the maximal subhypergraph of H where (1) every node in $C_{k, t=0}(H)$ has degree at least k and (2) at least two nodes remain in every hyperedge of $C_{k, t=0}(H)$. Such a definition exactly coincides with ${\tilde{C}}_{k; \ell = 2}(H)$, completing the proof. $\square$

1.5 A.5 Proof of lemma 3

We can understand the differences between $(k; \ell )$-hypercores and (k, t)-hypercores by two intuitions. When we obtain the $(k; \ell )$-hypercore of a given H with $\ell > 2$, all hyperedges of cardinality 2 are removed in the first place. Therefore, if we want to find an $\ell$ such that ${\tilde{C}}_{k; \ell } = C_{k, t}$ where $C_{k, t}$ contains any hyperedge of cardinality 2, the only possible $\ell$ value is $\ell = 2$. Since the threshold in the (k, t)-hypercore is proportional, it imposes different absolute cardinality thresholds for hyperedges of different sizes. On the contrary, the $(k; \ell )$-hypercore imposes the same absolute cardinality threshold for each hyperedge. We shall show two counterexamples from the two intuitions above.

Proof

Consider $H = (V, E)$ with $E = \{\{1, 2\}, \{1, 3\} \{1, 2, 3, 4\}, \{1, 3, 4, 5, 6\}\}.$ The $(k=2, t=3/4)$-hypercore of H consists of the hyperedges $\{\{1,2\}, \{1,3\}, \{1, 2, 3\}\},$ where the hyperedge $\{1,3,4,5,6\}$ is totally removed since only $3/5 < t = 3/4$ of the constituent nodes remain by the node-degree threshold $k = 2$. For the $(k; \ell )$-hypercore, the $(k=2; \ell =2)$-hypercore of H consists of the hyperedges $\{\{1, 2\}, \{1, 3\}, \{1, 2, 3, 4\}, \{1, 3, 4\}\}$; the $(k=2; \ell =3)$-hypercore of H consists of the hyperedges $\{\{1, 3, 4\}, \{1, 3, 4\}\}$; when $\ell \ge 4$, the $(k=2;\ell )$-hypercore of H is empty, completing the proof.

We show another counterexample. Consider $H = (V, E)$ with $E = \{\{1,2,3,4\}, \{1,2,5,6\}, \{5,6,7,8\},\{3,4,9,10,11\},\{1,2,3,4,5,6,7,8\}\}.$ The $(k=3, t=1/2)$-hypercore of H consists of the hyperedges $\{\{1,2\}, \{1,2,5,6\}, \{5,6\}, \{1,2,5,6\}\}.$ For the $(k; \ell )$-hypercore, when $\ell = 2$, the $(k=3; \ell =2)$-hypercore of H consists of the hyperedges $\{\{1,2,3,4\},\{1,2,5,6\},\{5,6\},\{3,4\},\{1,2,3,4,5,6\}\};$ when $\ell \ge 3$, the $(k=3; \ell )$-hypercore of H is empty, completing the proof. $\square$

Remark 1

Our proposed (k, t)-hypercore allows arbitrarily fine-grained adjustment since the value of t is continuous in [0, 1], while $\ell$ must be an integer. In real-world hypergraphs, many hyperedges are of cardinality 2. Therefore, the $(k; \ell )$-hypercore with $\ell > 2$ is significantly less meaningful than the (k, t)-hypercore since many hyperedges are not taken into consideration at all. See Tbl. 5 for the detailed number of hyperedges of different cardinality in each dataset we have used. See Figs. 8 and 9 for the performance of hypercoreness w.r.t $(k; \ell )$-hypercore to indicate the influence of nodes. Note again that the $(k; \ell =2)$-hypercore is included in our proposed concept as the $(k, t=0)$-hypercore. We observe that in most datasets, the $(k; \ell )$-hypercores become less meaningful and fail to indicate the influence of nodes when $\ell$ becomes large, as expected.

1.6 A.6 Proof of theorem 1

Proof

Correctness. The size of a hyperedge changes only when some node in ${\mathcal {R}}$ is removed from it, and the degree of a node changes only when some incident hyperedge is removed. Therefore, when Algorithm 1 ends, each node has degree at least k, otherwise it must have been included in ${\mathcal {R}}$ and removed, and each hyperedge satisfies the hyperedge-fraction condition, otherwise it must have been removed. This implies that the output of Algorithm 1 satisfies both the node-degree and hyperedge-fraction conditions w.r.t H, k, and t. We now show the maximality. Suppose not, and let $(v, e_i)$ be the first node-hyperedge pair that appears during the process of Algorithm 1 with $v \in e_i \in E(C_{k, t})$ but $v \notin e_i' \in E'$, where $C' = (V', E')$ is the returned hypergraph. This implies that v is removed from e, and thus v is included in ${\mathcal {R}}$ because its degree has been below k. However, by the definition of the (k, t)-hypercore and the assumption that $(v, e_i)$ is the first pair, before the deletion, the degree of v is at least k, which completes the proof by contradiction.

Time complexity. We assume the input hypergraph has been loaded in the memory and thus do not count the complexity of loading the hypergraph. Checking the initial degrees (Line 1) takes $O(|V|)$. In the while loop, each node is added to the set of nodes to be removed at most once since each node is added exactly when its degree decreases from k to $k - 1$. Therefore, this process takes $O(|V|)$. By checking the incident edges of each node in ${\mathcal {R}}$, we find all $e'_i$s intersecting with ${\mathcal {R}}$, which takes $O(|{\mathcal {R}}|) = O(|V|)$. Hash tables are used to implement the sets. Before a hyperedge $e \in E$ is totally removed, at least $\max \left( \lceil t|e| \rceil , 2\right)$ nodes remain in it (otherwise it has been removed earlier), and thus it can be visited at most $|e| - \max \left( \lceil t|e| \rceil , 2\right) + 1$ times (because one node is removed at each time). This process takes $O(\sum _{e \in E} (|e| - \max \left( \lceil t|e| \rceil , 2\right) + 1)) = O(|E| + (1 - t) \sum _{e \in E} |e|)$. Therefore, the total time complexity is $O(|V|) + O(|E| + (1 - t) \sum _{e \in E} |e|) = O(|E| + (1 - t) \sum _{e \in E} |e|)$. $\square$

1.7 A.7 Proof of theorem 2

Proof

Correctness. For each node v, the assignment of $c_t(v)$ happens only once when $v \in {\mathcal {R}}$, i.e., before its deletion. By Theorem 1, $c_t(v) = k - 1$ implies that v is not in the (k, t)-hypercore but in the previous $(k', t)$-hypercore where each node has degree at least $k - 1$, i.e., v is in the $(k-1, t)$-hypercore.

Time complexity. The values of k increases $O(c_t^*)$ times, thus the process in Lines 4 and 5 is repeated for $O(c_t^*)$ times and takes $O(c_t^* |V|)$. The assignment of t-hypercoreness of each node (Line 7) takes $O(|V|)$. As shown in the proof of Theorem 1, each hyperedge is visited at most $|e| - \max (\lceil t|e| \rceil , 2) + 1$ times before being deleted and each node is added to the set of nodes to be removed only once. Therefore, the remaining process takes $O(|V| + |E| + (1 - t) \sum _{e \in E} |e|)$. $\square$

1.8 A.8 Proof of theorem 3

Proof

Correctness. For each node v, the assignment of $f_k(v)$ happens only once when $v \in {\mathcal {R}}$, i.e., before its deletion. By Theorem 1, $f_k(v) = t$ implies that v is in the (k, t)-hypercore with degree k and is in at least one hyperedge that is in the (k, t)-hypercore but not in any $(k, t')$-hypercore with $t' > t$. Therefore, v is not in any $(k, t')$-hypercore with $t' > t$.

Time complexity. Recording the hyperedge sizes (Line 1) takes $O(|E|)$. By Theorem 1, computing $C_{k, 0}$ (Line 2) takes $O(\sum _{e \in E} |e|)$. As shown in the previous proofs, the while loop (Lines 4 to 12) takes $O(|V| + |E| + \sum _{e \in E} |e|) = O(\sum _{e \in E} |e|)$. $\square$

Details of datasets

In this section, we provide more details of the datasets used in our experiments.

• Coauth-DBLP/Geology. In these two coauthorship hypergraphs, each hyperedge represents a publication, and the constituent nodes of a hyperedge represent the authors of the corresponding publication.

• NDC-classes/substances. In these two hypergraphs from the National Drug Code (NDC) Directory, each hyperedge represents a drug (with its unique NDC code), and the constituent nodes of a hyperedge represent the class labels (for NDC-classes) or the ingredients (for NDC-substances) of the drug.

• Contact-high/primary. In these two contact hypergraphs, each hyperedge represents a group of interacting individuals (the constituent nodes) within a predetermined time period.

• Email-Enron/Eu. In these two email hypergraphs, each hyperedge represents an email (possibly sent to multiple people individually at the same time), which contains the sender and all the receivers as its constituent nodes.

• Tags-ubuntu/math/SO. In these three tags hypergraphs from https://stackoverflow.com/, each node represents a tag, and each hyperedge represents a question, where each constituent node represents a tag applied to the question.

• Threads-ubuntu/math/SO. In these three threads hypergraphs also from https://stackoverflow.com/, each hyperedge represents a thread, where each constituent node represents a person that participates in it.

We have used the preprocessed version of the datasets where each hyperedge consists of at most 25 nodes. In Table 5, we report the number of hyperedges of different cardinality in each dataset. Notably, on https://www.cs.cornell.edu/~arb/data/, the full version of the datasets, in which the cardinality of the hyperedges is not limited, is also available.

Table 5 The number of hyperedges of different cardinality in each dataset. For each dataset, we list the number of hyperedges of each specific size. Specifically, in most datasets, a large number of hyperedges are of cardinality 2. We use $E_s$ to denote the set of hyperedges of cardinality s, for each s

Table 6 For each dataset, we report the information gain over degree, for each of the considered quantities: $\ell$-hypercoreness with $\ell \in \{3, 4, 5\}$, neighbor-hypercoreness, and neighbor-degree-hypercoreness

Additional experimental results

In this section, we provide additional experimental results supplementing the main text. In Fig. 15, we report the results regarding the statistical difference between t-hypercoreness and other centrality measures, as well as among t-hypercoreness with different t, on the datasets not covered in the main text. In Fig. 16, we report the results regarding the information gain over degree, on the datasets not covered in the main text. In Table 6, for each dataset, we report the information gain over degree for the following quantities: $\ell$-hypercoreness with $\ell \in \{3, 4, 5\}$,⁹ neighbor-hypercoreness, and neighbor-degree-hypercoreness (Definitions. 9, 12, and 14). Notably, we do not claim that higher information gain is always better, since degree is still a reason measure by cohesiveness, and being too different from degrees can be negative as a cohesiveness measure. In Fig. 17, we report the results on influential-node identification, on the datasets not covered in the main text. In Table 7, we report the full results of the collapsed (k, t)-hypercore problem.

Fig. 15

Supplementary results for Fig. 6

Fig. 16

Supplementary results for Fig. 7

Fig. 17

Supplementary results for Fig. 8

Table 7 Full results of the collapsed (k, t)-hypercore problem ($b = 100$). Time: running time (in seconds). Red.: reduction in the hypercore size