Discriminative Streaming Network Embedding

Abstract

Many real-world networks (e.g., friendship network among Facebook users) generate data (e.g., friend requests) in a stream fashion. Recently, several network embedding methods are proposed to learn embeddings on such networks incrementally. However, these methods perform incremental updates in a heuristic manner and thus fail to quantitatively restrict the differences between incremental learning and direct learning on the entire network (i.e., the batch learning). Moreover, they ignore the node labels (e.g., interests) when learning node embeddings, which undermines the performance of network embeddings for applications such as node classification. To solve this problem, in this paper we propose a novel network embedding framework, Discriminative Streaming Network Embedding (DimSim). When an edge insertion/deletion occurs, DimSim fast learns node embeddings incrementally, which is desired for many online applications such as anomaly detection. With the incremental learning method, at any time, the objective function well approximates to that of batch learning on the current snapshot. More importantly, the average amount of updating operations of DimSim for processing each newly coming edge is about $O\left(n^{1-\lambda }\right)$, where $\lambda$ is constant between 1 and 2 and $n$ is the number of nodes. As a result, DimSim can be increasingly fast as the network grows, which adheres to the intuition that an edge insertion/deletion has a smaller impact on larger networks. In addition, we also utilize node label information to learn discriminative node embeddings and we deliberately design a new model to automatically trade-off the node-specific importance of topology and label information. Extensive experiments on real-world streaming networks show that our method DimSim is up to 50 times faster and 40% more accurate than the state-of-the-arts.

Introduction

Many real-world networks (e.g., friendships among Facebook or Tencent users, citations among research papers, and communications among telecom users) generate data (e.g., friend requests, new citations, and calling records) in a stream fashion, where an element (e.g., a friend request) records a link between two users. The data can be naturally modeled as a streaming network represented by a sequence of edge insertions/deletions. These real-world streaming networks are usually extensive and frequently evolves. For example, the number of monthly active users in the social networks of Facebook is about 2 billion and there are over 1500 friend requests per second. Therefore, analyzing these networks provides many new challenges in areas such as community detection [1], [2], [3], anomaly detection [4], [5], and recommendation [6], [7], and thus have received significant attention in the last few years.

Network embedding methods [8], [9], [10], [11], [12], [13] are effective tools for analyzing such huge networks. These methods aim to transform a network into a low-dimensional space where each node is represented by a low-dimensional feature vector, and the proximity of nodes is preserved. Using the learned embeddings, existing data mining and machine learning algorithms can work effectively in many tasks, such as node classification [8], [11], link prediction [9], and anomaly detection [10]. However, since the steaming network evolves rapidly, it is still challenging for the above static embedding methods to handle such networks, because one needs to perform them on the entire network (i.e., the batch learning) as long as the network changes, which results in a large number of node embedding updates. Recently, several streaming network embedding methods [14], [15], [16], [17], [18] are proposed to perform incremental updates when the topology changes. However, these methods perform incremental updates in a heuristic manner and thus fail to quantitatively restrict the differences between incremental learning and batch learning. Moreover, these embedding methods perform in an unsupervised manner, i.e., they ignore the label information when learning node embeddings, which leads to sub-optimal results for applications such as node classification.

Many real-world networks have abundant node labels (e.g., interests). For instance, social networks such as Tencent and Flickr, allow users to create groups of specific labels (e.g., group description) that other users can join. The coauthor network is another example where researchers’ labels are their research interests. Motivated by the fact that labels are potentially helpful in learning a better joint embedding representation, we propose to leverage label information when learning streaming network embedding.

It is a nontrivial task to take advantage of label information in streaming networks for embedding learning. There are three main challenges. Firstly, as the evolving of network topology, the label information may also change as time elapses. Take the coauthor network as an example; recently, many researchers change their interests to the popular deep learning and simultaneously they coauthor with other researchers in the deep learning field. How to process the evolving of both topology and label information is still an unresolved problem. Secondly, although several discriminative embedding methods [19], [20], [21], [22] are proposed to learn network embeddings from label and topology information, they are all accustomed to static networks. Therefore, one needs to apply them on the entire network as long as the network changes, which is computationally intensive and prohibited for many online applications. Moreover, these methods usually balance the importance of label and topology information in a coarse-grained way and thus ignore the node-specific difference. Third, the label information could be sparse and incomplete. For instance, in social networks, the proportion of active users with labels might also be quite small. Therefore, it is a challenging task to leverage the label information to learn embeddings on streaming networks efficiently.

To solve the above challenges, in this paper, we propose a novel network embedding framework, Discriminative Streaming Network Embedding (DimSim), which learns effective embeddings for streaming networks efficiently. For effectiveness, a novel objective function for constructing discriminative embeddings is designed to automatically trade-off the importance of topology and label information. Notably, we propose a method using a learnable parameter matrix to better model the trade-offs that differ among individual nodes. For efficiency, we design an incremental learning algorithm, which theoretically guarantees that the objective function of incremental embedding learning well approximates to that of batch learning on the current snapshot at any time. When an edge insertion/deletion occurs, DimSim learns node embeddings incrementally, which is desired for many online applications such as anomaly detection. More importantly, the average amount of updating operations of DimSim for processing each newly coming edge is about $O\left(n^{1-\lambda }\right)$, where $\lambda$ is constant between $1$ and $2$ and $n$ is the number of nodes in the current network. As a result, DimSim can be increasingly fast as the network grows, which adheres to the intuition that an edge insertion/deletion has a smaller impact on larger networks. Our main contributions can be summarized as:

• We propose a novel framework DimSim to efficiently learn effective network embeddings from both topology and label information of streaming networks. Different from previous works, we design a novel loss function with fine-grained trade-off parameters, which automatically trade-off the importance of topology and label information for each node.

• We propose an efficient incremental embedding learningmethod, of which the average time complexity for processing each newly coming edge is inversely proportional to the network size. To the opposite, previous works usually take $O\left(n\right)$ for each snapshot. As a result, DimSim can be increasingly fast as the network grows and thus can cope with the fast changes in streaming networks.

• Different from the heuristic updating manner of most previous works, our method DimSim theoretically guarantees that, at any time, the objective function of incremental learning well approximates to that of batch learning on the current snapshot at any time.

• Experimental results on benchmark datasets show that our method is up to $50$ times faster and $40\text{\%}$ more accurate than the state-of-the-arts.

The rest of this paper is organized as follows. We describe the problem of discriminative streaming network embedding in Section 2. Section 4 presents our framework DimSim. The performance evaluation and testing results are presented in Section 5. Section 3 summarizes related work. Conclusions then follow.