GL-GCN: Global and Local Dependency Guided Graph Convolutional Networks for aspect-based sentiment classification

Highlights

• Exploit the syntactic dependency structure to mine sentence local structure information.

• Construct a word-document graph to explore global word dependency information.

• Propose a novel architecture to encode both global and local structure signals.

• Conduct extensive experiments to verify the effectiveness of the proposed approach.

Abstract

Aspect-based sentiment classification, which aims at identifying the sentiment polarity of a sentence towards the specified aspect, has become a crucial task for sentiment analysis. Existing methods have proposed effective models and achieved satisfactory results, but they mainly focus on exploiting local structure information of a given sentence, such as locality, sequentiality or syntactical dependency constraints within the sentence. Recently, some research works, which utilizes global dependency information, has attracted increasing interest and significantly boosts the performance of text classification. In this paper, we simultaneously introduce both global structure information and local structure information into the task of aspect-based sentiment classification, and propose a novel aspect-based sentiment classification approach, i.e., Global and Local Dependency Guided Graph Convolutional Networks (GL-GCN). In particular, we exploit the syntactic dependency structure as well as sentence sequential information (e.g., the output of BiLSTM) to mine the local structure information of a sentence. On the other hand, we construct a word-document graph using the entire corpus to reveal the global dependency information between words. In addition, an attention mechanism is leveraged to effectively fuse both global and local dependency structure signals. Extensive experiments are conducted on five benchmark datasets in terms of both Accuracy and F1-Score, and the results illustrate that our proposed framework outperforms state-of-the-art methods for aspect-based sentiment classification. The model is implemented using PyTorch and is trained on GPU GeForce GTX 2080 Ti.

Introduction

With the rapid growth of online review sites such as Amazon, Yelp and IMDB, aspect-based sentiment classification has become a crucial topic in recent years. The main challenge of aspect-based sentiment classification is to identify the corresponding sentiment polarity (e.g., positive, neutral, or negative) towards a specified target when multiple targets are available in a sentence. Fig. 1 shows an example of aspect-based sentiment classification with multiple sentiment polarities, where the sentiment polarity of food is positive, while for the atmosphere it is negative.

In recent years, a number of efforts have been made towards effectively modeling semantic relatedness between context words and the aspects within a sentence. Liu and Zhang (2017) and Wang, Huang, Zhao, and Zhu (2016) propose to utilize attention mechanisms Bahdanau, Cho, and Bengio (2015) together with Recurrent Neural Networks (RNN) (Bengio et al., 2003, Hochreiter and Schmidhuber, 1997) for aspect-based sentiment classification. It assigns a positive weight for each context word, which reflects the importance of the word for determining the sentiment polarity of the specified target.

Fan et al. (2018) find that the sentiment of an aspect is usually determined by key phrases rather than individual words. Based on this observation, Li, Bing, Lam, and Shi (2018) and Xue and Li (2018) propose to employ Convolutional Neural Networks (CNNs) (Lecun, Bottou, Bengio, & Haffner, 1998) to capture multi-word phrases via the convolution operations over word sequences. As CNN and RNN prioritize locality and sequentiality (Battaglia, Hamrick, Bapst, et al., 2018), these models can effectively capture semantic and syntactic information in local consecutive word sequences. However, they lack a mechanism to account for long-range word dependencies, and may result in identifying irrelevant clues for determining aspect sentiment. It is worth noting that the long-range word dependencies mentioned here is a comparative statement. Although CNN can model word dependencies within a sentence, they only capture local phrase-level dependencies detected by filters (e.g., a sliding-window). In recent years, dependency tree has received considerable attention since it can capture dependency relationship between two distant words (e.g., there is a syntactic relationship between them). Many research works propose to leverage the dependency tree to address the issue. For example, Zhang, Li, and Song (2019) propose to exploit syntactical dependency structures within a sentence. They build a Graph Convolutional Network (GCN) over the dependency tree of a sentence and exploit syntactical information to bridge the long-range word dependency.

Whereas the aforementioned works are promising and achieve satisfactory results, their limitation is that they mainly rely on exploiting local structure dependency information, such as locality, sequentiality or syntactical dependency constraints within a sentence, while global structure dependency information is largely ignored. To be specific, existing methods lack an explicit modeling of the global dependency signal, which is latent in the entire corpus to reveal the global relationships between words.

Recent research (Peng et al., 2018, Yao et al., 2019) has shown that exploiting global structure dependency information can often significantly improve the performance of text classification. Peng et al. (2018) convert a document into a word co-occurrence graph, and then leverage graph convolution operations to convolve the graph. Yao et al. (2019) propose to build a text graph for an entire corpus, where nodes are words and documents. The edge between two word nodes relies on word co-occurrence, and the edge between a word node and a document node is using TFIDF. GCN is then used to capture high order neighborhood information.

In this paper, we propose Global and Local Dependency Guided Graph Convolutional Networks (GL-GCN), for aspect-based sentiment classification. In particular, we leverage two kinds of GCNs to learn different dependency structure information: (1) One GCN is leveraged to capture global dependency structure information via exploring the entire corpus; (2) Another GCN is utilized to model local dependency structure information given in each sentence. The framework of the proposed model is shown in Fig. 2.

We conducted extensive experiments on five datasets, i.e., TWITTER, LAPTOP, REST14, REST15, and REST16. All datasets are publicly available and have been widely used in the task of aspect-based sentiment classification. Experimental results demonstrate that the proposed GL-GCN approach can effectively model both global and local dependency structure information, and consistently outperforms the state-of-the-art baseline methods with a large margin.

In summary, this work makes the following main contributions:

• We propose a novel aspect-based sentiment classification approach exploiting both global and local dependency structure signals to better address the issue of long-range of multi-word dependency.

• We propose a novel architecture, which consists of two kinds of GCNs, to effectively encode both global and local structure signals. Moreover, a gating mechanism is leveraged to adaptively fuse these two kinds information.

• We conduct extensive experiments on five datasets demonstrating that GL-GCN is effective in improving the embedding quality by involving global structure information, thereby achieving the state-of-the-art performance

The rest of this paper is organized as follows. Section 2 presents the related work. Section 3 introduces the proposed approach GL-GCN to aspect-based sentiment classification. Section 4 presents extensive experiments to evaluate the effectiveness of our approach, and discusses the effectiveness of involving global dependency structure information. Section 5 concludes the paper.