Aspect-based sentiment classification with multi-attention network

Abstract

Aspect-based sentiment classification aims to predict the sentiment polarity of an aspect term in a sentence instead of the sentiment polarity of the entire sentence. Neural networks have been used for this task, and most existing methods have adopted sequence models, which require more training time than other models. When an aspect term comprises several words, most methods involve a coarse-level attention mechanism to model the aspect, and this may result in information loss. In this paper, we propose a multi-attention network (MAN) to address the above problems. The proposed model uses intra- and inter-level attention mechanisms. In the former, the MAN employs a transformer encoder instead of a sequence model to reduce training time. The transformer encoder encodes the input sentence in parallel and preserves long-distance sentiment relations. In the latter, the MAN uses a global and a local attention module to capture differently grained interactive information between aspect and context. The global attention module focuses on the entire relation, whereas the local attention module considers interactions at word level; this was often neglected in previous studies. Experiments demonstrate that the proposed model achieves superior results when compared to the baseline models.

Introduction

Aspect-based sentiment classification is a fine-grained task in aspect-based sentiment analysis (ABSA). Instead of predicting the sentiment polarity of an entire sentence, the sentiment polarity of a specific aspect in the sentence is determined [1]. For example, in the sentence ‘This is a high-speed computer, but it has short battery life’, the sentiment polarity of the aspects ‘speed’ and ‘battery life’ are positive and negative, respectively. Aspect-based sentiment classification overcomes the limitation of sentence-level sentiment classification that the sentiment polarity of each aspect may differ when a sentence contains more than one aspect. Aspect-based sentiment classification consists of two stages: aspect extraction [2], [3], [4], [5], [6], [7] and sentiment classification [8], [9]. The former explores the aspects that appear in reviews, and the latter classifies the opinions about these aspects. In this study, we focus only on sentiment classification.

Recently, sequence models such as long short-term memory (LSTM) [10] and gated recurrent units [11] have been successfully used in aspect-based sentiment classification [9], [12], [13]. Despite the effectiveness of these approaches, sequential models encode words individually, which is time-consuming. To overcome this, Xue and Li [14] proposed a parallelisable solution by using convolutional neural networks (CNNs). Although CNNs are effective in reducing training time, they cannot capture long-distance relations in sentences. In addition, aspect-level sentiment polarity is highly dependent on both review context and aspect. Some models utilise an attention mechanism to add aspect information [15], [16], [17]. However, most of them regard all aspect words as a whole. When an aspect contains several words, these approaches ignore the different importance between the words in the aspect phrase, resulting in information loss. For example, the aspect of the sentence ‘This place has many different styles of pizza, and they are all amazing’ contains three words. In the aspect phrase ‘styles of pizza’, ‘of’ contributes less than ‘styles’ and ‘pizza’. It is inappropriate to place the three aspect words in equal position.

In this paper, we propose a multi-attention network (MAN) to address the aforementioned issues. MAN is a parallelisable model, as no sequence model is involved. It contains an intra- and an inter-level attention mechanism. The former learns word representations through a transformer encoder [18], which is based on a self-attention mechanism that can process context and aspect in parallel. Self-attention also allows MAN to handle long-distance dependencies because it considers every two words in a sentence. The latter employs global and local attention to capture coarse- and fine-grained interactive information between aspect and context. Global attention captures the entire interaction, whereas local attention captures the word-level interaction between aspect and context words. The main contributions of this study can be summarised as follows:

• We propose a novel model (MAN) to process words in review sentences in parallel using an attention mechanism. The proposed model requires significantly less training time than sequence models. MAN can effectively capture long dependencies in sentences by self-attention.

• MAN introduces global and local attention modules to capture different-level interactions between aspect and context. The local attention module considers the difference between aspect words.

• We evaluated MAN on several datasets, namely laptop, restaurant, and twitter. Experiments demonstrate that the proposed model achieves superior results when compared to the baseline models.

The rest of this paper is organised as follows. In Section 2, we review related work. In Section 3, we define the problem of aspect-based sentiment classification and present the proposed model in detail. Section 4 reports experiments and evaluations. Section 5 concludes this paper.