Dual-Channel Interactive Graph Convolutional Networks for Aspect-Level Sentiment Analysis

Abstract

Aspect-level sentiment analysis aims to identify the sentiment polarity of one or more aspect terms in a sentence. At present, many researchers have applied dependency trees and graph neural networks (GNNs) to aspect-level sentiment analysis and achieved promising results. However, when a sentence contains multiple aspects, most methods model each aspect independently, ignoring the issue of sentiment connection between aspects. To address this problem, this paper proposes a dual-channel interactive graph convolutional network (DC-GCN) model for aspect-level sentiment analysis. The model considers both syntactic structure information and multi-aspect sentiment dependencies in sentences and employs graph convolutional networks (GCN) to learn its node information representation. Particularly, to better capture the representations of aspect and opinion words, we exploit the attention mechanism to interactively learn the syntactic information features and multi-aspect sentiment dependency features produced by the GCN. In addition, we construct the word embedding layer by the BERT pre-training model to better learn the contextual semantic information of sentences. The experimental results on the restaurant, laptop, and twitter datasets show that, compared with the state-of-the-art model, the accuracy is up to 1.86%, 2.50, 1.36%, and 0.38 and the Macro-F1 values are up to 1.93%, 0.61%, and 0.4%, respectively.

1. Introduction

With the rapid development of social networks and internet applications, an increasing amount of user comment data are emerging on the web. Usually, the emotional tendencies of these data are of great value in understanding users and businesses and mining user emotions. However, due to the complex structure of online review data, the performance of traditional sentiment analysis cannot achieve our expected results. Therefore, researchers propose a new task named aspect-based sentiment analysis [1,2]. Different from traditional coarse-grained sentiment analysis, aspect-based sentiment analysis is an entity-oriented, fine-grained sentiment analysis task that aims to identify the sentiment polarity of specific aspects in sentences (e.g., negative, neutral, and positive). Taking “the restaurant food is good but the service is dreadful” as an example, the emotional polarities of the aspect words “restaurant food” and “service” in the sentence are positive and negative, respectively. Therefore, aspect-based sentiment analysis can accurately identify the user’s attitude toward an aspect, rather than simply judging the sentiment polarity of a sentence. Aspect-based sentiment analysis mainly includes two stages: aspect-level word extraction [3,4] and aspect-level sentiment classification [5]. The former extracts the questions in the comments, and the latter classifies the opinions of these questions. In this paper, we mainly focus on sentiment classification.

The key to aspect-level sentiment analysis tasks is to model the connections between aspect words and their corresponding opinion words. Early sentiment analysis methods mainly used handcrafted features combined with traditional machine learning methods [6,7,8]. However, these methods heavily depend on the quality of handcrafted features in performance and have reached a bottleneck stage. In recent years, deep learning methods represented by neural networks have attracted increasing attention because they can automatically generate useful feature representations from aspects and their contexts, and can achieve better aspect-level sentiment classification without handcrafted features. In particular, the attention mechanism [9,10] and graph neural network [11,12,13,14,15] methods are widely used in aspect-level sentiment classification due to their ability to focus on aspect words in sentences and their ability to handle unstructured data [16,17,18,19,20,21,22,23]. For example, Su et al. [16] proposed a progressive self-supervision attention learning approach for attentional aspect-level sentiment analysis. Wu et al. [17] adopted a multi-head attention mechanism to generate aspect and context feature representations. Zhang et al. [22] used graph convolutional networks to learn node information in dependency trees and combined attention mechanisms for sentiment classification tasks. Xiao et al. [23] constructed a syntactic edge-enhanced graph convolutional network by considering the different types of neighborhoods with the edge constraint.

In this paper, we propose a GCN based method combining syntactic and multi-aspect sentiment features. We construct a multi-aspect sentiment dependency graph and elaborately design an information interaction layer to enhance the ability to precisely capture the semantic associations of different aspects with context.

The main contributions from this research are as follows.

• We propose a novel aspect-level sentiment classification approach exploiting both syntactic dependency trees and multi-aspect sentiment graphs to better capture the representation between aspects and opinion words. In addition, to better learn the context semantic information of sentences, we also introduce the BERT pre-training model to build the word embedding layer.
• We propose a novel architecture, which consists of two kinds of GCNs, to effectively encode both syntactic dependency trees and multi-aspect sentiment graphs. Moreover, the two kinds of information are learned interactively using the attention mechanism to obtain more informative representations.
• We evaluate DC-GCN on three datasets, namely twitter, laptop, and restaurant. Experiments show that the proposed model achieves superior performance over the state-of-the-art approaches.

The remainder of the paper is organized as follows: Section 2 introduces related work. Section 3 describes the details of the GCN. Section 4 describes the details of the proposed DC-GCN model. Section 5 analyses the experimental results of the proposed DC-GCN model. The conclusion of this work is provided in Section 6.

2. Related Work

Aspect-level sentiment analysis, also known as opinion mining, is a fine-grained task in natural language processing (NLP). Early work mainly adopted traditional machine learning aspects for sentiment classification. However, machine learning methods with supervised classifiers heavily rely on the quality of large handcrafted features [24]. In recent years, with the development in the field of neural network natural language processing, an increasing number of neural-network-based methods have been proposed to solve sentiment analysis tasks, such as recurrent neural networks (RNN) [25], convolutional neural networks (CNN) [26] and memory neural networks (MNN) [27]. Dong et al. [28] proposed an adaptive recurrent neural network to adaptively learn the tree structure of sentences. Xue et al. [29] proposed using convolutional neural networks to obtain feature representations of aspects and contexts, and to obtain the relationship between contexts and aspects through a gating mechanism. Tang et al. [30] proposed a memory-based neural network model to store important information in specific aspects. In addition, long short-term memory networks (LSTMs) have been widely used in aspect-level sentiment analysis due to their ability to effectively capture long-term dependencies and solve the vanishing gradient problem. Tang et al. [31] proposed a target-dependent long short-term memory (TD-LSTM) model. This method uses two LSTMs to model the context on the left and right sides of an aspect, and the last hidden state of the spliced two LTSMs is used as the classification feature of aspect-level sentiment analysis.

Subsequently, researchers found that the attention mechanism can effectively identify those words in a sentence that are more important in a certain aspect and are widely used in this task. Wang et al. [32] proposed an attention-based ATAE-LSTM model. This method introduces attention into LSTMs for the first time, and the experimental results show that the model with added attention is significantly better than LSTMs. Ma et al. [33] found that the same aspect expresses different emotional tendencies in different contexts and proposed an interactive attention network (IAN) model. This method further improves the accuracy of aspect-level sentiment analysis tasks by interactively learning context and aspect representations. Huang et al. [34] introduced an attention-over-attention mechanism based on IAN and proposed an AOA-LSTM model without pooling layers. The model can effectively capture the interaction information between aspects and contexts in sentences. Although the neural network model that introduces the attention mechanism can effectively improve the classification effect of aspect-level sentiment analysis, when the aspect consists of multiple words, the attention-based model may mistakenly focus on the context words that are not related to the aspect.

Recently, graph neural networks (GNNs) have been widely used to mine the syntactic structure information of sentences due to their excellent performance and high interpretability. Zhang et al. [22] used graph convolutional neural networks (GCNs) for the first time to obtain syntactic dependency information between contexts and aspects in syntactic dependency trees. Zhao et al. [35] proposed a GCN-based aspect-level sentiment analysis model, which can effectively capture the emotional dependencies of multiple aspects in sentences. Zhu et al. [36] proposed a global- and local-dependency-guided graph convolutional network (GL-GCN) model by combining global and local structural information. Miao et al. [37] proposed a contextual graph attention network (CGAT) model to address the problems of insufficient semantic information extraction and high computational complexity of the attention mechanism.

Although the above methods have achieved certain success, most of these methods model each aspect independently, which makes the hidden contact information of multiple aspects in a sentence underutilized. For example, “The location of this restaurant is convenient and the food is delicious, but the serving speed is too slow”. The sentiment polarity of the first aspect “location” is positive. From the conjunction “and”, we easily know that the second aspect “food” and “location” have the same polarity. Similarly, from the connection “but”, we can infer that the sentiment polarity of the last aspect is negative. Meanwhile, the syntactic dependency tree contains rich syntactic relation and distance information. However, most of the current graph-neural-network-based models are modelled around syntactic dependency trees, and there is no model that considers both syntactic dependency trees and multi-aspect sentiment dependencies. Therefore, to address the deficiencies of existing models, we propose a dual-channel interactive graph convolutional network for the aspect-based sentiment analysis (DC-GCN) model, which combines a syntactic dependency tree and a multi-aspect sentiment dependency graph.

3. Graph Convolutional Network (GCN)

3.1. Syntactic Dependency Tree

We employ the dependency parser in the Spacy toolkit to generate syntactic dependency trees for sentences. Taking “The food is okey while the atmosphere is not really good” as an example, its dependency tree structure is shown in Figure 1. As seen from the figure, green fonts represent aspect words, red fonts represent sentiment words, directed edges represent dependencies, and labels represent dependency types. In this paper, to fully exploit the syntactic information of syntactic structure, we use a graph convolutional neural network to encode a syntactic dependency tree to capture the syntactic dependencies of aspects and contexts.

3.2. Graph Convolutional Network

The graph convolutional network (GCN) is a variant of the GNN, which is inspired by the traditional convolutional neural network (CNN) and graph embedding to encode local features of unstructured data [38]. An example of a single-layer GCN is shown in Figure 2. Given a dependency graph with n nodes, the graph can be represented as an adjacency matrix A∈R^n×n. For convenience, we set the feature vector of any node i in the l-layer GCN to be $h_i^l$, l∈[1, 2, …, L]. Therefore, the mathematical model of graph convolution operating on node representation is described as follows.

$$h_i^l=\sigma ({\displaystyle \sum _{j=1}^N{A}_{ij}^{}}{h}_j^{l-1}{W}^l+b^l)$$

(1)

where $W^l$ is a weight matrix, $b^l$ is a bias term, and σ is an activation function (e.g., ReLU).

4. Methodology

Given a sentence S = {w₁, ···, w_t, ···, w_t+m, ···, w_n} of length n and an aspect a = {w_t,···, w_t+m} of length m, where the aspect word a is a subsequence of sentence S, the overall structure of the proposed DC-GCN model is shown in Figure 3. The DCGCN model consists of five parts, namely the word embedding layer, the position encoding layer, the graph convolutional network layer, the information interaction layer and the output layer. The individual components of the model are described separately in the remainder of this section.

4.1. Word Embedding Layer

Word embeddings map each word w_i in a given sentence to a low-dimensional real-valued vector space. This paper employs the pre-trained language model BERT [39] to vectorize the input sentence. The BERT pre-training model is a bidirectional Transformer model that can infer semantic information according to the contextual relationship of words, effectively solving the problem of polysemy. There are two variants of the BERT model, named BERT_base and BERT_large. The model used in this paper is BERT_base, and its structure is shown in Figure 4.

To facilitate the training and fine-tuning of the BERT model, we define the context and aspect words as “[CLS] + sentence + [SEP] + aspect + [SEP]” as the input of the model and take the output of the last layer of BERT as the next layer input. The mathematical model can be described as follows:

$$h_i=BERT(w_i)\hspace{0.33em}\hspace{0.33em}i=1,2,\dots .,n$$

(2)

4.2. Position Embedding Layer

The polarity of the target aspect is susceptible to context words that are closer to the aspect. Therefore, we introduce positional encoding to enhance the informative features of context words that are closer to the aspect and weaken the information of context words that are farther away from the aspect. The position encoding information can be obtained from the path length of aspect and context words in the sentence, and its mathematical model is defined as follows.

$$p_i=\left\{\begin{array}{l}1-\frac{a_{\mathrm{start}}-i}{n}\hspace{0.33em}\hspace{0.33em}0\le i<a_{\mathrm{start}}\\ 0\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}{a}_{\mathrm{start}}\le i<a_{\mathrm{start}}\\ 1-\frac{i-a_{\mathrm{end}}}{n}\hspace{0.33em}\hspace{0.33em}{a}_{\mathrm{end}}\le i\le n\end{array}\right.$$

(3)

where $a_{\mathrm{start}}$ and $a_{\mathrm{end}}$ represent the start index and end index of the aspect, respectively; n is the length of the sentence; and $p_i$ represents the relative distance from the context word $w_i$ in the sentence to the aspect. Finally, we can obtain the sentence feature vector with location information.

$$h_i=h_i\cdot p_i$$

(4)

4.3. DC-GCN Layer

To capture the syntactic information and multi-aspect sentiment dependency information in sentences, we employ a dual-channel graph convolutional layer in the model. This layer consists of two modules: a syntactic-based graph convolutional network and a multi-aspect dependency-based graph convolutional network. The former model is used to obtain the rich syntactic information in the dependency tree, and the latter is used to obtain the multi-aspect sentiment dependency information in the sentence.

4.3.1. Syntactic Graph Convolution Module

In this section, we construct a contextual syntactic graph G_sy(V, E), where node V represents the hidden state of a word and edge E represents the syntactic dependency between words. As mentioned above, we use the Spacy toolkit to obtain the dependency tree of the sentence. The weight between the $v_i$ and $v_j$ is defined as:

$$A_{ij}^{sy}=\left\{\begin{array}{l}1\hspace{0.33em}\hspace{0.33em}if\hspace{0.33em}{e}_{ij}\in E\\ 1\hspace{0.33em}\hspace{0.33em}if\hspace{0.33em}i=j\\ 0\hspace{0.33em}\hspace{0.33em}otherwise\end{array}\right.$$

(5)

where $A_{ij}$ denotes an adjacency matrix, i, j∈[1, n], and $e_{ij}$ ∈E denotes when node i depends on node j.

Then, we use the graph convolutional network to extract the syntactic information, and the node representation is calculated as follows:

$$h_i^{sy(l)}=RELU\left({\displaystyle \sum _{j=1}^n{\tilde{A}}_{ij}^{sy}{h}_j^{l-1}{W}^l+b^l}\right)$$

(6)

where ${\tilde{A}}_{}^{sy}={\tilde{D}}^{-\frac{1}{2}}{A}^{sy}{\tilde{D}}^{\frac{1}{2}}$ is the normalized symmetric adjacency matrix, $\tilde{D}$ is the degree matrix of ${\tilde{A}}_{}^{sy}$, $h_j^{l-1}$ is the word representation of the position information added to the previous layer, $h_i^{sy(l)}$ is the final representation of the i-th node of the first layer of the GCN, and $W^l$ and $b^l$ are the weight matrix and bias term of the GCN first layer, respectively.

4.3.2. Multi-Aspect Sentiment Graph Convolution Module

As we discuss in Section 1, existing models mostly model each aspect independently, ignoring the hidden connection information among multiple aspects. To address this problem, we construct a multi-aspect sentiment graph to capture dependency information between aspects in one sentence. As shown in Figure 3, the nodes of the multi-aspect sentiment graph are determined only by the number of aspects, where the edges represent the dependencies between aspects.

Then, we employ a graph convolutional network to extract the hidden information representation of the multi-aspect sentiment graph, and the node representation is computed as follows:

$$h_i^{ma(l)}=RELU\left({\displaystyle \sum _{j=1}^n{\tilde{A}}_{ij}^{ma}{h}_j^{l-1}{W}^l+b^l}\right)$$

(7)

where ${\tilde{A}}_{}^{ma}={\tilde{D}}^{-\frac{1}{2}}{A}^{ma}{\tilde{D}}^{\frac{1}{2}}$ is the normalized symmetric adjacency matrix, $\tilde{D}$ is the degree matrix of ${\tilde{A}}_{}^{ma}$, $h_j^{l-1}$ is the word representation of the position information added to the previous layer, $h_i^{ma(l)}$ is the final representation of the i-th node of the first layer of the GCN, and $W^l$ and $b^l$ are the weight matrix and bias term of the GCN first layer, respectively.

4.4. Information Interactive Layer

Considering the interactive information between aspects and contexts is also important for sentiment prediction. In this section, we use an attention mechanism to model the information representation between context and multi-aspect terms in a sentence, generating aspect-oriented and context-oriented representations, respectively. The context vectors with syntactic information are first average pooled, and then the attention weight ${\beta }_i$ for each aspect is calculated. The mathematical model is defined as follows.

$$h_{avg}^{sy}=\frac{1}{n}\left({\displaystyle \sum _{i=1}^n{h}_i^{sy}}\right)$$

(8)

$$\alpha (h_{avg}^{sy},h_i^{ma})=tanh\left(h_{avg}^{sy}\cdot W\cdot h_i^{maT}+b\right)$$

(9)

$${\beta }_i=\frac{exp(\alpha (h_{avg}^{sy},h_i^{ma}))}{{\displaystyle \sum _{j=1}^{n}exp(\alpha (h_{avg}^{sy},h_j^{ma}))}}$$

(10)

where, $\alpha$ is the score function, $h_i^{maT}$ is the transpose of $h_i^{ma}$, ${\beta }_i$ is the assigned weight, and W and b are the weight matrix and bias term, respectively.

For context effects, we first average pool the vectors with multi-aspect dependent features, and then compute the attention weight ${\phi }_i$ for each contextual word. The mathematical model is described as follows.

$$h_{avg}^{ma}=\frac{1}{m}\left({\displaystyle \sum _{i=1}^m{h}_i^{ma}}\right)$$

(11)

$$\varphi (h_{avg}^{ma},h_i^{sy})=tanh\left(h_{avg}^{ma}\cdot W\cdot h_i^{syT}+b\right)$$

(12)

$${\phi }_i=\frac{exp(\alpha (h_{avg}^{ma},h_i^{sy}))}{{\displaystyle \sum _{j=1}^{n}exp(\alpha (h_{avg}^{ma},h_j^{sy}))}}$$

(13)

Then, the weighted multi-aspect and context representations are calculated by Equations (14) and (15) below.

$$h_a={\displaystyle \sum _{i=1}^m{\beta }_i}\cdot h_i^{ma}$$

(14)

$$h_c={\displaystyle \sum _{i=1}^n{\phi }_i}\cdot h_i^{sy}$$

(15)

Finally, the generated multi-aspect feature vector $h_a$ is spliced with the context feature vector $h_c$ to obtain r = [$h_a$;$h_c$].

4.5. Output Layer

To predict the sentiment polarity for a given aspect, we input the final representation r obtained by the information interaction layer into a fully connected layer, which is finally output by softmax for classification. The mathematical model of the predicted probability p of sentiment polarity is defined as follows.

$$p=softmax(W_{p}r+b_p)$$

(16)

where p ∈$R^{d_p}$ is the polarity decision space, d_p represents the number of categories of sentiment polarity, and W_p and b_p are the weight parameter and bias term, respectively.

The training of the model adopts the cross-entropy loss function and L₂-regularization, and its calculation formula is as follows.

$$Loss=-{\displaystyle \sum _{i=1}^{d_p}{y}_{i}log}(p_i)+\lambda {‖\theta ‖}_2$$

(17)

where d_p represents the number of categories of sentiment polarity, $y_i$ represents the true sentiment polarity, $p_i$ is the predicted sentiment polarity, and λ is the L₂-regularization coefficient, which is the parameter that the model needs to train.

5. Experiments

5.1. Datasets and Evaluation Metrics

To verify the effectiveness of the DC-GCN model, we conduct experiments on three public datasets, namely Restaurant and Laptop for the SemEval-2014 task [40] and the Twitter dataset curated by Dong et al. [28] Each sample of the dataset is a sentence actually generated by the user. Sentences are marked with sentiment labels of one or more aspects, among which sentiment labels have three categories: positive, negative, and neutral. The statistical results of the dataset are shown in

Table 1. Detailed statistical of the three experimental datasets.

Dataset	Positive		Neutral		Negative
	Train	Test	Train	Test	Train	Test
Restaurant	2164	728	637	196	807	196
Laptop	994	341	464	169	870	128
Twitter	1561	173	3127	346	1560	173

.

We employ two evaluation metrics, Accuracy (Acc) and Macro-F1 (MF1), to evaluate the effectiveness of the DC-GCN model. For a single category, let TP be the number of correctly predicted samples, FP be the number of samples predicted to be the current category from other categories, and FN be the number of samples predicted to be other categories from the current category. Then, the calculation formulas for accuracy and MF1 are distributed as follows in (18) and (19).

$$\mathrm{Acc}=\frac{{\displaystyle \sum _{i=1}^{d_p}T{P}_i}}{{\displaystyle \sum _{i=1}^{d_p}(T{P}_i+F{P}_i+F{N}_i)}}$$

(18)

$$\left\{\begin{array}{l}\mathrm{MF}1=\frac{1}{d_p}{\displaystyle \sum _{i=1}^{d_p}\frac{2\cdot P_i\cdot R_i}{P_i+R_i}}\\ P=\frac{TP}{TP+FN}\\ R=\frac{TP}{TP+FP}\end{array}\right.$$

(19)

where d_p represents the number of categories of sentiment polarity, P is the precision rate, and R is the recall rate.

5.2. Experimental Settings

In our experiments, we employ a 768-dimensional pre-trained BERT model to generate word embedding vectors. All weight matrices are randomly initialized by a uniform distribution U(−0.01, 0.01), and all biases are set to 0. The batch size is set at 32, and the max length of a sentence is set to 85. The number of layers of the GCN is set to 2. The model uses the Adam optimizer to optimize the model parameters during the training process. The learning rate and L₂-regularization adjustment coefficients are 2 × 10⁻⁵ and 1 × 10⁻⁵, respectively. In addition, to avoid overfitting, the dropout rate is set to 0.3. Hyper-parameter settings are shown in

Table 2. Hyper-parameter settings.

Parameters	Values
Word embedding dimensions	768
GCN layers	32
Dropout rate	0.3
Learning rare	2 × 10−5
L2-ragularization coefficients	1 × 10−5

. We mainly use the PyTorch deep learning framework to train and predict the model. All the experiments are conducted on hardware with an Intel Core CPU i5-7500 3.6 GHz and an NVIDIA GTX 3090.

5.3. Baseline Models

We compare DC-GCN with state-of-the-art baselines. The models are briefly de-scribed as follows.

ATAE-LSTM [32] concatenates the aspect word feature vector and context feature vector as input and then uses an attention-added LSTM to capture the connection between aspect and context.

IAN [33] uses two LTSMs and an attention mechanism to interactively learn aspect and context representations.

RAM [21] uses an attention-based aggregation model to learn sentence representations with multi-layered structures.

AEN [41] uses an attention-based encoder to model the relationship between a given aspect and its corresponding context to avoid duplication.

ASGCN [22] uses a GCN for the first time to obtain node information on the syntactic dependency tree and combines the attention mechanism for sentiment classification.

ASEGCN [23] utilizes bidirectional LSTM and the multi-head attention mechanism to generate semantic codes, and introduces a GCN into the model.

GL-GCN [36] employs two GCN models to independently encode global and local structural information, and adaptively fuses the two types of information using a gating mechanism.

SK-GCN1 + BERT [42] uses two GCNs to independently model dependency trees and knowledge graphs, combined with multi-head attention for aspect-level sentiment classification.

SK-GCN2 + BERT [42] uses a GCN to jointly model syntax and knowledge graphs, combined with multi-head attention for aspect-level sentiment classification.

R-GAT + BERT [43] reconstructs an aspect-oriented syntactic dependency tree and utilizes a graph attention network (GAT) to learn feature representations.

CGAT + BERT [37] employs two GATs to aggregate syntactic structure information into target aspects and combines contextual attention networks to extract semantic information in sentence-aspect sequences.

5.4. Results and Analysis

In this section, we compare the aspect-level sentiment analysis tasks of the DC-GCN and other models on three datasets, where the experimental results of each comparison model are derived from their original papers. The experimental results are shown in

Table 3. Comparison results of different models on the three datasets.

Model	Restaurant		Laptop		Twitter
	Acc (%)	MF1 (%)	Acc (%)	MF1 (%)	Acc (%)	MF1 (%)
ATAE-LSTM [32]	77.20	-	68.90	-	-	-
IAN [33]	78.60	-	72.10	-	-	-
RAM [21]	80.23	70.80	74.49	71.35	69.36	67.30
AEN [41]	80.98	72.14	73.51	69.04	72.83	69.81
ASGCN [22]	80.77	72.02	75.55	71.05	72.15	70.40
ASEGCN [23]	81.61	73.42	76.49	72.64	73.84	72.42
GL-GCN [36]	82.11	73.46	76.91	72.76	73.26	71.26
SK-GCN1 + BERT [42]	81.87	73.42	79.31	75.11	74.71	73.36
SK-GCN2 + BERT [42]	83.48	75.19	79.00	75.57	75.00	73.01
R-GAR + BERT [43]	86.60	81.35	78.21	74.07	76.15	74.88
CGAT + BERT [37]	86.25	80.38	80.41	76.48	77.46	76.37
DC-GCN (our)	88.11	83.28	81.77	77.09	77.84	76.77

, in which the bold font represents the best results in the current indicators and “-” represents missing data in the literature.

It can be observed from the data in Table 3 that our proposed DC-GCN model consistently outperforms other models in the literature on the Restaurant, Laptop, and Twitter datasets, which proves the superiority of our model. Specifically, the DC-GCN model improves the accuracy by up to 1.86% and the MF1 value by 1.93% on the Restaurant dataset. In addition, the DC-GCN model improves the accuracy by up to 1.36% and the MF1 value by 0.61% on the Laptop dataset. However, the performance improvement is the least on the Twitter dataset, with only 0.38% and 0.40% improvement in accuracy and MF1 value, respectively. Because the syntactic information of the Twitter dataset is more chaotic than the other two kinds of data, it makes the modules in the DC-GCN model that rely on syntactic information unable to effectively capture its features. In summary, our proposed DC-GCN model outperforms all baselines. The experimental results demonstrate that the fusion of grammatical structure information and multi-aspect sentiment dependency information is crucial. DC-GCN can effectively capture two kinds of structural information through two kinds of GCNs and fuse the two kinds of structural information through the information interaction layer to mutually compensate for the lack of information of a single structure.

5.5. Ablation Study

To further investigate the impact of each component of the DC-GCN model on the performance, we design several ablation experiments. The specific description of each component is as follows.

DC-GCN/P removes the position embedding layer P.

DC-GCN/S removes the syntactic graph convolution module S.

DC-GCN/M removes the multi-aspect sentiment graph convolution module M.

According to the data in

Table 4. Ablation results of different components.

Model	Restaurant		Laptop		Twitter
	Acc (%)	MF1 (%)	Acc (%)	MF1 (%)	Acc (%)	MF1 (%)
DC-GCN/P	87.56	82.98	80.43	75.85	77.62	76.29
DC-GCN/S	86.63	80.71	79.75	76.33	76.54	74.91
DC-GCN/M	86.39	80.45	79.89	76.56	76.96	75.08
DC-GCN	88.11	83.28	81.77	77.09	77.84	76.77

, when the position embedding layer P is removed, the accuracy of DC-GCN on the three datasets is reduced by 0.55%, 1.34%, and 0.22%, respectively. This indicates that the position information between the aspect and the context word in the sentence is not negligible, especially in the Laptop dataset, and the contribution of the position information contributes significantly. Removing either S or M results in varying degrees of accuracy degradation, especially on the Restaurant dataset, and the accuracy and MF1 values drop significantly. This shows that a single GCN module cannot perform information interaction and thereby cannot learn more feature information between aspects and contexts. In general, DC-GCN incorporating all modules achieves the best performance.

5.6. Case Study

For an intuitive understanding of the DC-GCN model and without the multi-aspect sentiment graph convolution (MAGCN) module, we take a multi-aspect example from each of the laptop and restaurant datasets as a case study. We visualize the attention scores of DC-GCN and DC-GCN (w/o MAGCN) in

Table 5. The visualization of attention scores.

Model	Aspect	Attention Visualization
w/o MAGCN	keyboard	I love the keyboard and the screen.
	appetizers	The appetizers are ok, but the service is slow.
DC-GCN	keyboard	I love the keyboard and the screen.
	appetizers	The appetizers are ok, but the service is slow.

The darker the color in the table the higher the attention score for the word.

, where darker colors indicate higher attention scores for words.

For the first example “I love the keyboard and the screen”, with two aspects “keyboard” and “screen”, we can see that the w/o MAGCN model mainly focuses on the word “love” to predict the two aspects of polarity. For the DC-GCN model, in addition to the word “love”, it can also focus on the conjunction “and”. This result shows that DC-GCN, which fuses syntactic and multi-aspect sentiment graphs, can capture the sentiment dependencies of the two aspects through the conjunction “and”, and correctly predict the emotional polarity of the two aspects.

For the second example “The appetizers are ok, but the service is slow”, with two aspects “appetizers” and “service”, obviously, from the comma “,” and the conjunction “but”, we can know that the polarities of these two aspects are opposite. Although the w/o MAGCN model independently predicts the polarities of the two aspects, it ignores the relationship between the two aspects. In contrast, the DC-GCN model also pays attention to the conjunction “but” while predicting the polarity of both aspects.

The research case shows that our proposed DC-GCN model not only pays attention to the opinion words that help predict the sentiment of various aspects, but also considers the sentiment dependencies between different aspects in the sentence, which effectively improves the accuracy of aspect-level sentiment classification.

5.7. Impact of the DC-GCN Layer Number

To verify the impact of the number of DC-GCN layers on model performance, we compare the results using layers one to eight on the Restaurant and Laptop datasets. The experimental results are shown in Figure 5. From the data in the figure, we know that when the number of layers is 2, the performance of the model is the best. However, when the number of layers is less than 2 or more than 2, the accuracy and the MF1 value of the model are relatively lower. On the one hand, the feature learning between nodes is not sufficient when the number of layers is low. On the other hand, with the increase in the number of layers, the problem of gradient disappearance or gradient explosion easily occurs due to the increase in transformation operations in the GCN, which makes the model performance degenerate.

6. Conclusions

In this paper, we propose a dual-channel interactive graph convolutional network (DC-GCN) model for aspect-level sentiment analysis. This model adopts the pre-trained BERT model to initialize word embedding, which makes it better than common methods such as Word2vec or Glove. In addition, we simultaneously consider the syntactic structure information and multi-aspect sentiment dependencies of sentences and employ two GCNs to learn their hidden representations separately, which is completely missing from previous research studies. Finally, we employ an attention mechanism to interactively learn syntactic and multi-aspect sentiment dependency information to obtain the final feature vector. The experimental results on three datasets show that the interactive learning of sentence syntactic information and multi-aspect sentiment dependency information can indeed effectively improve the performance of the model. In future work, we will discuss how to build a more accurate sentiment graph structure between aspects and explore how to introduce graph attention networks (GATs) into models to better model the relationship between aspects and opinion words.