Ping Huang
ABSTRACT
The rapid development of the network has accelerated the speed of information circulation. Analyzing the emotional tendency contained in the network text is very helpful to tap the needs of users. However, most of the existing sentiment classification models rely on manually labeled text features, resulting in insufficient mining of deep semantic features hidden in the text, and it is difficult to improve the classification performance significantly. This paper presents a text sentiment classification model combining BERT and convolutional neural networks (CNN). The model uses BERT to complete the word embedding of the text, and then uses CNN to learn the deep semantic information about the text, so as mine the emotional tendency towards the text. Through verification on the large movie review dataset, BERT-CNN model can achieve an accuracy of 86.67%, which is significantly better than traditional classification method of textCNN. The results show that the method has good performance in this field.
- Apoorva Arya, Vishal Shukla, Arvind Negi, and Kapil Gupta. 2020. A Review: Sentiment Analysis and Opinion Mining. In Proceedings of the International Conference on Innovative Computing & Communications (ICICC). University of Delhi, New Delhi, India, Article 32, 6 Pages. http://dx.doi.org/10.2139/ssrn.3602548Google Scholar
Cross Ref
- Ramesh Wadawadagi, and Veerappa Pagi. 2020. Sentiment analysis with deep neural networks: comparative study and performance assessment. Artificial Intelligence Review. 53, (May 2020), 6155–6195. https://doi.org/10.1007/s10462-020-09845-2Google ScholarDigital Library
- Ying Xia. 2020. Research on statistical machine translation model based on deep neural network. Computing. 102, (March 2020), 643-661. https://link.springer.com/article/10.1007/s00607-019-00752-1Google Scholar
- Ashima Yadav, and Dinesh Kumar Vishwakarma. 2020. Sentiment analysis using deep learning architectures: a review. Artificial Intelligence Review. 53, (August 2020), 4335–4385. https://link.springer.com/article/10.1007/s10462-019-09794-5Google ScholarDigital Library
- Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013. Distributed Representations of Words and Phrases and their Compositionality. InProceedings of the 26th International Conference on Neural Information Processing Systems. ACM, NY, United States, 3111-3119. https://dl.acm.org/doi/10.5555/2999792.2999959Google Scholar
- Jeffrey Pennington, Richard Socher, and Christopher Manning. 2014. GloVe: Global Vectors for Word Representation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP). Association for Computational Linguistics, Doha, Qatar, 1532–1543. http://dx.doi.org/10.3115/v1/D14-1162Google ScholarCross Ref
- MSR Hitesh, Vedhosi Vaibhav, Y.J Abhishek Kalki, Suraj Harsha Kamtam, and Santoshi Kumari. 2019. Real-Time Sentiment Analysis of 2019 Election Tweets using Word2vec and Random Forest Model. In 2019 2nd International Conference on Intelligent Communication and Computational Techniques (ICCT). IEEE, Jaipur, India, 146-151. http://dx.doi.org/10.1109/ICCT46177.2019.8969049Google ScholarCross Ref
- Chi Sun, Xipeng Qiu, Yige Xu, and Xuanjing Huang. 2019. How to Fine-Tune BERT for Text Classification? In.China National Conference on Chinese Computational Linguistics CCL 2019: Chinese Computational Linguistics. Springer, Kunming, China,194-206. https://doi.org/10.1007/978-3-030-32381-3_16Google ScholarDigital Library
- Amit Kumar Sharma, Sandeep Chaurasia, and Devesh Kumar Srivastava. 2020. Sentimental Short Sentences Classification by Using CNN Deep Learning Model with Fine Tuned Word2Vec. Procedia Computer Science. 167, (January 2020), 1139-1147. http://dx.doi.org/10.1016/j.procs.2020.03.416Google Scholar
- Dongliang Xu, Zhihong Tian, Rufeng Lai, Xiangtao Kong, ZhiyuanTan, and WeiShie. 2020. Deep learning based emotion analysis of microblog texts. Information Fusion. 64, (December 2020), 1-11. https://doi.org/10.1016/j.inffus.2020.06.002Google Scholar
- Mohd Usama, Belal Ahmad, Enmin Song, M. Shamim Hossain, Mubarak Alrashoud, and Ghulam Muhammad. 2020. Attention-based sentiment analysis using convolutional and recurrent neural network. Future Generation Computer Systems. 113, (December 2020), 571-578. http://dx.doi.org/10.1016/j.future.2020.07.022Google Scholar
- Zhao A, and Yu Y. 2021. Knowledge-enabled BERT for aspect-based sentiment analysis. Knowledge-Based Systems. 227, (September 2021) , 107220. http://dx.doi.org/10.1016/j.knosys.2021.107220Google ScholarDigital Library
- Kawin Ethayarajh. 2019. How Contextual are Contextualized Word Representations? Comparing the Geometry of BERT, ELMo, and GPT-2 Embeddings. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), Association for Computational Linguistics, Hong Kong, China, 55-65. http://dx.doi.org/10.18653/v1/D19-1006Google Scholar
- Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. Retrieved May 24, 2019 from https://arxiv.org/abs/1810.04805Google Scholar
- Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In 31st Conference on Neural Information Processing Systems (NIPS 2017). Curran Associates Inc, California, USA, 5998-6008. http://dl.acm.org/doi/abs/10.5555/3295222.3295349Google Scholar
- Yaojie Zhang, Bing Xu, and Tiejun Zhao. 2020. Convolutional multi-head self-attention on memory for aspect sentiment classification. IEEE/CAA Journal of Automatica Sinica. 7, 4 (July 2020), 1038 - 1044. https://doi.org/10.1109/JAS.2020.1003243Google ScholarCross Ref
- Jürgen Schmidhuber. 2015. Deep learning in neural networks: An overview. Neural networks. 61, (January 2015), 85-117. https://doi.org/10.1016/j.neunet.2014.09.003Google ScholarDigital Library
- Jiuxiang Gu, Zhenhua Wang, Jason Kuen, Lianyang Ma, Amir Shahroudy, Bing Shuai, Ting Liu, Xingxing Wang, and Gang Wang. 2018. Recent advances in convolutional neural networks. Pattern Recognition. 77, (May 2018), 354-377. https://doi.org/10.1016/J.PATCOG.2017.10.013Google ScholarDigital Library
Recommendations
Sentiment Analysis of Chinese Text Based on CNN-BiLSTM Serial Hybrid Model
ICCPR '21: Proceedings of the 2021 10th International Conference on Computing and Pattern RecognitionConsidering the forward and backward dependencies of words in Chinese sentences, this paper combines convolutional neural networks (CNN) and bi-directional long short-term memory (BiLSTM) according to a certain framework to form CNN- BiLSTM model, and ...
Identification of Conflict Opinion in Aspect-Based Sentiment Analysis using BERT-based Method
IC3INA '22: Proceedings of the 2022 International Conference on Computer, Control, Informatics and Its ApplicationsAspect-based sentiment analysis (ABSA) is an NLP task for predicting sentiment polarities of specific aspects in a given opinion sentence. Recent research shows that deep learning and language modeling like BERT has become state-of-the-art in NLP tasks,...
Lexicon-Based Methods vs. BERT for Text Sentiment Analysis
Analysis of Images, Social Networks and TextsAbstractThe performance of sentiment analysis methods has greatly increased in recent years. This is due to the use of various models based on the Transformer architecture, in particular BERT. However, deep neural network models are difficult to train and ...
Comments