Abstract
Micro-blog has become increasingly popular among the general public. It has brought a lot of comment text to researchers for its great convenience, timely updating, and a wide variety of self-focused topics. Identifying the emotions expressed in these comments has become a valuable topic in order to make inferences for focused contents in Micro-blog. In this paper, we report on the effectiveness of the language representation model BERT [1] with respect to the sentiment classification tasks. Experimental results show that the pre-training of deep bidirectional transformers can improve the accuracy, recall and F1 score on sentiment classification. The final evaluation index of this problem by using a Github data set increased by 2.3% on average.
This work is supported by the National Natural Science Foundation (Grant Nos. 61872298, 61532009, 61602389, 61802316, and 61472329), the Chunhui Plan Cooperation and Research Project, Ministry of Education of China (Nos. Z2015100 and Z2015109), the Civil Aviation Administration of China (No. PSDSA201802), the Chengdu Science and Technology Bureau (No. 2016-XT00-00015-GX), and Science and Technology Department of Sichuan Province (Nos. 2018026, 2017HH0083, 2016JY0244, and 2017RZ0009).
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Devlin, J., et al.: Bert: pre-training of deep bidirectional transformers for language understanding (2018). arXiv preprint: arXiv:1810.04805
Hatzivassiloglou, V., McKeown, K.R.: Predicting the semantic orientation of adjectives. In: Proceedings of the 35th Annual Meeting of the Association for Computational Linguistics and Eighth Conference of the European Chapter of the Association for Computational Linguistics. Association for Computational Linguistics (1997)
Pang, B., Lee, L., Vaithyanathan, S.: Thumbs up?: sentiment classification using machine learning techniques. In: Proceedings of the ACL 2002 Conference on Empirical Methods in Natural Language Processing, vol. 10. Association for Computational Linguistics (2002)
Socher, R., et al.: Semi-supervised recursive autoencoders for predicting sentiment distributions. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics (2011)
Elman, J.L.: Finding structure in time. Cogn. Sci. 14(2), 179–211 (1990)
LeCun, Y., et al.: Backpropagation applied to handwritten zip code recognition. Neural Comput. 1(4), 541–551 (1989)
Qian, Q., et al.: Learning tag embeddings and tag-specific composition functions in recursive neural network. In: Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing. Long Papers, vol. 1 (2015)
Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)
Hinton, G.E.: Learning distributed representations of concepts. In: Proceedings of the Eighth Annual Conference of the Cognitive Science Society, vol. 1 (1986)
Bengio, Y., et al.: A neural probabilistic language model. J. Mach. Learn. Res. 3, 1137–1155 (2003)
Mikolov, T., et al.: Distributed representations of words and phrases and their compositionality. In: Advances in Neural Information Processing Systems (2013)
Peters, M.E., et al.: Deep contextualized word representations (2018). arXiv preprint: arXiv:1802.05365
Vaswani, A., et al.: Attention is all you need. In: Advances in Neural Information Processing Systems (2017)
Radford, A., et al.: Improving language understanding by generative pre-training (2018). https://s3-us-west-2.amazonaws.com/openai-assets/research-covers/language-unsupervised/language_understanding_paper.pdf
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Zheng, J., Chen, X., Du, Y., Li, X., Zhang, J. (2020). Short Text Sentiment Analysis of Micro-blog Based on BERT. In: Park, J., Yang, L., Jeong, YS., Hao, F. (eds) Advanced Multimedia and Ubiquitous Engineering. MUE FutureTech 2019 2019. Lecture Notes in Electrical Engineering, vol 590. Springer, Singapore. https://doi.org/10.1007/978-981-32-9244-4_56
Download citation
DOI: https://doi.org/10.1007/978-981-32-9244-4_56
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-32-9243-7
Online ISBN: 978-981-32-9244-4
eBook Packages: EngineeringEngineering (R0)