Skip to main content
SpringerLink
Log in

BERT Based Semi-Supervised Hybrid Approach for Aspect and Sentiment Classification

  • Published:
Neural Processing Letters Aims and scope Submit manuscript

Abstract

Aspect-based sentiment analysis (ABSA) includes two sub-tasks namely, aspect extraction and aspect-level sentiment classification. Most existing works address these sub-tasks independently by applying a supervised learning approach using labeled data. However, obtaining such labeled sentences is difficult and extremely expensive. Hence, it is important to solve ABSA without taking a dependency on labeled sentences. In this work, we propose a three-step semi-supervised hybrid approach that jointly detects an aspect and its associated sentiment in a given review sentence. The first step of our approach takes a small set of seed words for each aspect and sentiment class to construct respective semantically coherent class vocabularies. The second step makes use of these constructed vocabularies along with POS tags to label a subset of sentences from the training corpus. As we adopt a semi-automated approach to label the data, this process may induce noise in the labels during the annotation. In the last step, we use such labeled sentences to build a noise-robust deep neural network for aspect and sentiment classification. We conduct experiments on two real data sets to verify the effectiveness of our model (https://github.com/Raghu150999/UnsupervisedABSA).

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

Notes

  1. https://github.com/ruidan/Unsupervised-Aspect-Extraction.

  2. https://github.com/clips/cat.

References

  1. Hu M, Bing L (2004) Mining and summarizing customer reviews. In: Proceedings of the tenth ACM SIGKDD international conference on Knowledge discovery and data mining, pp 168–177

  2. Liu B, Lei Z (2012) A survey of opinion mining and sentiment analysis. In: Mining text data, pp 415–463. Springer

  3. Pontiki M, Dimitrios G, Haris P, Ion A, Suresh M, Al-Smadi M, Al-Ayyoub M, Yanyan Z, Bing Q, De Clercq O, et al (2016) Semeval-2016 task 5: aspect based sentiment analysis. In: 10th international workshop on semantic evaluation (SemEval 2016)

  4. Guang Qiu, Liu Bing Bu, Jiajun Chen Chun (2011) Opinion word expansion and target extraction through double propagation. Comput Linguist 37(1):9–27

    Article  Google Scholar 

  5. Blei DM, Ng AY, Jordan MT (2003) Latent dirichlet allocation. J Mach Learn Res 3(Jan):993–1022

  6. Brody S, Noemie E (2010) An unsupervised aspect-sentiment model for online reviews. In Human language technologies: the 2010 annual conference of the North American chapter of the association for computational linguistics. Association for Computational Linguistics, pp 804–812

  7. Wayne Xin Z, Jing J, Hongfei Y, Xiaoming L (2010) Jointly modeling aspects and opinions with a maxent-lda hybrid. In: Proceedings of the 2010 conference on empirical methods in natural language processing. Association for Computational Linguistics, pp 56–65

  8. Zhiyuan C, Arjun M, Bing L (2014) Aspect extraction with automated prior knowledge learning. In: Proceedings of the 52nd annual meeting of the association for computational linguistics (vol 1, long papers), pp 347–358

  9. Miao Y, Lei Y, Phil B (2016) Neural variational inference for text processing. In: International conference on machine learning, pp 1727–1736

  10. Akash S, Charles S (2017) Autoencoding variational inference for topic models. arXiv preprint arXiv:1703.01488

  11. Ming L, Jing L, Haisong Z, Lingzhi W, Xixin W, Kam-Fai W (2019) Coupling global and local context for unsupervised aspect extraction. In: Proceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing (EMNLP-IJCNLP), pp 4571–4581

  12. He R, Wee Sun L, Hwee Tou N, Daniel D (2017) An unsupervised neural attention model for aspect extraction. In: Proceedings of the 55th annual meeting of the association for computational linguistics (vol 1, long papers). Association for Computational Linguistics, Vancouver, Canada, pp 388–397

  13. Tulkens S, van Cranenburgh A (2020) Embarrassingly simple unsupervised aspect extraction. In: Proceedings of the 58th annual meeting of the association for computational linguistics. Association for Computational Linguistics, pp 3182–3187 (Online, July 2020)

  14. Mikolov T, Kai C, Greg C, Jeffrey D (2013) Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781

  15. Wang L, Kang L, Zhu C, Jun Z, De Melo G (2015) Sentiment-aspect extraction based on restricted boltzmann machines. In: Proceedings of the 53rd annual meeting of the association for computational linguistics and the 7th international joint conference on natural language processing (vol 1, long papers), pp 616–625

  16. Aitor G-P, Montse C, German R (2018) W2vlda: almost unsupervised system for aspect based sentiment analysis. Exp Syst Appl 91:127–137

    Article  Google Scholar 

  17. Jiaxin H, Meng Y, Fang G, Heng J, Jiawei H (2020) Weakly-supervised aspect-based sentiment analysis via joint aspect-sentiment topic embedding. arXiv preprint arXiv:2010.06705

  18. Devlin J, Ming-Wei C, Kenton L, Kristina T (2018) Bert: pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805

  19. Xu H, Bing L, Lei S, Yu Philip S (2019). Bert post-training for review reading comprehension and aspect-based sentiment analysis. arXiv preprint arXiv:1904.02232

  20. He R, Wee Sun L, Hwee Tou N, Daniel D (2018) Exploiting document knowledge for aspect-level sentiment classification. In: Proceedings of the 56th annual meeting of the association for computational linguistics. Association for Computational Linguistics

  21. Angelidis S, Mirella L (2018) Summarizing opinions: aspect extraction meets sentiment prediction and they are both weakly supervised. CoRR, abs/1808.08858

  22. Karamanolakis G, Daniel H, Luis G (2019) Leveraging just a few keywords for fine-grained aspect detection through weakly supervised co-training. In: Proceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing (EMNLP-IJCNLP). Association for Computational Linguistics, Hong Kong, China, pp 4611–4621

  23. Xu X, Songbo T, Yue L, Xueqi C, Zheng L (2012) Towards jointly extracting aspects and aspect-specific sentiment knowledge. In: Proceedings of the 21st ACM international conference on information and knowledge management, CIKM ’12. Association for Computing Machinery, New York, NY, USA, pp 1895–1899

  24. Zhao X, Jiang J, Hongfei Y, Xiaoming L (2010) Jointly modeling aspects and opinions with a MaxEnt-LDA hybrid. In: Proceedings of the 2010 conference on empirical methods in natural language processing. Association for Computational Linguistics, Cambridge, MA, pp 56–65

  25. Wang L, Kang L, Zhu C, Jun Z, de Melo G (2015) Sentiment-aspect extraction based on restricted boltzmann machines. In: Proceedings of the 53rd annual meeting of the association for computational linguistics and the 7th international joint conference on natural language processing (vol 1: long papers). Association for Computational Linguistics, Beijing, China, pp 616–625

  26. Zhuang H, Fang G, Chao Z, Liyuan L, Jiawei H (2020)Joint aspect-sentiment analysis with minimal user guidance. In: Proceedings of the 43rd international ACM SIGIR conference on research and development in information retrieval, SIGIR ’20. Association for Computing Machinery, New York, NY, USA, pp 1241–1250

  27. Honnibal M, Ines M, Van Landeghem S, Adriane B (2020) spaCy: industrial-strength natural language processing in python

  28. Zhang Z, Sabuncu Mert R (2018) Generalized cross entropy loss for training deep neural networks with noisy labels. arXiv preprint arXiv:1805.07836

  29. Tulkens S, van Cranenburgh A (2020) Embarrassingly simple unsupervised aspect extraction. arXiv preprint arXiv:2004.13580

  30. Kingma Diederik P, Jimmy B (2014) Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980

  31. Conneau A, Kartikay K, Naman G, Vishrav C, Guillaume W, Francisco G, Edouard G, Myle O, Luke Z, Veselin S (2019) Unsupervised cross-lingual representation learning at scale. arXiv preprint arXiv:1911.02116

Download references

Funding

All authors certify that they have no affiliations with or involvement in any organization or entity with any financial interest or non-financial interest in the subject matter or materials discussed in this manuscript.

Author information

Authors and Affiliations

  1. Birla Institute of Technology & Science, Pilani—Hyderabad, Hyderabad, 500078, India

    Avinash Kumar, Pranjal Gupta, Raghunathan Balan, Lalita Bhanu Murthy Neti & Aruna Malapati

Authors
  1. Avinash Kumar
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Pranjal Gupta
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Raghunathan Balan
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Lalita Bhanu Murthy Neti
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Aruna Malapati
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Avinash Kumar.

Ethics declarations

Conflict of interest

The authors declare that there is no conflict of interest regarding the publication of this paper.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kumar, A., Gupta, P., Balan, R. et al. BERT Based Semi-Supervised Hybrid Approach for Aspect and Sentiment Classification. Neural Process Lett 53, 4207–4224 (2021). https://doi.org/10.1007/s11063-021-10596-6

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11063-021-10596-6

Keywords

Search

Navigation