Skip to main content
SpringerLink
Log in

SentATN: learning sentence transferable embeddings for cross-domain sentiment classification

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Cross-domain Sentiment Classification (CDSC) aims to exploit useful knowledge from the source domain to obtain a high-performance classifier on the target domain. Most of the existing methods for CDSC mainly concentrate on extracting domain-shared features, while ignoring the importance of domain-specific features. Besides, these approaches focus on reducing the discrepancy of the source domain and target domain on the word-level. As a result, they cannot fully capture the whole meaning of a sentence, which makes these methods unable to learn enough transferable features. To address these issues, we present a Sentence-level Attention Transfer Network (SentATN) for CDSC, with two distinctive characteristics. Firstly, we design an efficient encoder unit to extract domain-specific features of a sentence. Secondly, SentATN provides a sentence-level adversarial training method, which can better transfer sentiment across domains by capturing complete semantic information of a sentence. Comprehensive experiments have been conducted on extended Amazon review datasets, and the results show that the proposed SentATN performs significantly better than state-of-the-art methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Deborah S A, Mirnalinee TT, Rajendram S M (2021) Emotion analysis on text using multiple kernel gaussian... Neural Process Lett 53(2):1187–1203

    Article  Google Scholar 

  2. Parcheta Z, Sanchis-Trilles G, Casacuberta F, Rendahl R (2021) Combining embeddings of input data for text classification. Neural Process Lett 53:3123–3153

    Article  Google Scholar 

  3. Zhang B, Xu X, Yang M, Chen X, Ye Y (2018) Cross-domain sentiment classification by capsule network with semantic rules. IEEE Access 6:58284–58294

    Article  Google Scholar 

  4. Du Y, He M, Wang L, Zhang H (2020) Wasserstein based transfer network for cross-domain sentiment classification. Knowl-Based Syst 204:106162

    Article  Google Scholar 

  5. Sharma R, Bhattacharyya P, Dandapat S, Bhatt H S (2018) Identifying transferable information across domains for cross-domain sentiment classification. In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp 968–978

  6. Blitzer J, Dredze M, Pereira F (2007) Biographies, bollywood, boom-boxes and blenders: Domain adaptation for sentiment classification. In: Proceedings of the 45th Annual Meeting of the Association for Computational Linguistics. The Association for Computational Linguistics

  7. Glorot X, Bordes A, Bengio Y (2011) Domain adaptation for large-scale sentiment classification: A deep learning approach. In: Proceedings of the 28th international conference on machine learning , pp 513–520

  8. Yu J, Jiang J (2016) Learning sentence embeddings with auxiliary tasks for cross-domain sentiment classification. In: Proceedings of the 2016 conference on empirical methods in natural language processing. The Association for Computational Linguistics, pp 236–246

  9. Li Z, Zhang Y, Wei Y, Wu Y, Yang Q (2017) End-to-end adversarial memory network for cross-domain sentiment classification. In: Proceedings of the 26th International Joint Conference on Artificial Intelligence, pp 2237–2243

  10. Li Z, Wei Y, Zhang Y, Yang Q (2018) Hierarchical attention transfer network for cross-domain sentiment classification. In: Thirty-Second AAAI Conference on Artificial Intelligence, pp 5852–5859

  11. Du C, Sun H, Wang J, Qi Q, Liao J (2020) Adversarial and domain-aware bert for cross-domain sentiment analysis. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, pp 4019–4028

  12. Sun B, Saenko K (2016) Deep coral: Correlation alignment for deep domain adaptation. In: Computer Vision - ECCV 2016 Workshops. Springer, pp 443–450

  13. Fu C, Huang H, Chen X, Tian Y, Zhao J (2021) Learn-to-share: A hardware-friendly transfer learning framework exploiting computation and parameter sharing. In: International Conference on Machine Learning. PMLR, pp 3469–3479

  14. Lashkaripour A, Rodriguez C, Mehdipour N, Mardian R, McIntyre D, Ortiz L, Campbell J, Densmore D (2021) Machine learning enables design automation of microfluidic flow-focusing droplet generation. Nat Commun 12(1):1–14

    Article  Google Scholar 

  15. Huang X, Paul M (2019) Neural temporality adaptation for document classification: Diachronic word embeddings and domain adaptation models. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp 4113–4123

  16. Li Z, Peng X, Zhang M, Wang R, Si L (2019) Semi-supervised domain adaptation for dependency parsing. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp 2386–2395

  17. Shu R, Bui H H, Narui H, Ermon S (2018) A dirt-t approach to unsupervised domain adaptation. In: Proceedings of the 6th International Conference on Learning Representations

  18. Ganin Y, Ustinova E, Ajakan H, Germain P, Larochelle H, Laviolette F, Marchand M, Lempitsky V (2016) Domain-adversarial training of neural networks. J Mach Learn Res 17(1):2096–2030

    MathSciNet  MATH  Google Scholar 

  19. Zhang K, Zhang H, Liu Q, Zhao H, Zhu H, Chen E (2019) Interactive attention transfer network for cross-domain sentiment classification. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol 33, pp 5773–5780

  20. Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A, Bengio Y (2014) Generative adversarial nets. In: Advances in Neural Information Processing Systems

  21. Peters M E, Neumann M, Iyyer M, Gardner M, Clark C, Lee K, Zettlemoyer L (2018) Deep contextualized word representations. In: Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics. Association for Computational Linguistics, pp 2227– 2237

  22. Radford A, Wu J, Child R, Luan D, Amodei D, Sutskever I (2019) Language models are unsupervised multitask learners. OpenAI Blog 1(8)

  23. Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez A N, Kaiser L, Polosukhin I (2017) Attention is all you need. In: Advances in neural information processing systems, pp 5998–6008

  24. Devlin J, Chang M-W, Lee K, Toutanova K (2019) Bert: Pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics, pp 4171–4186

  25. Yang Z, Dai Z, Yang Y, Carbonell J, Salakhutdinov R R, Le Q V (2019) Xlnet: Generalized autoregressive pretraining for language understanding. Adv Neural Inf Process Syst 32:5753– 5763

  26. Dai Z, Yang Z, Yang Y, Carbonell J G, Le Q, Salakhutdinov R (2019) Transformer-xl: Attentive language models beyond a fixed-length context. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp 2978– 2988

  27. Lan Z, Chen M, Goodman S, Gimpel K, Sharma P, Soricut R (2020) ALBERT: A lite BERT for self-supervised learning of language representations. In: 8th International Conference on Learning Representations. OpenReview.net

  28. Liu Y, Ott M, Goyal N, Du J, Joshi M, Chen D, Levy O, Lewis M, Zettlemoyer L, Stoyanov V (2019) Roberta: A robustly optimized bert pretraining approach. arXiv:1907.11692

  29. Sukhbaatar S, Weston J, Fergus R, et al. (2015) End-to-end memory networks. In: Advances in neural information processing systems, pp 2440–2448

  30. Yang Z, Yang D, Dyer C, He X, Smola A, Hovy E (2016) Hierarchical attention networks for document classification. In: Proceedings of the 2016 conference of the North American chapter of the association for computational linguistics: human language technologies. The Association for Computational Linguistics, pp 1480–1489

  31. Yue C, Cao H, Xu G, Dong Y (2021) Collaborative attention neural network for multi-domain sentiment classification. Appl Intell 51(6):3174–3188

    Article  Google Scholar 

  32. Liao W, Zeng B, Yin X, Wei P (2021) An improved aspect-category sentiment analysis model for text sentiment analysis based on roberta. Appl Intell 51(6):3522–3533

    Article  Google Scholar 

  33. Hovy D, Yang D (2021) The importance of modeling social factors of language: Theory and practice. In: Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp 588–602

  34. Tan Z, Chen J, Kang Q, Zhou M, Abusorrah A, Sedraoui K (2021) Dynamic embedding projection-gated convolutional neural networks for text classification. IEEE Transactions on Neural Networks and Learning Systems

  35. Zhou J, Huang J X, Hu Q V, He L (2020) Is position important? deep multi-task learning for aspect-based sentiment analysis. Appl Intell 50(10):3367–3378

    Article  Google Scholar 

  36. Ben-David S, Blitzer J, Crammer K, Kulesza A, Pereira F, Vaughan J W (2010) A theory of learning from different domains. Mach Learn 79(1):151–175

    Article  MathSciNet  MATH  Google Scholar 

  37. Bird S, Klein E, Loper E (2009) Natural language processing with python: analyzing text with the natural language toolkit. “O’Reilly Media, Inc.”

  38. Kingma D P, Ba J (2015) Adam: A method for stochastic optimization. In: Proceedings of the 3rd International Conference on Learning Representations

  39. Mikolov T, Chen K, Corrado G, Dean J (2013) Efficient estimation of word representations in vector space. In: Proceedings of the 1st International Conference on Learning Representations

  40. Sutskever I, Vinyals O, Le QV (2014) Sequence to sequence learning with neural networks. In: Advances in Neural Information Processing Systems 27: Annual Conference on Neural Information Processing Systems, pp 3104–3112

  41. Yin H, Liu P, Zhu Z, Li W, Wang Q (2019) Capsule network with identifying transferable knowledge for cross-domain sentiment classification. IEEE Access 7:153171–153182

    Article  Google Scholar 

  42. Manshu T, Xuemin Z (2019) Cchan: An end to end model for cross domain sentiment classification. IEEE Access 7:50232–50239

    Article  Google Scholar 

Download references

Acknowledgments

This research was supported in part by the National Key R&D Program of China, 2018YFB2101100, 2018YFB2101101, and NSFC under Grant No. 61972111.

Author information

Authors and Affiliations

  1. School of Computer Science and Technology, Harbin Institute of Technology, Shenzhen (HITSZ), Shenzhen, China

    Kuai Dai, Xutao Li, Xu Huang & Yunming Ye

Authors
  1. Kuai Dai
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xutao Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xu Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yunming Ye
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xutao Li.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Dai, K., Li, X., Huang, X. et al. SentATN: learning sentence transferable embeddings for cross-domain sentiment classification. Appl Intell 52, 18101–18114 (2022). https://doi.org/10.1007/s10489-022-03434-2

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-022-03434-2

Keywords

Search

Navigation