Ringki Das
Skip Abstract Section
Abstract
Sentiment analysis has come long way since it was introduced as a natural language processing task nearly 20 years ago. Sentiment analysis aims to extract the underlying attitudes and opinions toward an entity. It has become a powerful tool used by governments, businesses, medicine, marketing, and others. The traditional sentiment analysis model focuses mainly on text content. However, technological advances have allowed people to express their opinions and feelings through audio, image and video channels. As a result, sentiment analysis is shifting from unimodality to multimodality. Multimodal sentiment analysis brings new opportunities with the rapid increase of sentiment analysis as complementary data streams enable improved and deeper sentiment detection which goes beyond text-based analysis. Audio and video channels are included in multimodal sentiment analysis in terms of broadness. People have been working on different approaches to improve sentiment analysis system performance by employing complex deep neural architectures. Recently, sentiment analysis has achieved significant success using the transformer-based model. This paper presents a comprehensive study of different sentiment analysis approaches, applications, challenges, and resources then concludes that it holds tremendous potential. The primary motivation of this survey is to highlight changing trends in the unimodality to multimodality for solving sentiment analysis tasks.
- [1] . 2017. A survey of multimodal sentiment analysis. Image and Vision Computing 65 (2017), 3–14.Google Scholar
Cross Ref
- [2] . 2017. Current state of text sentiment analysis from opinion to emotion mining. ACM Computing Surveys (CSUR) 50, 2 (2017), 1–33.Google ScholarDigital Library
- [3] . 2003. Sentiment analysis: Capturing favorability using natural language processing. In Proceedings of the 2nd International Conference on Knowledge Capture. 70–77.Google ScholarDigital Library
- [4] . 2003. Mining the peanut gallery: Opinion extraction and semantic classification of product reviews. In Proceedings of the 12th International Conference on World Wide Web. 519–528.Google ScholarDigital Library
- [5] . 2002. Thumbs up?: Sentiment classification using machine learning techniques. In Proceedings of the ACL-02 Conference on Empirical Methods in Natural Language Processing-Volume 10. Association for Computational Linguistics, 79–86.Google ScholarDigital Library
- [6] . 2002. Thumbs up or thumbs down? Semantic orientation applied to unsupervised classification of reviews. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, Philadelphia, Pennsylvania, USA, 417–424.
DOI: Google ScholarDigital Library - [7] . 2015. Automated rule selection for aspect extraction in opinion mining. In Twenty-Fourth International Joint Conference on Artificial Intelligence.Google Scholar
- [8] . 2017. Sentiment analysis is a big suitcase. IEEE Intelligent Systems 32, 6 (2017), 74–80.Google ScholarCross Ref
- [9] . 2020. Survey on visual sentiment analysis. IET Image Processing 14, 8 (2020), 1440–1456.Google ScholarCross Ref
- [10] . 2018. Affective image content analysis: A comprehensive survey. (2018).Google Scholar
- [11] . 2019. An overview on image sentiment analysis: Methods, datasets and current challenges. ICETE (1) (2019), 296–306.Google Scholar
- [12] . 2020. Beneath the tip of the iceberg: Current challenges and new directions in sentiment analysis research. IEEE Transactions on Affective Computing (2020).Google ScholarDigital Library
- [13] . 2018. BERT: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018).Google Scholar
- [14] . 1907. RoBERTa: A robustly optimized BERT pretraining approach. arXiv 2019. arXiv preprint arXiv:1907.11692 (1907).Google Scholar
- [15] . 2019. A survey on sentiment analysis and opinion mining for social multimedia. Multimedia Tools and Applications 78, 6 (2019), 6939–6967.Google ScholarDigital Library
- [16] . 2023. Recent advances on social media analytics and multimedia systems: Issues and challenges. (2023).Google Scholar
- [17] . 2019. A survey of computational approaches and challenges in multimodal sentiment analysis. Int. J. Comput. Sci. Eng. 7, 1 (2019), 876–883.Google Scholar
- [18] . 2021. Multimodal sentimental analysis for social media applications: A comprehensive review. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery 11, 5 (2021), e1415.Google ScholarCross Ref
- [19] . 2022. Multimodal sentiment analysis: A survey and comparison. Research Anthology on Implementing Sentiment Analysis Across Multiple Disciplines (2022), 1846–1870.Google ScholarCross Ref
- [20] . 2022. Multimodal sentiment analysis: A systematic review of history, datasets, multimodal fusion methods, applications, challenges and future directions. Information Fusion (2022).Google Scholar
- [21] , , , and (Eds.). 2021. Proceedings of the First Workshop on Multimodal Machine Translation for Low Resource Languages (MMTLRL 2021). INCOMA Ltd., Online (Virtual Mode). https://aclanthology.org/2021.mmtlrl-1.0.Google Scholar
- [22] . 2010. Opinion-polarity identification in Bengali. In International Conference on Computer Processing of Oriental Languages. 169–182.Google Scholar
- [23] . 2014. Verb based Manipuri sentiment analysis. Int. J. Nat. Lang. Comput. 3, 3 (2014), 113–118.Google ScholarCross Ref
- [24] . 2021. A step towards sentiment analysis of Assamese news articles using lexical features. In Proceedings of the International Conference on Computing and Communication Systems: I3CS 2020, NEHU, Shillong, India, Vol. 170. Springer, 15.Google ScholarCross Ref
- [25] . 2012. Multilingual sentiment analysis using machine translation?. In Proceedings of the 3rd Workshop in Computational Approaches to Subjectivity and Sentiment Analysis. 52–60.Google ScholarDigital Library
- [26] . 2020. Cross-lingual deep neural transfer learning in sentiment analysis. Procedia Computer Science 176 (2020), 128–137.Google ScholarCross Ref
- [27] . 2007. Learning multilingual subjective language via cross-lingual projections. In Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics. 976–983.Google Scholar
- [28] . 2005. OpinionFinder: A system for subjectivity analysis. In Proceedings of HLT/EMNLP 2005 Interactive Demonstrations. 34–35.Google ScholarDigital Library
- [29] . 2008. Using SentiWordNet for multilingual sentiment analysis. In 2008 IEEE 24th International Conference on Data Engineering Workshop. IEEE, 507–512.Google ScholarCross Ref
- [30] . 2010. SentiWordNet 3.0: An enhanced lexical resource for sentiment analysis and opinion mining. In LREC, Vol. 10. 2200–2204.Google Scholar
- [31] . 2016. Multilingual sentiment analysis: State of the art and independent comparison of techniques. Cognitive Computation 8, 4 (2016), 757–771.Google ScholarCross Ref
- [32] . 2013. Improving sentiment analysis in Twitter using multilingual machine translated data. In Proceedings of the International Conference Recent Advances in Natural Language Processing RANLP 2013. 49–55.Google Scholar
- [33] . 2011. Emotion tokens: Bridging the gap among multilingual Twitter sentiment analysis. In Asia Information Retrieval Symposium. Springer, 238–249.Google Scholar
- [34] . 2012. Cross-lingual mixture model for sentiment classification. In Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 572–581.Google ScholarDigital Library
- [35] . 2011. Instance level transfer learning for cross lingual opinion analysis. In Proceedings of the 2nd Workshop on Computational Approaches to Subjectivity and Sentiment Analysis (WASSA 2.011). 182–188.Google ScholarDigital Library
- [36] . 2010. Boosting for transfer learning with multiple sources. In 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. IEEE, 1855–1862.Google ScholarCross Ref
- [37] . 2017. Multilingual sentiment analysis: From formal to informal and scarce resource languages. Artificial Intelligence Review 48, 4 (2017), 499–527.Google ScholarDigital Library
- [38] . 2020. A comparative study of machine translation for multilingual sentence-level sentiment analysis. Information Sciences 512 (2020), 1078–1102.Google ScholarDigital Library
- [39] . 2021. A survey of sentiment analysis in the Portuguese language. Artificial Intelligence Review 54, 2 (2021), 1087–1115.Google ScholarDigital Library
- [40] . 2004. Mining and summarizing customer reviews. In Proceedings of the Tenth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD’04). Association for Computing Machinery, New York, NY, USA, 168–177.
DOI: Google ScholarDigital Library - [41] . 2018. Machine learning-based sentiment analysis for Twitter accounts. Mathematical and Computational Applications 23, 1 (2018), 11.Google ScholarCross Ref
- [42] . 2017. Weakly-supervised deep embedding for product review sentiment analysis. IEEE Transactions on Knowledge and Data Engineering 30, 1 (2017), 185–197.Google ScholarCross Ref
- [43] . 2015. Implicit feature identification in Chinese reviews using explicit topic mining model. Knowledge-Based Systems 76 (2015), 166–175.Google ScholarDigital Library
- [44] . 2015. Sentence compression for aspect-based sentiment analysis. IEEE/ACM Transactions on Audio, Speech, and Language Processing 23, 12 (2015), 2111–2124.Google ScholarDigital Library
- [45] . 2020. Multi-task learning for aspect term extraction and aspect sentiment classification. Neurocomputing 398 (2020), 247–256.Google ScholarCross Ref
- [46] . 2016. Attention-based LSTM for aspect-level sentiment classification. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing. 606–615.Google ScholarCross Ref
- [47] . 2004. Determining the sentiment of opinions. In COLING 2004: Proceedings of the 20th International Conference on Computational Linguistics. 1367–1373.Google ScholarDigital Library
- [48] . 1995. WordNet: A lexical database for English. Commun. ACM 38, 11 (1995), 39–41.Google ScholarDigital Library
- [49] . 2009. Generating high-coverage semantic orientation lexicons from overtly marked words and a thesaurus. In Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing. 599–608.Google ScholarDigital Library
- [50] . 2011. Lexicon-based methods for sentiment analysis. Computational Linguistics 37, 2 (2011), 267–307.Google ScholarDigital Library
- [51] . 2015. A framework for sentiment analysis in Hindi using HSWN. International Journal of Computer Applications 119, 19 (2015).Google ScholarCross Ref
- [52] . 2010. DASA: Dissatisfaction-oriented advertising based on sentiment analysis. Expert Systems with Applications 37, 9 (2010), 6182–6191.Google ScholarDigital Library
- [53] . 2010. Exploring the sentiment strength of user reviews. In International Conference on Web-Age Information Management. Springer, 471–482.Google ScholarCross Ref
- [54] . 2014. Sentiment analysis algorithms and applications: A survey. Ain Shams Engineering Journal 5, 4 (2014), 1093–1113.Google ScholarCross Ref
- [55] . 2004. A sentimental education: Sentiment analysis using subjectivity summarization based on minimum cuts(
ACL’04 ). Association for Computational Linguistics, USA, 271–es.DOI: Google ScholarDigital Library - [56] . 2006. Comparative experiments on sentiment classification for online product reviews. In AAAI’06: Proceedings of the 21st National Conference on Artificial Intelligence - Volume 2, July 2006 Pages 1265–1270. 1–6.Google Scholar
- [57] . 2004. Towards a robust metric of opinion. In AAAI Spring Symposium on Exploring Attitude and Affect in Text, Vol. 598603.Google Scholar
- [58] . 2005. Analysis of polarity information in medical text. In AMIA Annual Symposium Proceedings, Vol. 2005. American Medical Informatics Association, 570.Google Scholar
- [59] . 2011. Sentiment classification of internet restaurant reviews written in Cantonese. Expert Systems with Applications 38, 6 (2011), 7674–7682.Google ScholarDigital Library
- [60] . 2021. Review comments of Manipuri online video: Good, bad or ugly. In Proceedings of the International Conference on Computing and Communication Systems: I3CS 2020, NEHU, Shillong, India, Vol. 170. Springer, 45.Google ScholarCross Ref
- [61] . 2021. Low resource language specific pre-processing and features for sentiment analysis task. Language Resources and Evaluation (2021), 1–23.Google Scholar
- [62] . 2014. Sentic patterns: Dependency-based rules for concept-level sentiment analysis. Knowledge-Based Systems 69 (2014), 45–63.Google ScholarDigital Library
- [63] . 2016. A hybrid approach to the sentiment analysis problem at the sentence level. Know.-Based Syst. 108, C (
Sept. 2016), 110–124.DOI: Google ScholarDigital Library - [64] . 2014. Sentiment analysis in Facebook and its application to e-learning. Computers in Human Behavior 31 (2014), 527–541.Google ScholarDigital Library
- [65] . 2016. Deep learning for sentiment analysis. Language and Linguistics Compass 10, 12 (2016), 701–719.Google ScholarCross Ref
- [66] . 2016. A study of sentiment analysis using deep learning techniques on Thai Twitter data. In 2016 13th International Joint Conference on Computer Science and Software Engineering (JCSSE). IEEE, 1–6.Google ScholarCross Ref
- [67] . 2006. A fast learning algorithm for deep belief nets. Neural Computation 18, 7 (2006), 1527–1554.Google ScholarDigital Library
- [68] . 2013. Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing. 1631–1642.Google Scholar
- [69] . 2018. Fine-tuning convolutional neural networks for fine art classification. Expert Systems with Applications 114 (2018), 107–118.Google ScholarCross Ref
- [70] . 2019. Sentiment classification using convolutional neural networks. Applied Sciences 9, 11 (2019), 2347.Google ScholarCross Ref
- [71] . 2017. Deep sentiment representation based on CNN and LSTM. In International Conference on Green Informatics (ICGI).30–33.Google ScholarCross Ref
- [72] . 2019. A CNN-BiLSTM model for document-level sentiment analysis. Machine Learning and Knowledge Extraction 1, 3 (2019), 832–847.Google ScholarCross Ref
- [73] . 2018. Sentiment analysis of Indian languages using convolutional neural networks. In 2018 International Conference on Computer Communication and Informatics (ICCCI). IEEE, 1–4.Google Scholar
- [74] . 2020. NITS-Hinglish-SentiMix at SemEval-2020 task 9: Sentiment analysis for code-mixed social media text using an ensemble model. (
Dec. 2020), 1298–1303. https://aclanthology.org/2020.semeval-1.175.Google Scholar - [75] . 2016. Hierarchical attention networks for document classification. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 1480–1489.Google ScholarCross Ref
- [76] . 2018. LSTM with sentence representations for document-level sentiment classification. Neurocomputing 308 (2018), 49–57.Google ScholarDigital Library
- [77] . 2019. A survey of sentiment analysis based on transfer learning. IEEE Access 7 (2019), 85401–85412.Google ScholarCross Ref
- [78] . 2022. A survey on sentiment analysis methods, applications, and challenges. Artificial Intelligence Review (2022), 1–50.Google Scholar
- [79] . 2018. Financial aspect-based sentiment analysis using deep representations. arXiv preprint arXiv:1808.07931 (2018).Google Scholar
- [80] . 2018. Universal language model fine-tuning for text classification. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 328–339.Google ScholarCross Ref
- [81] . 2019. Aspect-based sentiment analysis using BERT. In Proceedings of the 22nd Nordic Conference on Computational Linguistics. 187–196.Google Scholar
- [82] . 2020. Improving BERT performance for aspect-based sentiment analysis. arXiv preprint arXiv:2010.11731 (2020).Google Scholar
- [83] . 2018. Deep contextualized word representations. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers). Association for Computational Linguistics, New Orleans, Louisiana, 2227–2237.
DOI: Google ScholarCross Ref - [84] . 2022. More than a feeling: Accuracy and application of sentiment analysis. International Journal of Research in Marketing (2022).Google Scholar
- [85] . 2010. Object bank: A high-level image representation for scene classification & semantic feature sparsification. Advances in Neural Information Processing Systems 23 (2010).Google Scholar
- [86] . 2014. Object-based visual sentiment concept analysis and application. In Proceedings of the 22nd ACM International Conference on Multimedia (MM’14). Association for Computing Machinery, New York, NY, USA, 367–376.
DOI: Google ScholarDigital Library - [87] . 2013. Sentribute: Image sentiment analysis from a mid-level perspective. In Proceedings of the Second International Workshop on Issues of Sentiment Discovery and Opinion Mining. 1–8.Google ScholarDigital Library
- [88] . 2013. Large-scale visual sentiment ontology and detectors using adjective noun pairs. In Proceedings of the 21st ACM International Conference on Multimedia. 223–232.Google ScholarDigital Library
- [89] . 2013. SentiBank: Large-scale ontology and classifiers for detecting sentiment and emotions in visual content. In Proceedings of the 21st ACM International Conference on Multimedia. 459–460.Google ScholarDigital Library
- [90] . 1980. A general psychoevolutionary theory of emotion. In Theories of Emotion. Elsevier, 3–33.Google ScholarCross Ref
- [91] . 2016. Visual sentiment topic model based microblog image sentiment analysis. Multimedia Tools and Applications 75, 15 (2016), 8955–8968.Google ScholarDigital Library
- [92] . 2018. Image sentiment prediction based on textual descriptions with adjective noun pairs. Multimedia Tools and Applications 77, 1 (2018), 1115–1132.Google ScholarDigital Library
- [93] . 2014. Predicting viewer affective comments based on image content in social media. In Proceedings of International Conference on Multimedia Retrieval. 233–240.Google ScholarDigital Library
- [94] . 2020. Exploiting objective text description of images for visual sentiment analysis. Multimedia Tools and Applications (2020), 1–24.Google Scholar
- [95] . 2015. The effect of whitening transformation on pooling operations in convolutional autoencoders. EURASIP Journal on Advances in Signal Processing 2015, 1 (2015), 1–11.Google ScholarCross Ref
- [96] . 2015. Robust image sentiment analysis using progressively trained and domain transferred deep networks. In Twenty-ninth AAAI Conference on Artificial Intelligence.Google ScholarDigital Library
- [97] . 2015. Image sentiment analysis using deep convolutional neural networks with domain specific fine tuning. In 2015 International Conference on Information Processing (ICIP). IEEE, 447–451.Google ScholarCross Ref
- [98] . 2017. ImageNet classification with deep convolutional neural networks. Commun. ACM 60, 6 (2017), 84–90.Google ScholarDigital Library
- [99] . 2017. From pixels to sentiment: Fine-tuning CNNs for visual sentiment prediction. Image and Vision Computing 65 (2017), 15–22.Google ScholarDigital Library
- [100] . 2014. Caffe: Convolutional architecture for fast feature embedding. In Proceedings of the 22nd ACM International Conference on Multimedia. 675–678.Google ScholarDigital Library
- [101] . 2010. Analyzing and predicting sentiment of images on the social web. In Proceedings of the 18th ACM International Conference on Multimedia. 715–718.Google ScholarDigital Library
- [102] . 2006. SentiWordNet: A publicly available lexical resource for opinion mining. In Proceedings of the Fifth International Conference on Language Resources and Evaluation (LREC’06).Google Scholar
- [103] . 2016. Beyond object recognition: Visual sentiment analysis with deep coupled adjective and noun neural networks. In IJCAI. 3484–3490.Google Scholar
- [104] . 2021. Visual sentiment analysis based on image caption and adjective–noun–pair description. Soft Computing (2021), 1–13.Google Scholar
- [105] . 2015. Visual affect around the world: A large-scale multilingual visual sentiment ontology. In Proceedings of the 23rd ACM International Conference on Multimedia. 159–168.Google ScholarDigital Library
- [106] . 2016. Complura: Exploring and leveraging a large-scale multilingual visual sentiment ontology. In Proceedings of the 2016 ACM on International Conference on Multimedia Retrieval. 417–420.Google ScholarDigital Library
- [107] . 2017. Towards using visual attributes to infer image sentiment of social events. In 2017 International Joint Conference on Neural Networks (IJCNN). IEEE, 1372–1379.Google ScholarCross Ref
- [108] . 2016. SentiCap: Generating image descriptions with sentiments. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 30.Google ScholarCross Ref
- [109] . 2017. Visual sentiment analysis by attending on local image regions. In Thirty-First AAAI Conference on Artificial Intelligence.Google ScholarDigital Library
- [110] . 2018. Boosting image sentiment analysis with visual attention. Neurocomputing 312 (2018), 218–228.Google ScholarDigital Library
- [111] . 2012. Sentiment analysis of call centre audio conversations using text classification. International Journal of Computer Information Systems and Industrial Management Applications 4, 1 (2012), 619–627.Google Scholar
- [112] . 2015. Automatic audio sentiment extraction using keyword spotting. In Sixteenth Annual Conference of the International Speech Communication Association.Google ScholarCross Ref
- [113] . 2017. Automatic sentiment detection in naturalistic audio. IEEE/ACM Transactions on Audio, Speech, and Language Processing 25, 8 (2017), 1668–1679.Google ScholarDigital Library
- [114] . 2017. Sentiment analysis using image-based deep spectrum features. In 2017 Seventh International Conference on Affective Computing and Intelligent Interaction Workshops and Demos (ACIIW). IEEE, 26–29.Google ScholarCross Ref
- [115] . 2016. Improved multimodal sentiment detection using stressed regions of audio. In 2016 IEEE Region 10 Conference (TENCON). IEEE, 2834–2837.Google ScholarCross Ref
- [116] . 2016. AudioSentiBank: Large-scale semantic ontology of acoustic concepts for audio content analysis. arXiv preprint arXiv:1607.03766 (2016).Google Scholar
- [117] . 2014. Sentiment retrieval on web reviews using spontaneous natural speech. In 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 4583–4587.Google ScholarCross Ref
- [118] . 2014. Microblog sentiment analysis based on cross-media bag-of-words model. In Proceedings of International Conference on Internet Multimedia Computing and Service (ICIMCS’14). Association for Computing Machinery, New York, NY, USA, 76–80.
DOI: Google ScholarDigital Library - [119] . 2015. A cross-media sentiment analytics platform for microblog(
MM’15 ). Association for Computing Machinery, New York, NY, USA, 767–769.DOI: Google ScholarDigital Library - [120] . 2015. Joint visual-textual sentiment analysis with deep neural networks(
MM’15 ). Association for Computing Machinery, New York, NY, USA, 1071–1074.DOI: Google ScholarDigital Library - [121] . 2019. An image-text consistency driven multimodal sentiment analysis approach for social media. Information Processing & Management 56, 6 (2019), 102097.Google ScholarDigital Library
- [122] . 2020. Convolutional transformer with sentiment-aware attention for sentiment analysis. In 2020 International Joint Conference on Neural Networks (IJCNN). 1–8.
DOI: Google ScholarCross Ref - [123] . 2022. A multi-stage multimodal framework for sentiment analysis of Assamese in low resource setting. Expert Systems with Applications (2022), 117575.Google ScholarDigital Library
- [124] . 2023. Image-text multimodal sentiment analysis framework of Assamese news articles using late fusion. ACM Trans. Asian Low-Resour. Lang. Inf. Process. (
Feb. 2023).DOI: Just Accepted. Google ScholarDigital Library - [125] . 2018. Domain attention model for multi-domain sentiment classification. Knowledge-Based Systems 155 (2018), 1–10.Google ScholarCross Ref
- [126] . 2016. Robust visual-textual sentiment analysis: When attention meets tree-structured recursive neural networks. In Proceedings of the 24th ACM International Conference on Multimedia. 1008–1017.Google ScholarDigital Library
- [127] . 2019. Image–text sentiment analysis via deep multimodal attentive fusion. Knowledge-Based Systems 167 (2019), 26–37.Google ScholarDigital Library
- [128] . 2019. Adapting BERT for target-oriented multimodal sentiment classification. IJCAI.Google Scholar
- [129] . 2015. Multimodal sentiment analysis: Sentiment analysis using audiovisual format. In 2015 2nd International Conference on Computing for Sustainable Global Development (INDIACom). IEEE, 1415–1419.Google Scholar
- [130] . 2017. Audio-visual sentiment analysis for learning emotional arcs in movies. In 2017 IEEE International Conference on Data Mining (ICDM). IEEE, 829–834.Google ScholarCross Ref
- [131] . 2014. Predicting emotions in user-generated videos. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 28.Google ScholarCross Ref
- [132] . 2021. A novel context-aware multimodal framework for Persian sentiment analysis. Neurocomputing 457 (2021), 377–388.Google ScholarDigital Library
- [133] . 2011. Towards multimodal sentiment analysis: Harvesting opinions from the web. In Proceedings of the 13th International Conference on Multimodal Interfaces. 169–176.Google ScholarDigital Library
- [134] . 2017. Benchmarking multimodal sentiment analysis. In International Conference on Computational Linguistics and Intelligent Text Processing. Springer, 166–179.Google Scholar
- [135] . 2015. Deep convolutional neural network textual features and multiple kernel learning for utterance-level multimodal sentiment analysis. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. 2539–2544.Google ScholarDigital Library
- [136] . 2016. Fusing audio, visual and textual clues for sentiment analysis from multimodal content. Neurocomputing 174 (2016), 50–59.Google ScholarDigital Library
- [137] . 2013. Multimodal sentiment analysis of Spanish online videos. IEEE Intelligent Systems 28, 3 (2013), 38–45.Google ScholarDigital Library
- [138] . 2013. YouTube movie reviews: Sentiment analysis in an audio-visual context. IEEE Intelligent Systems 28, 3 (2013), 46–53.Google ScholarDigital Library
- [139] . 2018. Multimodal sentiment analysis using hierarchical fusion with context modeling. Knowledge-Based Systems 161 (2018), 124–133.Google ScholarCross Ref
- [140] . 2015. Exploiting multimodal affect and semantics to identify politically persuasive web videos. In Proceedings of the 2015 ACM on International Conference on Multimodal Interaction (ICMI’15). Association for Computing Machinery, New York, NY, USA, 203–210.
DOI: Google ScholarDigital Library - [141] . 2016. Fusing audio, textual, and visual features for sentiment analysis of news videos. In Tenth International AAAI Conference on Web and Social Media.Google Scholar
- [142] . 2014. Why we watch the news: A dataset for exploring sentiment in broadcast video news. In Proceedings of the 16th International Conference on Multimodal Interaction (ICMI’14). Association for Computing Machinery, New York, NY, USA, 104–111.
DOI: Google ScholarDigital Library - [143] . 2021. Attention-based multimodal contextual fusion for sentiment and emotion classification using bidirectional LSTM. Multimedia Tools and Applications 80, 9 (2021), 13059–13076.Google ScholarDigital Library
- [144] . 2017. Tensor fusion network for multimodal sentiment analysis. arXiv preprint arXiv:1707.07250 (2017).Google Scholar
- [145] . 2020. Multi-modal embeddings using multi-task learning for emotion recognition. arXiv preprint arXiv:2009.05019 (2020).Google Scholar
- [146] . 2021. Efficient multi-modal embeddings from structured data. arXiv preprint arXiv:2110.02577 (2021).Google Scholar
- [147] . 2016. Training and evaluating multimodal word embeddings with large-scale web annotated images. Advances in Neural Information Processing Systems 29 (2016).Google Scholar
- [148] . 2021. Cross-modality transfer learning for image-text information management. ACM Transactions on Management Information System (TMIS) 13, 1 (2021), 1–14.Google Scholar
- [149] . 1963. A computer approach to content analysis: Studies using the general inquirer system. In Proceedings of the May 21-23, 1963, Spring Joint Computer Conference (AFIPS’63 (Spring)). Association for Computing Machinery, New York, NY, USA, 241–256.
DOI: Google ScholarDigital Library - [150] . 2004. WordNet Affect: An affective extension of WordNet. In Proceedings of the Fourth International Conference on Language Resources and Evaluation (LREC’04). European Language Resources Association (ELRA), Lisbon, Portugal. http://www.lrec-conf.org/proceedings/lrec2004/pdf/369.pdf.Google Scholar
- [151] . 2005. Annotating expressions of opinions and emotions in language. Language Resources and Evaluation 39, 2 (2005), 165–210.Google ScholarCross Ref
- [152] . 2005. Opinion observer: Analyzing and comparing opinions on the web. In Proceedings of the 14th International Conference on World Wide Web. 342–351.Google ScholarDigital Library
- [153] . 2013. Enhanced SenticNet with affective labels for concept-based opinion mining. IEEE Intelligent Systems 28, 2 (2013), 31–38.Google ScholarDigital Library
- [154] . 2021. Sentiment classification and aspect-based sentiment analysis on Yelp reviews using deep learning and word embeddings. Journal of Decision Systems 30, 2-3 (2021), 259–281.Google ScholarCross Ref
- [155] . 2011. Learning word vectors for sentiment analysis. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies. 142–150.Google ScholarDigital Library
- [156] . 2015. Document modeling with gated recurrent neural network for sentiment classification. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. 1422–1432.Google ScholarCross Ref
- [157] . 2013. Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, Seattle, Washington, USA, 1631–1642. https://aclanthology.org/D13-1170.Google Scholar
- [158] . 2017. Improving sentiment analysis via sentence type classification using BiLSTM-CRF and CNN. Expert Systems with Applications 72 (2017), 221–230.Google ScholarDigital Library
- [159] . 2007. Biographies, Bollywood, boom-boxes and blenders: Domain adaptation for sentiment classification. In Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics. 440–447.Google Scholar
- [160] . 2018. Sentiment analysis on large scale Amazon product reviews. In 2018 IEEE International Conference on Innovative Research and Development (ICIRD). IEEE, 1–6.Google ScholarCross Ref
- [161] . 2007. SemEval-2007 task 14: Affective text. In Proceedings of the 4th International Workshop on Semantic Evaluations (SemEval’07). Association for Computational Linguistics, USA, 70–74.Google ScholarCross Ref
- [162] . 2016. Cross-modality consistent regression for joint visual-textual sentiment analysis of social multimedia. In Proceedings of the Ninth ACM International Conference on Web Search and Data Mining. 13–22.Google ScholarDigital Library
- [163] . 2016. MOSI: Multimodal corpus of sentiment intensity and subjectivity analysis in online opinion videos. arXiv preprint arXiv:1606.06259 (2016).Google Scholar
- [164] . 2018. Contextual inter-modal attention for multi-modal sentiment analysis. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. 3454–3466.Google ScholarCross Ref
- [165] . 2020. Gated mechanism for attention based multi modal sentiment analysis. In ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 4477–4481.Google ScholarCross Ref
- [166] . 2013. Utterance-level multimodal sentiment analysis. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 973–982.Google Scholar
- [167] . 2017. Select-additive learning: Improving generalization in multimodal sentiment analysis. In 2017 IEEE International Conference on Multimedia and Expo (ICME). IEEE, 949–954.Google ScholarCross Ref
- [168] . 2018. Deep sentiment hashing for text retrieval in social CIoT. Future Generation Computer Systems 86 (2018), 362–371.Google ScholarDigital Library
- [169] . 2018. Convolutional recurrent deep learning model for sentence classification. IEEE Access 6 (2018), 13949–13957.Google ScholarCross Ref
- [170] . 2019. MOCA: Multi-objective, collaborative, and attentive sentiment analysis. IEEE Access 7 (2019), 10927–10936.Google ScholarCross Ref
- [171] . 2019. On interpretability and feature representations: An analysis of the sentiment neuron. In European Conference on Information Retrieval. Springer, 795–802.Google ScholarDigital Library
- [172] . 2019. Adding prior knowledge in hierarchical attention neural network for cross domain sentiment classification. IEEE Access 7 (2019), 32578–32588.Google ScholarCross Ref
- [173] . 2017. Context-dependent sentiment analysis in user-generated videos. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (volume 1: Long Papers). 873–883.Google ScholarCross Ref
- [174] . 2019. Video-based sentiment analysis with hvnLBP-TOP feature and bi-LSTM. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 33. 9963–9964.Google ScholarDigital Library
- [175] . 2017. Multi-level multiple attentions for contextual multimodal sentiment analysis. In 2017 IEEE International Conference on Data Mining (ICDM). IEEE, 1033–1038.Google ScholarCross Ref
- [176] . 2017. Dyadic memory networks for aspect-based sentiment analysis. In Proceedings of the 2017 ACM on Conference on Information and Knowledge Management. 107–116.Google ScholarDigital Library
- [177] . 2018. Towards Twitter sentiment classification by multi-level sentiment-enriched word embeddings. Neurocomputing 275 (2018), 2459–2466.Google ScholarDigital Library
- [178] . 2016. Sentiment analysis on multi-view social data. In International Conference on Multimedia Modeling. Springer, 15–27.Google ScholarCross Ref
- [179] . 2019. Fuzzy commonsense reasoning for multimodal sentiment analysis. Pattern Recognition Letters 125 (2019), 264–270.Google ScholarDigital Library
- [180] . 2015. Deep learning for sentiment analysis: Successful approaches and future challenges. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery 5, 6 (2015), 292–303.Google ScholarDigital Library
- [181] . 2016. YZU-NLP team at SemEval-2016 task 4: Ordinal sentiment classification using a recurrent convolutional network. In Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval-2016). 251–255.Google ScholarCross Ref
- [182] . 2019. A roadmap towards implementing parallel aspect level sentiment analysis. Multimedia Tools and Applications 78, 20 (2019), 29463–29492.Google ScholarDigital Library
- [183] . 2017. Deep sequential model for review rating prediction. In 2017 Tenth International Conference on Contemporary Computing (IC3). IEEE, 1–6.Google ScholarCross Ref
- [184] . 2017. An effective gated and attention-based neural network model for fine-grained financial target-dependent sentiment analysis. In International Conference on Knowledge Science, Engineering and Management. Springer, 42–54.Google ScholarCross Ref
- [185] . 2010. Opinion digger: An unsupervised opinion miner from unstructured product reviews. In Proceedings of the 19th ACM International Conference on Information and Knowledge Management. 1825–1828.Google ScholarDigital Library
- [186] . 2009. Multi-facet rating of product reviews. In European Conference on Information Retrieval. Springer, 461–472.Google ScholarDigital Library
- [187] . 2014. Clustering aspect-related phrases by leveraging sentiment distribution consistency. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP). 1614–1623.Google ScholarCross Ref
- [188] . 2022. Sememe knowledge and auxiliary information enhanced approach for sarcasm detection. Information Processing & Management 59, 3 (2022), 102883.Google ScholarDigital Library
- [189] . 2021. A comprehensive survey on sentiment analysis: Approaches, challenges and trends. Knowledge-Based Systems 226 (2021), 107134.Google ScholarCross Ref
- [190] . 2017. Viraliency: Pooling local virality. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 6080–6088.Google ScholarCross Ref
Index Terms
- Multimodal Sentiment Analysis: A Survey of Methods, Trends, and Challenges
Recommendations
Multimodal Social Media Sentiment Analysis Based on Cross-Modal Hierarchical Attention Fusion
Artificial Intelligence and Mobile Services – AIMS 2021AbstractWith the diversification of data forms on social media, more and more multimodal information mixed with image and text replaces the traditional single text description. Compared with single-modal data, multimodal data can more fully express people’...
A comprehensive survey on deep learning-based approaches for multimodal sentiment analysis
AbstractSentiment analysis is an important natural language processing issue that has many applications in various fields. The increasing popularity of social networks and growth and development of their related tools and technologies has led to share the ...
Multimodal Video Sentiment Analysis Using Deep Learning Approaches, a Survey
Highlights- Multimodal sentiment analysis survey.
- New categorization of 35 models according ...
ABSTRACTDeep learning has emerged as a powerful machine learning technique to employ in multimodal sentiment analysis tasks. In the recent years, many deep learning models and various algorithms have been proposed in the field of multimodal ...
Comments