Upendra Singh
Hiteshwar Kumar Azad
Skip Abstract Section
Abstract
The rapid growth of the internet has reached the fourth generation, i.e., web 4.0, which supports Sentiment Analysis (SA) in many applications such as social media, marketing, risk management, healthcare, businesses, websites, data mining, e-learning, psychology, and many more. Sentiment analysis is a powerful tool for governments, businesses, and researchers to analyse users’ emotions and mental states in order to generate opinions and reviews about products, services, and daily activities. In the past years, several SA techniques based on Machine Learning (ML), Deep Learning (DL), and other soft computing approaches were proposed. However, growing data size, subjectivity, and diversity pose a significant challenge to enhancing the efficiency of existing techniques and incorporating current development trends, such as Multimodal Sentiment Analysis (MSA) and fusion techniques. With the aim of assisting the enthusiastic researcher to navigating the current trend, this article presents a comprehensive study of various literature to handle different aspects of SA, including current trends and techniques across multiple domains. In order to clarify the future prospects of MSA, this article also highlights open issues and research directions that lead to a number of unresolved challenges.
- [1] . 2021. Multimodal video sentiment analysis using deep learning approaches, a survey. Information Fusion 76 (2021), 204–226.Google Scholar
Digital Library
- [2] . 2018. A survey of textual emotion detection. In 2018 8th International Conference on Computer Science and Information Technology (CSIT). IEEE, 136–142.Google ScholarCross Ref
- [3] . 2014. Predicting personality traits using multimodal information. In Proceedings of the 2014 ACM Multi Media on Workshop on Computational Personality Recognition. 15–18.Google ScholarDigital Library
- [4] . 2017. Emotion recognition based on EEG using LSTM recurrent neural network. Emotion 8, 10 (2017), 355–358.Google Scholar
- [5] . 2020. A survey of state-of-the-art approaches for emotion recognition in text. Knowledge and Information Systems (2020), 1–51.Google Scholar
- [6] . 2018. Learning to generate 3D stylized character expressions from humans. In 2018 IEEE Winter Conference on Applications of Computer Vision (WACV). IEEE, 160–169.Google ScholarCross Ref
- [7] . 2008. The American Sign Language lexicon video dataset. In 2008 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops. IEEE, 1–8.Google ScholarCross Ref
- [8] . 2010. Multimodal fusion for multimedia analysis: A survey. Multimedia Systems 16, 6 (2010), 345–379.Google ScholarDigital Library
- [9] . 2016. SoundNet: Learning sound representations from unlabeled video. In Advances in Neural Information Processing Systems. 892–900.Google ScholarDigital Library
- [10] . 2010. SentiWordNet 3.0: An enhanced lexical resource for sentiment analysis and opinion mining. In LREC, Vol. 10. 2200–2204.Google Scholar
- [11] . 2014. Computational approaches to subjectivity and sentiment analysis: Present and envisaged methods and applications. Computer Speech & Language 28, 1 (2014), 1–6.Google Scholar
- [12] . 2016. OpenFace: An open source facial behavior analysis toolkit. In 2016 IEEE Winter Conference on Applications of Computer Vision (WACV). IEEE, 1–10.Google ScholarCross Ref
- [13] . 2017. Lexicon generation for emotion detection from text. IEEE Intelligent Systems 32, 1 (2017), 102–108.Google ScholarDigital Library
- [14] . 2015. The SenticNet Sentiment Lexicon: Exploring Semantic Richness in Multi-Word Concepts. Ph. D. Dissertation. Springer.Google Scholar
- [15] . 2021. A comprehensive survey on sentiment analysis: Approaches, challenges and trends. Knowledge-Based Systems 226 (2021), 107134.Google ScholarCross Ref
- [16] . 2016. A non-EEG biosignals dataset for assessment and visualization of neurological status. In 2016 IEEE International Workshop on Signal Processing Systems (SiPS). IEEE, 110–114.Google ScholarCross Ref
- [17] . 2012. G3D: A gaming action dataset and real time action recognition evaluation framework. In 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops. IEEE, 7–12.Google ScholarCross Ref
- [18] . 2001. Speak and unSpeak with PRAAT. Glot International 5, 9/10 (2001), 341–347.Google Scholar
- [19] . 2009. A machine learning approach to sentiment analysis in multilingual Web texts. Information Retrieval 12, 5 (2009), 526–558.Google ScholarDigital Library
- [20] . 2019. COMET: Commonsense transformers for knowledge graph construction. In Association for Computational Linguistics (ACL) 4762–4779.Google ScholarCross Ref
- [21] . 2016. Learning to decode human emotions with Echo State Networks. Neural Networks 78 (2016), 112–119.Google ScholarDigital Library
- [22] . 2014. Meta-level sentiment models for big social data analysis. Knowledge-Based Systems 69 (2014), 86–99.Google ScholarDigital Library
- [23] . 2005. A database of German emotional speech. In Ninth European Conference on Speech Communication and Technology.Google ScholarCross Ref
- [24] . 2008. IEMOCAP: Interactive emotional dyadic motion capture database. Language Resources and Evaluation 42, 4 (2008), 335.Google ScholarCross Ref
- [25] . 2011. Iterative feature normalization for emotional speech detection. In 2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 5692–5695.Google ScholarCross Ref
- [26] . 2017. A Practical Guide to Sentiment Analysis. Springer.Google ScholarCross Ref
- [27] . 2013. Sentic blending: Scalable multimodal fusion for the continuous interpretation of semantics and sentics. In 2013 IEEE Symposium on Computational Intelligence for Human-Like Intelligence (CIHLI). IEEE, 108–117.Google ScholarCross Ref
- [28] . 2014. SenticNet 3: A common and common-sense knowledge base for cognition-driven sentiment analysis. In Proceedings of the Twenty-Eighth AAAI Conference on Artificial Intelligence. 1515–1521.Google ScholarDigital Library
- [29] . 2013. Natural interaction expressivity modeling and analysis. In Proceedings of the 6th International Conference on PErvasive Technologies Related to Assistive Environments. 1–6.Google ScholarDigital Library
- [30] . 2019. SemEval-2019 Task 3: Emocontext contextual emotion detection in text. In Proceedings of the 13th International Workshop on Semantic Evaluation. 39–48.Google ScholarCross Ref
- [31] . 2020. Minority positive sampling for switching points-an anecdote for the code-mixing language modeling. In Proceedings of the 12th Language Resources and Evaluation Conference. 6228–6236.Google Scholar
- [32] . 2017. Detecting variation of emotions in online activities. Expert Systems with Applications 89 (2017), 318–332.Google ScholarDigital Library
- [33] . 1998. Emotion recognition from audiovisual information. In 1998 IEEE Second Workshop on Multimedia Signal Processing (Cat. No. 98EX175). IEEE, 83–88.Google ScholarCross Ref
- [34] . 2012. A new 6D motion gesture database and the benchmark results of feature-based statistical recognition. In 2012 IEEE International Conference on Emerging Signal Processing Applications. IEEE, 131–134.Google ScholarCross Ref
- [35] . 2017. Multimodal sentiment analysis with word-level fusion and reinforcement learning. In Proceedings of the 19th ACM International Conference on Multimodal Interaction. 163–171.Google ScholarDigital Library
- [36] . 2014. DeepSentiBank: Visual sentiment concept classification with deep convolutional neural networks. arXiv preprint arXiv:1410.8586 (2014).Google Scholar
- [37] . 2018. QuAC: Question answering in context. arXiv preprint arXiv:1808.07036 (2018).Google Scholar
- [38] . 2017. Coarse-grained+/-effect word sense disambiguation for implicit sentiment analysis. IEEE Transactions on Affective Computing 8, 4 (2017), 471–479.Google ScholarCross Ref
- [39] . 2013. Toward a 3D body part detection video dataset and hand tracking benchmark. In Proceedings of the 6th International Conference on PErvasive Technologies Related to Assistive Environments. 1–6.Google ScholarDigital Library
- [40] . 2001. Active appearance models. IEEE Transactions on Pattern Analysis and Machine Intelligence 23, 6 (2001), 681–685.Google ScholarDigital Library
- [41] . 2021. Dempster-Shafer-based fusion of multi-modal biometrics for supporting identity verification effectively and efficiently. In 2021 IEEE 2nd International Conference on Human-Machine Systems (ICHMS). IEEE, 1–8.Google ScholarCross Ref
- [42] . 2014. Tweet sentiment analysis with classifier ensembles. Decision Support Systems 66 (2014), 170–179.Google ScholarDigital Library
- [43] . 2023. Multimodal sentiment analysis: A survey of methods, trends and challenges. Comput. Surveys (2023).Google ScholarDigital Library
- [44] . 2009. Multimodal recognition of emotions in car environments. DCI&I 2009 (2009).Google Scholar
- [45] . 2014. Semantic audio-visual data fusion for automatic emotion recognition. Emotion Recognition: A Pattern Analysis Approach (2014), 411–435.Google Scholar
- [46] . 2016. Multi-class emotions classification by sentic levels as features in sentiment analysis. In 2016 5th Brazilian Conference on Intelligent Systems (BRACIS). IEEE, 486–491.Google ScholarCross Ref
- [47] . 2017. Entity/Event-Level Sentiment Detection and Inference. Ph. D. Dissertation. University of Pittsburgh.Google Scholar
- [48] . 2014. Sentiment propagation via implicature constraints. In Proceedings of the 14th Conference of the European Chapter of the Association for Computational Linguistics. 377–385.Google ScholarCross Ref
- [49] . 2014. A study of supervised term weighting scheme for sentiment analysis. Expert Systems with Applications 41, 7 (2014), 3506–3513.Google ScholarDigital Library
- [50] . 2020. EEG synchronization analysis for seizure prediction: A study on data of noninvasive recordings. Processes 8, 7 (2020), 846.Google ScholarCross Ref
- [51] . 2014. SimSensei Kiosk: A virtual human interviewer for healthcare decision support. In Proceedings of the 2014 International Conference on Autonomous Agents and Multi-Agent Systems. 1061–1068.Google ScholarDigital Library
- [52] . 2016. Emotion analysis on Twitter: The hidden challenge. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC’16). 3953–3958.Google Scholar
- [53] . 2015. A review and meta-analysis of multimodal affect detection systems. ACM Computing Surveys (CSUR) 47, 3 (2015), 1–36.Google ScholarDigital Library
- [54] . 2010. Measuring the happiness of large-scale written expression: Songs, blogs, and presidents. Journal of Happiness Studies 11, 4 (2010), 441–456.Google ScholarCross Ref
- [55] . 2015. I-CARE: Intelligent context aware system for recognizing emotions from text. In 2015 10th International Conference on Intelligent Systems: Theories and Applications (SITA). IEEE, 1–5.Google ScholarCross Ref
- [56] . 2009. Automatic annotation of human actions in video. In 2009 IEEE 12th International Conference on Computer Vision. IEEE, 1491–1498.Google ScholarCross Ref
- [57] . 2011. Survey on speech emotion recognition: Features, classification schemes, and databases. Pattern Recognition 44, 3 (2011), 572–587.Google ScholarDigital Library
- [58] . 2014. Why we watch the news: A dataset for exploring sentiment in broadcast video news. In Proceedings of the 16th International Conference on Multimodal Interaction. 104–111.Google ScholarDigital Library
- [59] . 2006. SentiWordNet: A publicly available lexical resource for opinion mining.. In LREC, Vol. 6. Citeseer, 417–422.Google Scholar
- [60] . 2009. OpenEAR–introducing the munich open-source emotion and affect recognition toolkit. In 2009 3rd International Conference on Affective Computing and Intelligent Interaction and Workshops. IEEE, 1–6.Google ScholarCross Ref
- [61] . 2014. How are you doing?: Emotions and personality in Facebook. In 2nd Workshop on Emotions and Personality in Personalized Services (EMPIRE 2014); Workshop at the 22nd Conference on User Modelling, Adaptation and Personalization (UMAP 2014). 45–56.Google Scholar
- [62] . 1998. WordNet: An Electronic Lexical Database. MIT Press.Google Scholar
- [63] . 2016. Smart environment architecture for emotion detection and regulation. Journal of Biomedical Informatics 64 (2016), 55–73.Google ScholarDigital Library
- [64] . 2012. Instructing people for training gestural interactive systems. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 1737–1746.Google ScholarDigital Library
- [65] . 2005. Evaluating implicit measures to improve web search. ACM Trans. Inf. Syst. 23, 2 (
April 2005), 147–168.DOI: Google ScholarDigital Library - [66] . 1978. Facial action coding system: A technique for the measurement of facial movement. Palo Alto 3 (1978).Google Scholar
- [67] . 2014. A survey on multimodal sentiment analysis. International Journal of Engineering Research and Technology 3, 11 (2014), 1233–1238.Google Scholar
- [68] . 2011. Actom sequence models for efficient action detection. In CVPR 2011. IEEE, 3201–3208.Google ScholarDigital Library
- [69] . 2016. Comparing the level of code-switching in corpora. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC’16). 1850–1855.Google Scholar
- [70] . 2023. Multimodal sentiment analysis: A systematic review of history, datasets, multimodal fusion methods, applications, challenges and future directions. Information Fusion 91 (2023), 424–444.Google ScholarDigital Library
- [71] . 2013. A comparison of heartbeat detectors for the seismocardiogram. In Computing in Cardiology 2013. IEEE, 461–464.Google Scholar
- [72] . 2018. W2VLDA: Almost unsupervised system for aspect based sentiment analysis. Expert Systems with Applications 91 (2018), 127–137.Google ScholarCross Ref
- [73] . 2017. Audio set: An ontology and human-labeled dataset for audio events. In 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 776–780.Google ScholarDigital Library
- [74] . 2013. Twitter brand sentiment analysis: A hybrid system using n-gram analysis and dynamic artificial neural network. Expert Systems with Applications 40, 16 (2013), 6266–6282.Google ScholarDigital Library
- [75] . 2013. Geometric feature-based facial expression recognition in image sequences using multi-class AdaBoost and support vector machines. Sensors 13, 6 (2013), 7714–7734.Google ScholarCross Ref
- [76] . 2015. Recognition of facial expressions based on tracking and selection of discriminative geometric features. International Journal of Multimedia and Ubiquitous Engineering 10, 3 (2015), 35–44.Google ScholarCross Ref
- [77] . 2014. Vader: A parsimonious rule-based model for sentiment analysis of social media text. In Eighth International Conference on Weblogs and Social Media (ICWSM-14), Vol. 81. 82.Google Scholar
- [78] . 2009. The i3DPost multi-view and 3D human action/interaction database. In 2009 Conference for Visual Media Production. IEEE, 159–168.Google ScholarDigital Library
- [79] . 2010. Inducing genuine emotions in simulated speech-based human-machine interaction: The NIMITEK corpus. IEEE Transactions on Affective Computing 1, 2 (2010), 132–144.Google ScholarDigital Library
- [80] . 2007. Actions as space-time shapes. IEEE Transactions on Pattern Analysis and Machine Intelligence 29, 12 (2007), 2247–2253.Google ScholarDigital Library
- [81] . 2012. The human motion database: A cognitive and parametric sampling of human motion. Image and Vision Computing 30, 3 (2012), 251–261.Google ScholarDigital Library
- [82] . 2005. Fusing face and body display for bi-modal emotion recognition: Single frame analysis and multi-frame post integration. In International Conference on Affective Computing and Intelligent Interaction. Springer, 102–111.Google ScholarDigital Library
- [83] . 2012. Static hand gesture recognition using local Gabor filter. Procedia Engineering 41 (2012), 827–832.Google ScholarCross Ref
- [84] . 2014. Supervised sentiment analysis in Czech social media. Information Processing & Management 50, 5 (2014), 693–707.Google ScholarDigital Library
- [85] . 2020. Sentiment analysis using deep learning approaches: An overview. Science China Information Sciences 63, 1 (2020), 1–36.Google ScholarCross Ref
- [86] . 2016. Using YouTube comments for text-based emotion recognition. Procedia Computer Science 83 (2016), 292–299.Google ScholarCross Ref
- [87] . 2014. Speech emotion recognition using deep neural network and extreme learning machine. In Interspeech. ISCA, 223–227.Google Scholar
- [88] . 2015. The Indian spontaneous expression database for emotion recognition. IEEE Transactions on Affective Computing 8, 1 (2015), 131–142.Google ScholarDigital Library
- [89] . 2018. Generating a large dataset for neural question answering over the DBpedia knowledge base. In Workshop on Linked Data Management, WEBBR.Google Scholar
- [90] . 2014. Using hashtags as labels for supervised learning of emotions in Twitter messages. In ACM SIGKDD Workshop on Health Informatics, New York, USA.Google Scholar
- [91] . 2014. EMOTEX: Detecting emotions in Twitter messages. (2014).Google Scholar
- [92] . 2009. Automatically suggesting semantic structure for a generative Lexicon ontology. In Proceedings of Fifth International Conference on Generative Approaches to the Lexicon.Google Scholar
- [93] . 2014. Under pressure: Sensing stress of computer users. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 51–60.Google ScholarDigital Library
- [94] . 2017. CNN architectures for large-scale audio classification. In 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 131–135.Google ScholarDigital Library
- [95] . 2014. Multi-lingual support for lexicon-based sentiment analysis guided by semantics. Decision Support Systems 62 (2014), 43–53.Google ScholarCross Ref
- [96] . 2011. Learning a mixture of sparse distance metrics for classification and dimensionality reduction. In 2011 International Conference on Computer Vision. IEEE, 906–913.Google ScholarDigital Library
- [97] . 2007. GMM supervector based SVM with spectral features for speech emotion recognition. In 2007 IEEE International Conference on Acoustics, Speech and Signal Processing-ICASSP’07, Vol. 4. IEEE, IV–413.Google ScholarCross Ref
- [98] . 2017. Multimodal learning for topic sentiment analysis in microblogging. Neurocomputing 253 (2017), 144–153.Google ScholarDigital Library
- [99] . 2017. Densely connected convolutional networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 4700–4708.Google ScholarCross Ref
- [100] . 2015. Sign language recognition using 3D convolutional neural networks. In 2015 IEEE International Conference on Multimedia and Expo (ICME). IEEE, 1–6.Google ScholarCross Ref
- [101] . 2016. Are emoticons good enough to train emotion classifiers of Arabic tweets?. In 2016 7th International Conference on Computer Science and Information Technology (CSIT). IEEE, 1–6.Google ScholarCross Ref
- [102] . 2005. Gestures: Body language and nonverbal communication. Retrieved Oct. 2020 (2005).Google Scholar
- [103] . 2019. A survey of emotion recognition methods with emphasis on e-learning environments. Journal of Network and Computer Applications 147 (2019), 102423.Google ScholarDigital Library
- [104] . 2014. Fuzzy logic based sentiment analysis of product review documents. In 2014 First International Conference on Computational Systems and Communications (ICCSC). IEEE, 18–22.Google ScholarCross Ref
- [105] . 2018. Twitter sentiment analysis on 2013 curriculum using ensemble features and k-nearest neighbor. International Journal of Electrical and Computer Engineering (IJECE) 8, 6 (2018), 5409–14.Google ScholarCross Ref
- [106] . 2017. Extraction of emotions from multilingual text using intelligent text processing and computational linguistics. Journal of Computational Science 21 (2017), 316–326.Google ScholarCross Ref
- [107] . 2013. Real-time EEG-based happiness detection system. The Scientific World Journal 618649 (2013), 1–12.Google ScholarCross Ref
- [108] . 2011. Physiological signals based human emotion recognition: A review. In 2011 IEEE 7th International Colloquium on Signal Processing and its Applications. IEEE, 410–415.Google ScholarCross Ref
- [109] . 2012. Determination of fetal heart rate from abdominal signals: Evaluation of beat-to-beat accuracy in relation to the direct fetal electrocardiogram. Biomedical Engineering/BiomedizinisTechnik 57, 5 (2012), 383–394.Google Scholar
- [110] . 2016. MIMIC-III, a freely accessible critical care database. Scientific Data 3, 1 (2016), 1–9.Google ScholarCross Ref
- [111] . 1984. Test of the mention theory of irony. Journal of Experimental Psychology: General 113, 1 (1984), 112.Google ScholarCross Ref
- [112] . 2017. Automatic sarcasm detection: A survey. ACM Computing Surveys (CSUR) 50, 5 (2017), 1–22.Google ScholarDigital Library
- [113] . 2015. Harnessing context incongruity for sarcasm detection. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 2: Short Papers). 757–762.Google ScholarCross Ref
- [114] . 2000. Comprehensive database for facial expression analysis. In Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580). IEEE, 46–53.Google ScholarCross Ref
- [115] . 2018. Opinion mining using ensemble text hidden Markov models for text classification. Expert Systems with Applications 94 (2018), 218–227.Google ScholarDigital Library
- [116] . 2013. Emotion recognition system by gesture analysis using fuzzy sets. In International Conference on Swarm, Evolutionary, and Memetic Computing. Springer, 354–363.Google ScholarDigital Library
- [117] . 2018. Pixel-level reconstruction and classification for noisy handwritten Bangla characters. In 2018 16th International Conference on Frontiers in Handwriting Recognition (ICFHR). IEEE, 511–516.Google ScholarCross Ref
- [118] . 2018. A fuzzy computational model of emotion for cloud based sentiment analysis. Information Sciences 433 (2018), 448–463.Google ScholarCross Ref
- [119] . 2000. Analysis of a sleep-dependent neuronal feedback loop: The slow-wave microcontinuity of the EEG. IEEE Transactions on Biomedical Engineering 47, 9 (2000), 1185–1194.Google ScholarCross Ref
- [120] . 2008. Emotion recognition based on physiological changes in music listening. IEEE Transactions on Pattern Analysis and Machine Intelligence 30, 12 (2008), 2067–2083.Google ScholarDigital Library
- [121] . 2016. Trending autoregulatory indices during treatment for traumatic brain injury. Journal of Clinical Monitoring and Computing 30, 6 (2016), 821–831.Google ScholarCross Ref
- [122] . 2017. Comparison and analysis of facial expression database. In Proceedings of the Korea Information Processing Society Conference. Korea Information Processing Society, 686–687.Google Scholar
- [123] . 2006. Extracting opinions, opinion holders, and topics expressed in online news media text. In Proceedings of the Workshop on Sentiment and Subjectivity in Text. 1–8.Google ScholarDigital Library
- [124] . 2008. Canonical correlation analysis of video volume tensors for action categorization and detection. IEEE Transactions on Pattern Analysis and Machine Intelligence 31, 8 (2008), 1415–1428.Google Scholar
- [125] . 2014. Sentiment analysis of short informal texts. Journal of Artificial Intelligence Research 50 (2014), 723–762.Google ScholarCross Ref
- [126] . 2011. DEAP: A database for emotion analysis; using physiological signals. IEEE Transactions on Affective Computing 3, 1 (2011), 18–31.Google ScholarDigital Library
- [127] . 2020. Analysing affective behavior in the first ABAW 2020 competition. arXiv preprint arXiv:2001.11409 (2020).Google Scholar
- [128] . 2019. Deep affect prediction in-the-wild: Aff-Wild database and challenge, deep architectures, and beyond. International Journal of Computer Vision 127, 6-7 (2019), 907–929.Google ScholarDigital Library
- [129] . 2019. Expression, affect, action unit recognition: Aff-Wild2, multi-task learning and ArcFace. arXiv preprint arXiv:1910.04855 (2019).Google Scholar
- [130] . 2013. Ontology-based sentiment analysis of Twitter posts. Expert Systems with Applications 40, 10 (2013), 4065–4074.Google ScholarDigital Library
- [131] . 2011. Twitter sentiment analysis: The good the bad and the OMG!. In Fifth International AAAI Conference on Weblogs and Social Media. Citeseer.Google Scholar
- [132] . 2015. Active learning for sentiment analysis on data streams: Methodology and workflow implementation in the ClowdFlows platform. Information Processing & Management 51, 2 (2015), 187–203.Google ScholarCross Ref
- [133] . 2012. ImageNet classification with deep convolutional neural networks. In Advances in Neural Information Processing Systems. 1097–1105.Google ScholarDigital Library
- [134] . 2020. Systematic literature review of sentiment analysis on Twitter using soft computing techniques. Concurrency and Computation: Practice and Experience 32, 1 (2020), e5107.Google ScholarCross Ref
- [135] . 2016. Affective modelling of users in HCI using EEG. Procedia Computer Science 84 (2016), 107–114.
DOI: Proceeding of the Seventh International Conference onIntelligent Human Computer Interaction (IHCI 2015) .Google ScholarCross Ref - [136] . 2016. Bispectral analysis of EEG for emotion recognition. Procedia Computer Science 84 (2016), 31–35.Google ScholarCross Ref
- [137] . 2002. A theoretical study on six classifier fusion strategies. IEEE Transactions on Pattern Analysis and Machine Intelligence 24, 2 (2002), 281–286.Google ScholarDigital Library
- [138] . 2012. A real time system for dynamic hand gesture recognition with a depth sensor. In 2012 Proceedings of the 20th European Signal Processing Conference (EUSIPCO). IEEE, 1975–1979.Google Scholar
- [139] . 2003. Emotion recognition by speech signals. In Eighth European Conference on Speech Communication and Technology.Google ScholarCross Ref
- [140] . 2015. An approach to EEG based emotion recognition and classification using kernel density estimation. Procedia Computer Science 48 (2015), 574–581.Google ScholarCross Ref
- [141] . 2019. De-mixing sentiment from code-mixed text. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics: Student Research Workshop. 371–377.Google ScholarCross Ref
- [142] . 2014. Emotion recognition through speech signal for human-computer interaction. In 2014 Fifth International Symposium on Electronic System Design. IEEE, 217–218.Google ScholarCross Ref
- [143] . 2010. Presentation and Validation of the Radboud Faces Database. Cognition and Emotion 24, 8 (2010), 1377–1388.Google ScholarCross Ref
- [144] . 2014. Social analytics: Learning fuzzy product ontologies for aspect-oriented sentiment analysis. Decision Support Systems 65 (2014), 80–94.Google ScholarDigital Library
- [145] . 2005. Toward detecting emotions in spoken dialogs. IEEE Transactions on Speech and Audio Processing 13, 2 (2005), 293–303.Google ScholarCross Ref
- [146] . 2010. A text-driven rule-based system for emotion cause detection. In Proceedings of the NAACL HLT 2010 Workshop on Computational Approaches to Analysis and Generation of Emotion in Text. 45–53.Google ScholarDigital Library
- [147] . 2015. Multi-view learning for emotion detection in code-switching texts. In 2015 International Conference on Asian Language Processing (IALP). IEEE, 90–93.Google ScholarCross Ref
- [148] . 2014. Towards building a social emotion detection system for online news. Future Generation Computer Systems 37 (2014), 438–448.Google ScholarDigital Library
- [149] . 2013. Umigon: Sentiment analysis for tweets based on lexicons and heuristics. In Proceedings of the International Workshop on Semantic Evaluation, SemEval, Vol. 13, 1–4.Google Scholar
- [150] . 2017. Learning latent sentiment scopes for entity-level sentiment analysis. In AAAI. 3482–3489.Google ScholarCross Ref
- [151] . 2017. Reflections on sentiment/opinion analysis. In A Practical Guide to Sentiment Analysis. Springer, 41–59.Google ScholarCross Ref
- [152] . 2010. Object bank: A high-level image representation for scene classification & semantic feature sparsification. In Advances in Neural Information Processing Systems. 1378–1386.Google ScholarDigital Library
- [153] . 2014. News impact on stock price return via sentiment analysis. Knowledge-Based Systems 69 (2014), 14–23.Google ScholarDigital Library
- [154] . 2018. Investigation of multimodal features, classifiers and fusion methods for emotion recognition. arXiv preprint arXiv:1809.06225 (2018).Google Scholar
- [155] . 2014. Sentiment classification based on AS-LDA model. In ITQM. 511–516.Google ScholarCross Ref
- [156] . 2010. Using the Dempster–Shafer theory of evidence with a revised lattice structure for activity recognition. IEEE Transactions on Information Technology in Biomedicine 15, 1 (2010), 74–82.Google ScholarDigital Library
- [157] . 2018. Denoising distantly supervised open-domain question answering. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 1736–1745.Google ScholarCross Ref
- [158] . 2005. Speech emotion recognition based on HMM and SVM. In 2005 International Conference on Machine Learning and Cybernetics, Vol. 8. IEEE, 4898–4901.Google ScholarCross Ref
- [159] . 2009. Recognizing actions by shape-motion prototype trees. In 2009 IEEE 12th International Conference on Computer Vision. IEEE, 444–451.Google ScholarCross Ref
- [160] . 2010. Sentiment analysis and subjectivity. Handbook of Natural Language Processing 2, 2010 (2010), 627–666.Google Scholar
- [161] . 2012. Sentiment analysis and opinion mining. Synthesis Lectures on Human Language Technologies 5, 1 (2012), 1–167.Google ScholarCross Ref
- [162] . 2012. A survey of opinion mining and sentiment analysis. In Mining Text Data. Springer, 415–463.Google ScholarCross Ref
- [163] . 2013. Online and offline handwritten Chinese character recognition: Benchmarking on new databases. Pattern Recognition 46, 1 (2013), 155–162.Google ScholarDigital Library
- [164] . 2017. Multi-modal emotion recognition with temporal-band attention based on LSTM-RNN. In Pacific Rim Conference on Multimedia. Springer, 194–204.Google Scholar
- [165] . 2013. Learning discriminative representations from RGB-D video data. In Twenty-Third International Joint Conference on Artificial Intelligence.Google ScholarDigital Library
- [166] . 2003. Gesture classification using hidden Markov models and Viterbi path counting. In VIIth Digital Image Computing: Techniques and Applications. 273–282.Google Scholar
- [167] . 2019. PCGAN-CHAR: Progressively trained classifier generative adversarial networks for classification of noisy handwritten Bangla characters. In International Conference on Asian Digital Libraries. Springer, 3–15.Google ScholarDigital Library
- [168] . 2015. Automated rule selection for aspect extraction in opinion mining. In Twenty-Fourth International Joint Conference on Artificial Intelligence.Google Scholar
- [169] . 2015. A multi-label classification based approach for sentiment classification. Expert Systems with Applications 42, 3 (2015), 1083–1093.Google ScholarDigital Library
- [170] . 2018. The Ryerson audio-visual database of emotional speech and song (RAVDESS): A dynamic, multimodal set of facial and vocal expressions in North American English. PloS One 13, 5 (2018), e0196391.Google ScholarCross Ref
- [171] . 2008. The UJIpenchars database: A pen-based database of isolated handwritten characters. In LREC.Google Scholar
- [172] . 2009. Towards building a competitive opinion summarization system: Challenges and keys. In Proceedings of Human Language Technologies: The 2009 Annual Conference of the North American Chapter of the Association for Computational Linguistics, Companion Volume: Student Research Workshop and Doctoral Consortium. 72–77.Google ScholarDigital Library
- [173] . 2014. Real-time emotion recognition novel method for geometrical facial features extraction. In 2014 International Conference on Computer Vision Theory and Applications (VISAPP), Vol. 1. IEEE, 378–385.Google Scholar
- [174] . 2009. Emotional Aspects of the Online Information Retrieval Process. Ph. D. Dissertation. Rutgers University-Graduate School-New Brunswick.Google Scholar
- [175] . 2011. Theories, methods and current research on emotions in library and information science, information retrieval and human–computer interaction. Information Processing & Management 47, 4 (2011), 575–592.Google ScholarDigital Library
- [176] . 2010. The extended Cohn-Kanade dataset (CK+): A complete dataset for action unit and emotion-specified expression. In 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition-Workshops. IEEE, 94–101.Google ScholarCross Ref
- [177] . 2005. Emotion estimation and reasoning based on affective textual interaction. In International Conference on Affective Computing and Intelligent Interaction. Springer, 622–628.Google ScholarDigital Library
- [178] . 2014. Automatic recognition of attitudes in video blogs–prosodic and visual feature analysis. In Fifteenth Annual Conference of the International Speech Communication Association. 1826–1830.Google Scholar
- [179] . 2013. A template matching approach of one-shot-learning gesture recognition. Pattern Recognition Letters 34, 15 (2013), 1780–1788.Google ScholarDigital Library
- [180] . 2014. Building an Arabic sentiment lexicon using semi-supervised learning. Journal of King Saud University-Computer and Information Sciences 26, 4 (2014), 417–424.Google ScholarDigital Library
- [181] . 2014. Emotion recognition from geometric facial features using self-organizing map. Pattern Recognition 47, 3 (2014), 1282–1293.Google ScholarDigital Library
- [182] . 2016. Associating gesture expressivity with affective representations. Engineering Applications of Artificial Intelligence 51 (2016), 124–135.Google ScholarDigital Library
- [183] . 2010. Multimodal information fusion application to human emotion recognition from face and speech. Multimedia Tools and Applications 49, 2 (2010), 277–297.Google ScholarDigital Library
- [184] . 2009. GPU based optical character transcription for ancient inscription recognition. In 2009 15th International Conference on Virtual Systems and Multimedia. IEEE, 154–159.Google ScholarDigital Library
- [185] . 2014. Hand gesture recognition with leap motion and Kinect devices. In 2014 IEEE International Conference on Image Processing (ICIP). IEEE, 1565–1569.Google ScholarCross Ref
- [186] . 2016. Hand gesture recognition with jointly calibrated leap motion and depth sensor. Multimedia Tools and Applications 75, 22 (2016), 14991–15015.Google ScholarDigital Library
- [187] . 2017. Ontology-based aspect extraction for an improved sentiment analysis in summarization of product reviews. In Proceedings of the 8th International Conference on Computer Modeling and Simulation. 100–104.Google ScholarDigital Library
- [188] . 2009. Actions in context. In 2009 IEEE Conference on Computer Vision and Pattern Recognition. IEEE, 2929–2936.Google ScholarCross Ref
- [189] . 2006. The eNTERFACE’05 audio-visual emotion database. In 22nd International Conference on Data Engineering Workshops (ICDEW’06). IEEE, 8–8.Google ScholarDigital Library
- [190] . 2015. SentiCap: Generating image descriptions with sentiments. arXiv preprint arXiv:1510.01431 (2015).Google Scholar
- [191] . 2017. Towards the automated localisation of targets in rapid image-sifting by collaborative brain-computer interfaces. PLoS One 12, 5 (2017), e0178498.Google ScholarCross Ref
- [192] . 2013. DISFA: A spontaneous facial action intensity database. IEEE Transactions on Affective Computing 4, 2 (2013), 151–160.Google ScholarDigital Library
- [193] . 2014. Who cares about sarcastic tweets? Investigating the impact of sarcasm on sentiment analysis. In LREC 2014 Proceedings. ELRA, 4238–4243.Google Scholar
- [194] . 2014. Predicting ad liking and purchase intent: Large-scale analysis of facial responses to ads. IEEE Transactions on Affective Computing 6, 3 (2014), 223–235.Google ScholarDigital Library
- [195] . 2007. Topic sentiment mixture: Modeling facets and opinions in Weblogs. In Proceedings of the 16th International Conference on World Wide Web. 171–180.Google ScholarDigital Library
- [196] . 2011. Movement segmentation using a primitive library. In 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems. IEEE, 3407–3412.Google ScholarCross Ref
- [197] . 2018. Head-mounted gesture controlled interface for human-computer interaction. Multimedia Tools and Applications 77, 1 (2018), 27–53.Google ScholarDigital Library
- [198] . 2011. 3D human face reconstruction using principal components spaces. In Proceedings of WTD SIBGRAPI Conference on Graphics, Patterns and Images, Vol. 6.Google Scholar
- [199] . 2009. Activity recognition using the velocity histories of tracked keypoints. In 2009 IEEE 12th International Conference on Computer Vision. IEEE, 104–111.Google ScholarCross Ref
- [200] . 1996. Interface system based on hand gestures and verbal expressions for 3-D shape generation, Terebijon Gakkaishi. Journal of the Institute of Television Engineers of Japan. v50 i10 (1996), 1482–1488.Google Scholar
- [201] . 2013. Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781 (2013).Google Scholar
- [202] . 2015. Exploiting silhouette descriptors and synthetic data for hand gesture recognition. In Smart Tools and Apps for Graphics-Eurographics Italian Chapter Conference. Eurographics, 15–23.Google Scholar
- [203] . 2012. # Emotional tweets. In * SEM 2012: The First Joint Conference on Lexical and Computational Semantics–Volume 1: Proceedings of the Main Conference and the Shared Task, and Volume 2: Proceedings of the Sixth International Workshop on Semantic Evaluation (SemEval 2012). 246–255.Google Scholar
- [204] . 2018. Obtaining reliable human ratings of valence, arousal, and dominance for 20,000 English words. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 174–184.Google ScholarCross Ref
- [205] . 2018. SemEval-2018 Task 1: Affect in tweets. In Proceedings of the 12th International Workshop on Semantic Evaluation. 1–17.Google ScholarCross Ref
- [206] . 2010. Emotions evoked by common words and phrases: Using Mechanical Turk to create an emotion lexicon. In Proceedings of the NAACL HLT 2010 Workshop on Computational Approaches to Analysis and Generation of Emotion in Text. 26–34.Google ScholarDigital Library
- [207] . 2016. Sentiment analysis: Detecting valence, emotions, and other affectual states from text. In Emotion Measurement. Elsevier, 201–237.Google ScholarCross Ref
- [208] . 2017. Emotion intensities in tweets. arXiv preprint arXiv:1708.03696 (2017).Google Scholar
- [209] . 2013. Crowdsourcing a word–emotion association lexicon. Computational Intelligence 29, 3 (2013), 436–465.Google ScholarCross Ref
- [210] . 2017. AffectNet: A database for facial expression, valence, and arousal computing in the wild. IEEE Transactions on Affective Computing 10, 1 (2017), 18–31.Google ScholarDigital Library
- [211] . 2014. Crowd explicit sentiment analysis. Knowledge-Based Systems 69 (2014), 134–139.Google ScholarDigital Library
- [212] . 2014. Ranked WordNet graph for sentiment polarity classification in Twitter. Computer Speech & Language 28, 1 (2014), 93–107.Google ScholarDigital Library
- [213] . 2008. The physionet/computers in cardiology challenge 2008: T-wave alternans. In 2008 Computers in Cardiology. IEEE, 505–508.Google ScholarCross Ref
- [214] . 2013. Document-level sentiment classification: An empirical comparison between SVM and ANN. Expert Systems with Applications 40, 2 (2013), 621–633.Google ScholarDigital Library
- [215] . 2011. Towards multimodal sentiment analysis: Harvesting opinions from the web. In Proceedings of the 13th International Conference on Multimodal Interfaces. 169–176.Google ScholarDigital Library
- [216] . 2003. A neural network-based approach to recognizing 3D arm movement. Biomedical Engineering Application. Basis and Communication 15, 1 (2003), 17–26.Google ScholarCross Ref
- [217] . 2015. ASTD: Arabic sentiment tweets dataset. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. 2515–2519.Google ScholarCross Ref
- [218] . 2008. Method for automated image indexing and retrieval.
US Patent 7,324,711 .Google Scholar - [219] . 2004. Emotion recognition from physiological signals using wireless sensors for presence technologies. Cognition, Technology & Work 6, 1 (2004), 4–14.Google ScholarDigital Library
- [220] . 2008. Automated de-identification of free-text medical records. BMC Medical Informatics and Decision Making 8, 1 (2008), 32.Google ScholarCross Ref
- [221] . 2006. Emotion recognition in spontaneous speech using GMMs. In Ninth International Conference on Spoken Language Processing. 809–812.Google ScholarCross Ref
- [222] . 2017. Exploiting evidential theory in the fusion of textual, audio, and visual modalities for affective music video retrieval. In 2017 3rd International Conference on Pattern Recognition and Image Analysis (IPRIA). IEEE, 222–228.Google ScholarCross Ref
- [223] . 2009. Compositionality principle in recognition of fine-grained emotions from text. In Third International AAAI Conference on Weblogs and Social Media.Google ScholarCross Ref
- [224] . 2010. @AM: Textual attitude analysis model. In Proceedings of the NAACL HLT 2010 Workshop on Computational Approaches to Analysis and Generation of Emotion in Text. 80–88.Google Scholar
- [225] Payal Bajaj, Daniel Campos, Nick Craswell, Li Deng, Jianfeng Gao, Xiaodong Liu, Rangan Majumder, Andrew McNamara, Bhaskar Mitra, Tri Nguyen, Mir Rosenberg, Xia Song, Alina Stoica, Saurabh Tiwary, Tong Wang. 2018. MS MARCO: A human generated machine reading comprehension dataset. arXiv preprint arXiv:1611.09268v3 (2018).Google Scholar
- [226] . 2014. Robust canonical correlation analysis: Audio-visual fusion for learning continuous interest. In 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 1522–1526.Google ScholarCross Ref
- [227] . 2016. Comparative evaluation of state-of-the-art algorithms for SSVEP-based BCIs. arXiv preprint arXiv:1602.00904 (2016).Google Scholar
- [228] . 2014. Sentiment analysis in Facebook and its application to e-learning. Computers in Human Behavior 31 (2014), 527–541.Google ScholarDigital Library
- [229] . 2013. Improved part-of-speech tagging for online conversational text with word clusters. In Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 380–390.Google Scholar
- [230] . 2008. Toward emotion indexing of multimedia excerpts. In 2008 International Workshop on Content-Based Multimedia Indexing. IEEE, 425–432.Google ScholarCross Ref
- [231] . 2012. Speech emotion recognition using support vector machine. International Journal of Smart Home 6, 2 (2012), 101–108.Google Scholar
- [232] . 2017. Emotion recognition from speech with Gaussian mixture models & via boosted GMM. International Journal of Research In Science & Engineering 3 (2017), 47–53.Google Scholar
- [233] . 2011. Measuring emotions in students’ learning and performance: The achievement emotions questionnaire (AEQ). Contemporary Educational Psychology 36, 1 (2011), 36–48.Google ScholarCross Ref
- [234] . 2016. Multimodal emotion recognition with evolutionary computation for human-robot interaction. Expert Systems with Applications 66 (2016), 42–61.Google ScholarDigital Library
- [235] . 2013. Utterance-level multimodal sentiment analysis. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 973–982.Google Scholar
- [236] . 2013. Recognizing emotion presence in natural language sentences. In International Conference on Engineering Applications of Neural Networks. Springer, 30–39.Google ScholarCross Ref
- [237] . 2018. Deep contextualized word representations. arXiv preprint arXiv:1802.05365 (2018).Google Scholar
- [238] . 2014. Sign language recognition using convolutional neural networks. In European Conference on Computer Vision. Springer, 572–578.Google Scholar
- [239] . 2017. A review of affective computing: From unimodal analysis to multimodal fusion. Information Fusion 37 (2017), 98–125.Google ScholarDigital Library
- [240] . 2016. Fusing audio, visual and textual clues for sentiment analysis from multimodal content. Neurocomputing 174 (2016), 50–59.Google ScholarDigital Library
- [241] . 2015. Towards an intelligent framework for multimodal affective data analysis. Neural Networks 63 (2015), 104–116.Google ScholarDigital Library
- [242] . 2016. Convolutional MKL based multimodal emotion recognition and sentiment analysis. In 2016 IEEE 16th International Conference on Data Mining (ICDM). IEEE, 439–448.Google ScholarCross Ref
- [243] . 2013. Enhanced SenticNet with affective labels for concept-based opinion mining. IEEE Intelligent Systems 28, 2 (2013), 31–38.Google ScholarDigital Library
- [244] . 2020. Beneath the Tip of the Iceberg: Current challenges and new directions in sentiment analysis research. arXiv preprint arXiv:2005.00357 (2020).Google Scholar
- [245] . 2017. Ensemble application of convolutional neural networks and multiple kernel learning for multimodal sentiment analysis. Neurocomputing 261 (2017), 217–230.Google ScholarCross Ref
- [246] . 2016. Modelling valence and arousal in Facebook posts. In Proceedings of the 7th Workshop on Computational Approaches to Subjectivity, Sentiment and Social Media Analysis. 9–15.Google ScholarCross Ref
- [247] . 2014. Automatically annotating a five-billion-word corpus of Japanese blogs for sentiment and affect analysis. Computer Speech & Language 28, 1 (2014), 38–55.Google ScholarDigital Library
- [248] . 2017. A novel deep-learning based framework for multi-subject emotion recognition. In 2017 4th International Conference on Information, Cybernetics and Computational Social Systems (ICCSS). IEEE, 181–185.Google ScholarCross Ref
- [249] . 2011. Opinion word expansion and target extraction through double propagation. Computational Linguistics 37, 1 (2011), 9–27.
DOI: . arXiv:https://doi.org/10.1162/coli_a_00034Google ScholarDigital Library - [250] . 2018. DeepInf: Social influence prediction with deep learning. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 2110–2119.Google ScholarDigital Library
- [251] . 2015. Latent discriminative models for social emotion detection with emotional dependency. ACM Transactions on Information Systems (TOIS) 34, 1 (2015), 1–19.Google ScholarDigital Library
- [252] . 2016. Squad: 100,000+ questions for machine comprehension of text. arXiv preprint arXiv:1606.05250 (2016).Google Scholar
- [253] . 2015. Analysis of students emotion for Twitter data using Naïve Bayes and non linear support vector machine approachs. International Journal on Recent and Innovation Trends in Computing and Communication. ISSN (2015), 2321–8169.Google Scholar
- [254] . 2006. An empirical study of machine learning techniques for affect recognition in human–robot interaction. Pattern Analysis and Applications 9, 1 (2006), 58–69.Google ScholarDigital Library
- [255] . 2011. Recognition of emotions from video using neural network models. Expert Systems with Applications 38, 10 (2011), 13181–13185.Google ScholarDigital Library
- [256] . 2016. Social emotion classification of short text via topic-level maximum entropy model. Information & Management 53, 8 (2016), 978–986.Google ScholarDigital Library
- [257] . 2019. CoQA: A conversational question answering challenge. Transactions of the Association for Computational Linguistics 7 (2019), 249–266.Google ScholarCross Ref
- [258] . 2011. Robust hand gesture recognition with Kinect sensor. In Proceedings of the 19th ACM International Conference on Multimedia. 759–760.Google ScholarDigital Library
- [259] . 1995. Robust text-independent speaker identification using Gaussian mixture speaker models. IEEE Transactions on Speech and Audio Processing 3, 1 (1995), 72–83.Google ScholarCross Ref
- [260] . 2014. PoliTwi: Early detection of emerging political topics on Twitter and the impact on concept-level sentiment analysis. Knowledge-Based Systems 69 (2014), 24–33.Google ScholarDigital Library
- [261] . 2008. Action MACH a spatio-temporal maximum average correlation height filter for action recognition. In 2008 IEEE Conference on Computer Vision and Pattern Recognition. IEEE, 1–8.Google ScholarCross Ref
- [262] . 2014. Auto-encoder based bagging architecture for sentiment analysis. Journal of Visual Languages & Computing 25, 6 (2014), 840–849.Google ScholarDigital Library
- [263] . 2013. Multimodal sentiment analysis of Spanish online videos. IEEE Intelligent Systems 28, 3 (2013), 38–45.Google ScholarDigital Library
- [264] . 2020. A survey of classification techniques in speech emotion recognition. Mathematical Methods in Interdisciplinary Sciences (2020), 33–48.Google ScholarCross Ref
- [265] . 2013. ChAirGest: A challenge for multimodal mid-air gesture recognition for close HCI. In Proceedings of the 15th ACM on International Conference on Multimodal Interaction. 483–488.Google ScholarDigital Library
- [266] . 2013. Sentiment analysis using neuro-fuzzy and hidden Markov models of text. In 2013 Proceedings of IEEE Southeastcon. IEEE, 1–6.Google ScholarCross Ref
- [267] . 2014. Frame-based facial expression recognition using geometrical features. Advances in Human-Computer Interaction (2014).Google ScholarDigital Library
- [268] . 2018. Emotion detection from text and speech: A survey. Social Network Analysis and Mining 8, 1 (2018), 28.Google ScholarCross Ref
- [269] . 2005. A systems approach to appraisal mechanisms in emotion. Neural Networks 18, 4 (2005), 317–352.Google ScholarDigital Library
- [270] . 2014. Automatic analysis of facial affect: A survey of registration, representation, and recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence 37, 6 (2014), 1113–1133.Google ScholarDigital Library
- [271] . 2014. Feature analysis for computational personality recognition using YouTube personality data set. In Proceedings of the 2014 ACM Multi Media on Workshop on Computational Personality Recognition. 11–14.Google ScholarDigital Library
- [272] . 2009. Towards emotion recognition from electroencephalographic signals. In 2009 3rd International Conference on Affective Computing and Intelligent Interaction and Workshops. IEEE, 1–6.Google ScholarCross Ref
- [273] . 2004. BCI2000: A general-purpose brain-computer interface (BCI) system. IEEE Transactions on Biomedical Engineering 51, 6 (2004), 1034–1043.Google ScholarCross Ref
- [274] . 2002. Frustrating the user on purpose: A step toward building an affective computer. Interacting with Computers 14, 2 (2002), 93–118.Google ScholarCross Ref
- [275] . 2015. Survey on aspect-level sentiment analysis. IEEE Transactions on Knowledge and Data Engineering 28, 3 (2015), 813–830.Google ScholarDigital Library
- [276] . 2004. Speech emotion recognition combining acoustic features and linguistic information in a hybrid support vector machine-belief network architecture. In 2004 IEEE International Conference on Acoustics, Speech, and Signal Processing, Vol. 1. IEEE, I–577.Google ScholarCross Ref
- [277] . 2005. Multimodal approaches for emotion recognition: A survey. In Internet Imaging VI, Vol. 5670. International Society for Optics and Photonics, 56–67.Google ScholarCross Ref
- [278] . 2020. SeetaFaceEngine. https://github.com/seetaface/SeetaFaceEngine
Accessed: 2024-03-08 .Google Scholar - [279] . 2007. Beyond facial expressions: Learning human emotion from body gestures. In BMVC. 1–10.Google ScholarCross Ref
- [280] . 2019. A dataset of continuous affect annotations and physiological signals for emotion analysis. Scientific Data 6, 1 (2019), 1–13.Google ScholarCross Ref
- [281] . 2018. Multimodal sentiment analysis using deep learning. In 2018 17th IEEE International Conference on Machine Learning and Applications (ICMLA). IEEE, 1475–1478.Google ScholarCross Ref
- [282] . 2014. A fuzzy logic based on sentiment classification. International Journal of Data Mining & Knowledge Management Process 4, 4 (2014), 27.Google ScholarCross Ref
- [283] . 2007. A gesture recognition system using stereo vision and arm model fitting. In International Congress Series, Vol. 1301. Elsevier, 89–92.Google ScholarCross Ref
- [284] . 2017. Development of emotion recognition interface using complex EEG/ECG bio-signal for interactive contents. Multimedia Tools and Applications 76, 9 (2017), 11449–11470.Google ScholarDigital Library
- [285] . 2015. EmotionFinder: Detecting emotion from blogs and textual documents. In International Conference on Computing, Communication & Automation. IEEE, 52–57.Google ScholarCross Ref
- [286] . 2009. Application of Machine Learning to Epileptic Seizure Onset Detection and Treatment. Ph. D. Dissertation. Massachusetts Institute of Technology.Google Scholar
- [287] . 2018. A review of emotion recognition using physiological signals. Sensors 18, 7 (2018), 2074.Google ScholarCross Ref
- [288] . 2017. Lifelong learning CRF for supervised aspect extraction. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), Association for Computational Linguistics. 148–154.
DOI: Google ScholarCross Ref - [289] . 2015. Exploiting multimodal affect and semantics to identify politically persuasive web videos. In Proceedings of the 2015 ACM on International Conference on Multimodal Interaction. 203–210.Google ScholarDigital Library
- [290] . 2010. HumanEva: Synchronized video and motion capture dataset and baseline algorithm for evaluation of articulated human motion. International Journal of Computer Vision 87, 1-2 (2010), 4.Google ScholarDigital Library
- [291] . 2014. A survey for multimodal sentiment analysis methods. International Journal Computer Technology & Applications 5, 4 (2014), 1470–1476.Google Scholar
- [292] . 2014. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014).Google Scholar
- [293] . 2020. Sentiment analysis: A review and comparative analysis over social media. Journal of Ambient Intelligence and Humanized Computing 11, 1 (2020), 97–117.Google ScholarCross Ref
- [294] . 2019. Indian semi-acted facial expression (iSAFE) dataset for human emotions recognition. In International Symposium on Signal Processing and Intelligent Recognition Systems. Springer, 150–162.Google Scholar
- [295] . 2011. The Belfast induced natural emotion database. IEEE Transactions on Affective Computing 3, 1 (2011), 32–41.Google ScholarDigital Library
- [296] . 2017. A survey of multimodal sentiment analysis. Image and Vision Computing 65 (2017), 3–14.Google ScholarCross Ref
- [297] . 2011. Multimodal emotion recognition in response to videos. IEEE Transactions on Affective Computing 3, 2 (2011), 211–223.Google ScholarDigital Library
- [298] . 2008. A robust multimodal approach for emotion recognition. Neurocomputing 71, 10-12 (2008), 1913–1920.Google ScholarDigital Library
- [299] . 2011. Tracking body and hands for gesture recognition: NATOPS aircraft handling signals database. In Face and Gesture 2011. IEEE, 500–506.Google ScholarCross Ref
- [300] . 2017. SPARQL as a foreign language. arXiv preprint arXiv:1708.07624 (2017).Google Scholar
- [301] . 2013. ConceptNet 5: A large semantic network for relational knowledge. In The People’s Web Meets NLP. Springer, 161–176.Google ScholarCross Ref
- [302] . 2008. Learning to identify emotions in text. In Proceedings of the 2008 ACM Symposium on Applied Computing. 1556–1560.Google ScholarDigital Library
- [303] . 2004. WordNet affect: An affective extension of WordNet. In LREC, Vol. 4. Citeseer, 40.Google Scholar
- [304] . 2017. iSelf: Towards cold-start emotion labeling using transfer learning with smartphones. ACM Transactions on Sensor Networks (TOSN) 13, 4 (2017), 1–22.Google ScholarDigital Library
- [305] . 2018. Improving machine reading comprehension with general reading strategies. arXiv preprint arXiv:1810.13441 (2018).Google Scholar
- [306] . 2018. CliCR: A dataset of clinical case reports for machine reading comprehension. arXiv preprint arXiv:1803.09720 (2018).Google Scholar
- [307] . 2012. A methodology for validating artifact removal techniques for physiological signals. IEEE Transactions on Information Technology in Biomedicine 16, 5 (2012), 918–926.Google ScholarDigital Library
- [308] . 2016. Rethinking the inception architecture for computer vision. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2818–2826.Google ScholarCross Ref
- [309] . 2005. Linear dimensionality reduction using relevance weighted LDA. Pattern Recognition 38, 4 (2005), 485–493.Google ScholarDigital Library
- [310] . 2017. Dyadic memory networks for aspect-based sentiment analysis. In Proceedings of the 2017 ACM on Conference on Information and Knowledge Management. 107–116.Google ScholarDigital Library
- [311] . 2010. Convolutional learning of spatio-temporal features. In European Conference on Computer Vision. Springer, 140–153.Google ScholarDigital Library
- [312] . 2009. The TUM kitchen data set of everyday manipulation activities for motion tracking and action recognition. In 2009 IEEE 12th International Conference on Computer Vision Workshops, ICCV Workshops. IEEE, 1089–1096.Google ScholarCross Ref
- [313] . 2015. Ontology-based sentiment analysis process for social media content. In INNS Conference on Big Data. 199–207.Google ScholarCross Ref
- [314] . 2010. Sentiment strength detection in short informal text. Journal of the American Society for Information Science and Technology 61, 12 (2010), 2544–2558.Google ScholarCross Ref
- [315] . 2017. The HASYv2 dataset. arXiv preprint arXiv:1701.08380 (2017).Google Scholar
- [316] . 2008. Human activity recognition with metric learning. In European Conference on Computer Vision. Springer, 548–561.Google ScholarDigital Library
- [317] . 2016. Emotion analysis from text: A survey. Center for Indian Language Technology Surveys 11, 8 (2016), 66–69.Google Scholar
- [318] . 2016. NewsQA: A machine comprehension dataset. arXiv preprint arXiv:1611.09830 (2016).Google Scholar
- [319] . 2010. ICWSM-A great catchy name: Semi-supervised recognition of sarcastic sentences in online product reviews. In ICWSM. Washington, DC, 162–169.Google ScholarCross Ref
- [320] . 2015. A rule-based approach to implicit emotion detection in text. In International Conference on Applications of Natural Language to Information Systems. Springer, 197–203.Google ScholarCross Ref
- [321] . 2010. Induced disgust, happiness and surprise: An addition to the MMI facial expression database. In Proc. 3rd Intern. Workshop on EMOTION (Satellite of LREC): Corpora for Research on Emotion and Affect. 65.Google Scholar
- [322] . 2011. Human–computer interaction for smart environment applications using fuzzy hand posture and gesture models. IEEE Transactions on Instrumentation and Measurement 60, 5 (2011), 1505–1514.Google ScholarCross Ref
- [323] . 2006. The function space of an activity. In 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’06), Vol. 1. IEEE, 959–968.Google ScholarDigital Library
- [324] . 2014. Multimodal fusion framework: A multiresolution approach for emotion classification and recognition from physiological signals. NeuroImage 102 (2014), 162–172.Google ScholarCross Ref
- [325] . 2006. Emotional speech recognition: Resources, features, and methods. Speech Communication 48, 9 (2006), 1162–1181.Google ScholarCross Ref
- [326] . 2019. Assessment of multi-ion channel block in a phase i randomized study design: Results of the Ci PA phase i ECG biomarker validation study. Clinical Pharmacology & Therapeutics 105, 4 (2019), 943–953.Google ScholarCross Ref
- [327] . 2017. Universal, unsupervised (rule-based), uncovered sentiment analysis. Knowledge-Based Systems 118 (2017), 45–55.Google ScholarCross Ref
- [328] . 2009. CASIA-OLHWDB1: A database of online handwritten Chinese characters. In 2009 10th International Conference on Document Analysis and Recognition. IEEE, 1206–1210.Google ScholarDigital Library
- [329] . 2015. POS-RS: A random subspace method for sentiment classification based on part-of-speech analysis. Information Processing & Management 51, 4 (2015), 458–479.Google ScholarDigital Library
- [330] . 2017. Select-additive learning: Improving generalization in multimodal sentiment analysis. In 2017 IEEE International Conference on Multimedia and Expo (ICME). IEEE, 949–954.Google ScholarCross Ref
- [331] . 2018. SHINE: Signed heterogeneous information network embedding for sentiment link prediction. In Proceedings of the Eleventh ACM International Conference on Web Search and Data Mining. 592–600.Google ScholarDigital Library
- [332] . 2013. A k-nearest-neighbor classifier with heart rate variability feature-based transformation algorithm for driving stress recognition. Neurocomputing 116 (2013), 136–143.Google ScholarCross Ref
- [333] . 2016. Attention-based LSTM for aspect-level sentiment classification. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing. 606–615.Google ScholarCross Ref
- [334] . 2013. Emotion feature selection from physiological signals using tabu search. In 2013 25th Chinese Control and Decision Conference (CCDC). IEEE, 3148–3150.Google ScholarCross Ref
- [335] . 2022. A survey on sentiment analysis methods, applications, and challenges. Artificial Intelligence Review (2022), 1–50.Google Scholar
- [336] . 2013. Norms of valence, arousal, and dominance for 13,915 English lemmas. Behavior Research Methods 45, 4 (2013), 1191–1207.Google ScholarCross Ref
- [337] . 2006. Free viewpoint action recognition using motion history volumes. Computer Vision and Image Understanding 104, 2-3 (2006), 249–257.Google ScholarDigital Library
- [338] . 2014. Emotion recognition based on multi-variant correlation of physiological signals. IEEE Transactions on Affective Computing 5, 2 (2014), 126–140.Google ScholarCross Ref
- [339] . 2015. Opinion holder and target extraction based on the induction of verbal categories. In Proceedings of the Nineteenth Conference on Computational Natural Language Learning. 215–225.Google ScholarCross Ref
- [340] . 2015. A Text Mining Application of Emotion Classifications of Twitter’s Users Using Naïve Bayes Method International Conference on Wireless & Telematics. In 1st International Conference on Wireless and Telematics (ICWT), IEEE, 1–6.Google Scholar
- [341] . 2017. GermEval 2017: Shared task on aspect-based sentiment in social media customer feedback. Proceedings of the GermEval (2017), 1–12.Google Scholar
- [342] . 2013. YouTube movie reviews: Sentiment analysis in an audio-visual context. IEEE Intelligent Systems 28, 3 (2013), 46–53.Google ScholarDigital Library
- [343] . 2014. Using relation selection to improve value propagation in a ConceptNet-based sentiment dictionary. Knowledge-Based Systems 69 (2014), 100–107.Google ScholarDigital Library
- [344] . 2016. Incorporating sentiment into tag-based user profiles and resource profiles for personalized search in folksonomy. Information Processing & Management 52, 1 (2016), 61–72.Google ScholarDigital Library
- [345] . 1992. A. Krzyzak, and C. Y. Suen. Methods of combining multiple classifiers and their applications to handwriting recognition. IEEE Transactions on Systems, Man, and Cybernetics 22, 3 (1992), 418–435.Google ScholarCross Ref
- [346] . 2015. Prediction of user ratings of oral presentations using label relations. In Proceedings of the 1st International Workshop on Affect & Sentiment in Multimedia. 33–38.Google ScholarDigital Library
- [347] . 2014. A bilingual approach for conducting Chinese and English social media sentiment analysis. Computer Networks 75 (2014), 491–503.Google ScholarDigital Library
- [348] . 2019. XLNet: Generalized autoregressive pretraining for language understanding. In Advances in Neural Information Processing Systems. 5753–5763.Google Scholar
- [349] . 2018. HotpotQA: A dataset for diverse, explainable multi-hop question answering. arXiv preprint arXiv:1809.09600 (2018).Google Scholar
- [350] . 2003. Towards answering opinion questions: Separating facts from opinions and identifying the polarity of opinion sentences. In Proceedings of the 2003 Conference on Empirical Methods in Natural Language Processing. 129–136.Google ScholarDigital Library
- [351] . 2015. Micro-opinion sentiment intensity analysis and summarization in online videos. In Proceedings of the 2015 ACM on International Conference on Multimodal Interaction. 587–591.Google ScholarDigital Library
- [352] . 2017. Tensor fusion network for multimodal sentiment analysis. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, Copenhagen, Denmark, 1103–1114.
DOI: Google ScholarCross Ref - [353] . 2018. Memory fusion network for multi-view sequential learning. arXiv preprint arXiv:1802.00927 (2018).Google Scholar
- [354] . 2018. Multi-attention recurrent network for human communication comprehension. Proceedings of the AAAI Conference on Artificial Intelligence, 32 1 (2018).Google ScholarCross Ref
- [355] . 2016. MOSI: Multimodal corpus of sentiment intensity and subjectivity analysis in online opinion videos. arXiv preprint arXiv:1606.06259 (2016).Google Scholar
- [356] . 2016. Multimodal sentiment intensity analysis in videos: Facial gestures and verbal messages. IEEE Intelligent Systems 31, 6 (2016), 82–88.Google ScholarDigital Library
- [357] . 2018. Multimodal language analysis in the wild: CMU-MOSEI dataset and interpretable dynamic fusion graph. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2236–2246.Google ScholarCross Ref
- [358] . 2017. Aff-Wild: Valence and arousal ’in-the-wild’ challenge. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. 34–41.Google Scholar
- [359] . 2008. A survey of affect recognition methods: Audio, visual, and spontaneous expressions. IEEE Transactions on Pattern Analysis and Machine Intelligence 31, 1 (2008), 39–58.Google ScholarDigital Library
- [360] . 2016. PNN for EEG-based emotion recognition. In 2016 IEEE International Conference on Systems, Man, and Cybernetics (SMC). IEEE, 002319–002323.Google ScholarDigital Library
- [361] . 2010. A k-nearest neighbor based multi-instance multi-label learning algorithm. In 2010 22nd IEEE International Conference on Tools with Artificial Intelligence, Vol. 2. IEEE, 207–212.Google ScholarDigital Library
- [362] . 2018. Sentiment analysis of Chinese micro-blog text based on extended sentiment dictionary. Future Generation Computer Systems 81 (2018), 395–403.Google ScholarDigital Library
- [363] . 2018. EgoGesture: A new dataset and benchmark for egocentric hand gesture recognition. IEEE Transactions on Multimedia 20, 5 (2018), 1038–1050.Google ScholarCross Ref
- [364] . 2007. Dynamic texture recognition using local binary patterns with an application to facial expressions. IEEE Transactions on Pattern Analysis and Machine Intelligence 29, 6 (2007), 915–928.Google ScholarDigital Library
- [365] . 2016. Emotion distribution learning from texts. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing. 638–647.Google ScholarCross Ref
- [366] . 1996. FORMS: A flexible object recognition and modelling system. International Journal of Computer Vision 20, 3 (1996), 187–212.Google ScholarCross Ref
- [367] . 2009. Hilbert-Huang transform based physiological signals analysis for emotion recognition. In 2009 IEEE International Symposium on Signal Processing and Information Technology (ISSPIT). IEEE, 334–339.Google ScholarCross Ref
- [368] . 2019. Electroencephalograms during mental arithmetic task performance. Data 4, 1 (2019), 14.Google ScholarCross Ref
Index Terms
- A Survey of Cutting-edge Multimodal Sentiment Analysis
Recommendations
Multimodal Sentiment Analysis: A Survey of Methods, Trends, and Challenges
Sentiment analysis has come long way since it was introduced as a natural language processing task nearly 20 years ago. Sentiment analysis aims to extract the underlying attitudes and opinions toward an entity. It has become a powerful tool used by ...
Subjectivity and sentiment analysis: An overview of the current state of the area and envisaged developments
In this introduction, we present an overview of the current state of research in the Natural Language Processing tasks of subjectivity and sentiment analysis, as well as their application domains and closely-related research field of emotion detection. ...
Multimodal Video Sentiment Analysis Using Deep Learning Approaches, a Survey
Highlights- Multimodal sentiment analysis survey.
- New categorization of 35 models according ...
ABSTRACTDeep learning has emerged as a powerful machine learning technique to employ in multimodal sentiment analysis tasks. In the recent years, many deep learning models and various algorithms have been proposed in the field of multimodal ...
Comments