Konstantin Kobs
Abstract
In recent years, streaming platforms for video games have seen increasingly large interest, as so-called esports have developed into a lucrative branch of business. Like for other sports, watching esports has become a new kind of entertainment medium, which is possible due to platforms that allow gamers to live stream their gameplay, the most popular platform being Twitch.tv. On these platforms, users can comment on streams in real time and thereby express their opinion about the events in the stream. Due to the popularity of Twitch.tv, this can be a valuable source of feedback for streamers aiming to improve their reception in a gaming-oriented audience. In this work, we explore the possibility of deriving feedback for video streams on Twitch.tv by analyzing the sentiment of live text comments made by stream viewers in highly active channels. Automatic sentiment analysis on these comments is a challenging task, as one can compare the language used in Twitch.tv with that used by an audience in a stadium, shouting as loud as possible in sometimes nonorganized ways. This language is very different from common English, mixing Internet slang and gaming-related language with abbreviations, intentional and unintentional grammatical and orthographic mistakes, and emoji-like images called emotes. Classic lexicon-based sentiment analysis techniques therefore fail when applied to Twitch comments.
To overcome the challenge posed by the nonstandard language, we propose two unsupervised lexicon-based approaches that make heavy use of the information encoded in emotes, as well as a weakly supervised neural network–based classifier trained on the lexicon-based outputs, which is supposed to help generalization to unknown words by use of domain-specific word embeddings. To enable better understanding of Twitch.tv comments, we analyze a large dataset of comments, uncovering specific properties of their language, and provide a smaller set of comments labeled with sentiment information by crowdsourcing.
We present two case studies showing the effectiveness of our methods in generating sentiment trajectories for events live streamed on Twitch.tv that correlate well with specific topics in the given stream. This allows for a new kind of implicit real-time feedback gathering for Twitch streamers and companies producing games or streaming content on Twitch.
We make our datasets and code publicly available for further research.1
Supplemental Material
Available for Download
Supplemental movie, appendix, image and software files for, Emote-Controlled: Obtaining Implicit Viewer Feedback Through Emote-Based Sentiment Analysis on Comments of Popular Twitch.tv Channels
- Ricardo Baeza-Yates and Berthier Ribeiro-Neto. 1999. Modern Information Retrieval. Vol. 463. ACM Press, New York, NY.Google Scholar
- Francesco Barbieri, Luis Espinosa Anke, Miguel Ballesteros, Juan Soler, and Horacio Saggion. 2017. Towards the understanding of gaming audiences by modeling Twitch emotes. In Proceedings of the 3rd Workshop on Noisy User-Generated Text. 11--20. http://dblp.uni-trier.de/db/conf/aclnut/aclnut2017.html#BarbieriABSS17.Google ScholarCross Ref
- Valerio Basile and Malvina Nissim. 2013. Sentiment analysis on Italian tweets. In Proceedings of the 4th Workshop on Computational Approaches to Subjectivity, Sentiment, and Social Media Analysis. 100--107.Google Scholar
- Piotr Bojanowski, Edouard Grave, Armand Joulin, and Tomas Mikolov. 2017. Enriching word vectors with subword information. Transactions of the Association for Computational Linguistics 5 (2017), 135--146.Google ScholarCross Ref
- Meeyoung Cha, Haewoon Kwak, Pablo Rodriguez, Yong-Yeol Ahn, and Sue Moon. 2007. I tube, you tube, everybody tubes: Analyzing the world’s largest user generated content video system. In Proceedings of the 7th ACM SIGCOMM Conference on Internet Measurement. ACM, New York, NY, 1--14.Google ScholarDigital Library
- Xu Cheng, Cameron Dale, and Jiangchuan Liu. 2008. Statistics and social network of YouTube videos. In Proceedings of the 16th International Workshop on Quality of Service (IWQoS’08). IEEE, Los Alamitos, CA, 229--238.Google ScholarCross Ref
- Ben Eisner, Tim Rocktäschel, Isabelle Augenstein, Matko Bosnjak, and Sebastian Riedel. 2016. emoji2vec: Learning emoji representations from their description. In SocialNLP@EMNLP, L.-W. Ku, J. Y. J. Hsu, and C.-T. Li (Eds.). Association for Computational Linguistics, Stroudsburg, PA, 48--54. http://dblp.uni-trier.de/db/conf/acl-socialnlp/acl-socialnlp2016.html#EisnerRABR16.Google Scholar
- Joseph L. Fleiss. 1971. Measuring nominal scale agreement among many raters.Psychological Bulletin 76, 5 (1971), 378.Google Scholar
- C. J. Hutto and Eric Gilbert. 2014. VADER: A parsimonious rule-based model for sentiment analysis of social media text. In Proceedings of the 8th International Conference on Weblogs and Social Media (ICWSM-14). http://comp.social.gatech.edu/papers/icwsm14.vader.hutto.pdf.Google Scholar
- Alec Go, Richa Bhayani, and Lei Huang. 2009. Twitter Sentiment Classification Using Distant Supervision. CS224N Project Report. Stanford.Google Scholar
- Fréderic Godin, Baptist Vandersmissen, Wesley De Neve, and Rik Van de Walle. 2015. Multimedia lab @ ACL WNUT NER shared task: Named entity recognition for Twitter microposts using distributed word representations. In Proceedings of the Workshop on Noisy User-Generated Text. 146--153.Google ScholarCross Ref
- Martin J. Halvey and Mark T. Keane. 2007. Exploring social dynamics in online media sharing. In Proceedings of the 16th International Conference on World Wide Web. ACM, New York, NY, 1273--1274.Google Scholar
- H. S. Heaps. 1978. Information Retrieval: Computational and Theoretical Aspects. Academic Press, San Diego, CA.Google ScholarDigital Library
- Mehdi Kaytoue, Arlei Silva, Loïc Cerf, Wagner Meira Jr., and Chedy Raïssi. 2012. Watch me playing, I am a professional: A first study on video game live streaming. In Proceedings of the 21st International Conference on World Wide Web (WWW’12 Companion). ACM, New York, NY, 1181--1188. http://dblp.uni-trier.de/db/conf/www/www2012c.html#KaytoueSCMR12.Google ScholarDigital Library
- Yoon Kim. 2014. Convolutional neural networks for sentence classification. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP’14). 1746--1751.Google ScholarCross Ref
- Petra Kralj Novak, Jasmina Smailović, Borut Sluban, and Igor Mozetič. 2015. Sentiment of emojis. PLoS ONE 10, 12 (2015), e0144296. http://dx.doi.org/10.1371/journal.pone.0144296.Google Scholar
- Omer Levy and Yoav Goldberg. 2014. Linguistic regularities in sparse and explicit word representations. In Proceedings of the 18th Conference on Computational Natural Language Learning. 171--180.Google ScholarCross Ref
- Ruokuang Lin, Qianli D. Y. Ma, and Chunhua Bian. 2014. Scaling laws in human speech, decreasing emergence of new words and a generalized model. arxiv:cs.CL/1412.4846.Google Scholar
- Diana Löffler, Lennart Giron, and Jörn Hurtienne. 2017. Night mode, dark thoughts: Background color influences the perceived sentiment of chat messages. In INTERACT. Lecture Notes in Computer Science, Vol. 10514. Springer, 184--201. http://dblp.uni-trier.de/db/conf/interact/interact2017-2.html#LofflerGH17.Google Scholar
- Vittorio Loreto, Vito D. P. Servedio, Steven H. Strogatz, and Francesca Tria. 2016. Dynamics on expanding spaces: Modeling the emergence of novelties. In Creativity and Universality in Language. Springer, 59--83.Google Scholar
- Andrew L. Maas, Raymond E. Daly, Peter T. Pham, Dan Huang, Andrew Y. Ng, and Christopher Potts. 2011. Learning word vectors for sentiment analysis. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies—Volume 1 (HLT’11). 142--150. http://dl.acm.org/citation.cfm?id=2002472.2002491.Google ScholarDigital Library
- Julian J. McAuley, Rahul Pandey, and Jure Leskovec. 2015. Inferring networks of substitutable and complementary products. arXiv:1506.08839. http://dblp.uni-trier.de/db/journals/corr/corr1506.html#McAuleyPL15.Google Scholar
- Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013a. Efficient estimation of word representations in vector space. arXiv:1301.3781.Google Scholar
- Tomas Mikolov, Wen-Tau Yih, and Geoffrey Zweig. 2013b. Linguistic regularities in continuous space word representations. In Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (HLT-NAACL’13). 746--751.Google Scholar
- Preslav Nakov, Alan Ritter, Sara Rosenthal, Fabrizio Sebastiani, and Veselin Stoyanov. 2016. SemEval-2016 task 4: Sentiment analysis in Twitter. In Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval’16). 1--18.Google ScholarCross Ref
- Preslav Nakov, Sara Rosenthal, Zornitsa Kozareva, Veselin Stoyanov, Alan Ritter, and Theresa Wilson. 2013. SemEval-2013 task 2: Sentiment analysis in Twitter. In Proceedings of the 2nd Joint Conference on Lexical and Computational Semantis (*SEM), Volume 2: Proceedings of the 7th International Workshop on Semantic Evaluation (SemEval’13). 312--320.Google Scholar
- Sascha Narr, Michael Hulfenhaus, and Sahin Albayrak. 2012. Language-independent Twitter sentiment analysis. In Proceedings of the Workshop on Knowledge Discovery, Data Mining, and Machine Learning (KDML at LWA’12). 12--14.Google Scholar
- Gustavo Nascimento, Manoel Ribeiro, Loïc Cerf, Natália Cesário, Mehdi Kaytoue, Chedy Raïssi, Thiago Vasconcelos, and Wagner Meira. 2014. Modeling and analyzing the video game live-streaming community. In Proceedings of the 2014 9th Latin American Web Congress (LA-WEB’14). IEEE, Los Alamitos, CA, 1--9.Google ScholarDigital Library
- Polygon. 2018. Diablo: Immortal Broke the Unspoken Rules of Blizzard, and BlizzCon. Retrieved March 7, 2020 from https://www.polygon.com/2018/11/5/18064290/blizzard-diablo-immortal-reaction-explainer-blizzcon.Google Scholar
- Sara Rosenthal, Noura Farra, and Preslav Nakov. 2017. SemEval-2017 task 4: Sentiment analysis in Twitter. In Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval’17). 502--518.Google ScholarCross Ref
- Sara Rosenthal, Preslav Nakov, Svetlana Kiritchenko, Saif Mohammad, Alan Ritter, and Veselin Stoyanov. 2015. Semeval-2015 task 10: Sentiment analysis in Twitter. In Proceedings of the 9th International Workshop on Semantic Evaluation (SemEval’15). 451--463.Google ScholarCross Ref
- Sara Rosenthal, Alan Ritter, Preslav Nakov, and Veselin Stoyanov. 2014. SemEval-2014 task 9: Sentiment analysis in Twitter. In Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval’14). 73--80. http://www.aclweb.org/anthology/S14-2009.Google ScholarCross Ref
- Herbert A. Simon. 1955. On a class of skew distribution functions. Biometrika 42, 3--4 (1955), 425--440.Google ScholarCross Ref
- Thomas Smith, Marianna Obrist, and Peter C. Wright. 2013. Live-streaming changes the (video) game. In EuroITV, P. Paolini, P. Cremonesi, and G. Lekakos (Eds.). ACM, New York, NY, 131--138. http://dblp.uni-trier.de/db/conf/euroitv/euroitv2013.html#SmithOW13.Google Scholar
- Duyu Tang, Furu Wei, Nan Yang, Ming Zhou, Ting Liu, and Bing Qin. 2014. Learning sentiment-specific word embedding for Twitter sentiment classification. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 1555--1565. http://dblp.uni-trier.de/db/conf/acl/acl2014-1.html#TangWYZLQ14.Google ScholarCross Ref
- Alexander Yeh. 2000. More accurate tests for the statistical significance of result differences. In Proceedings of the 18th Conference on Computational Linguistics—Volume 2. 947--953.Google ScholarDigital Library
- G. Udny Yule. 1925. II. A mathematical theory of evolution, based on the conclusions of Dr. JC Willis, FR S. Philosophical Transactions of the Royal Society of London: Series B 213, 402--410 (1925), 21--87.Google Scholar
- Cong Zhang and Jiangchuan Liu. 2015. On crowdsourced interactive live streaming: A Twitch.tv-based measurement study. In Proceedings of the 25th ACM Workshop on Network and Operating Systems Support for Digital Audio and Video. ACM, New York, NY, 55--60.Google ScholarDigital Library
- Ye Zhang and Byron Wallace. 2017. A sensitivity analysis of (and practitioners’ guide to) convolutional neural networks for sentence classification. In Proceedings of the 8th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). 253--263.Google Scholar
Index Terms
- Emote-Controlled: Obtaining Implicit Viewer Feedback Through Emote-Based Sentiment Analysis on Comments of Popular Twitch.tv Channels
Recommendations
Who will Subscribe to My Streaming Channel?: The Case of Twitch
CSCW '19 Companion: Companion Publication of the 2019 Conference on Computer Supported Cooperative Work and Social ComputingThis paper investigates whether subscribers (paying viewers) can be identified by their chat messages in live streaming services, Twitch. Collecting data of twenty streamers on Twitch, this study conducts two studies: (i) presenting sentimental ...
Joint sentiment/topic model for sentiment analysis
CIKM '09: Proceedings of the 18th ACM conference on Information and knowledge managementSentiment analysis or opinion mining aims to use automated tools to detect subjective information such as opinions, attitudes, and feelings expressed in text. This paper proposes a novel probabilistic modeling framework based on Latent Dirichlet ...
Tell Me Before You Stream Me: Managing Information Disclosure in Video Game Live Streaming
Live streaming is a new media format that simultaneously records and broadcasts in real time, in multiple channels including video, audio, and text. A new application area of live streaming is in the video game community, where players stream their ...
Comments