Ramsha Saeed
Skip Abstract Section
Abstract
Continuous proliferation of hate speech in different languages on social media has drawn significant attention from researchers in the past decade. Detecting hate speech is indispensable irrespective of the scale of use of language, as it inflicts huge harm on society. This work presents a first resource for classifying the severity of hate speech in addition to classifying offensive and hate speech content. Current research mostly limits hate speech classification to its primary categories, such as racism, sexism, and hatred of religions. However, hate speech targeted at different protected characteristics also manifests in different forms and intensities. It is important to understand varying severity levels of hate speech so that the most harmful cases of hate speech may be identified and dealt with earlier than the less harmful ones. In this work, we focus on detecting offensive speech, hate speech, and multiple levels of hate speech in the Urdu language. We investigate three primary target categories of hate speech: religion, racism, and national origin. We further divide these categories into levels based on the severity of hate conveyed. The severity levels are referred to as symbolization, insult, and attribution. A corpus comprising more than 20,000 tweets against the corresponding hate speech categories and severity levels is collected and annotated. A comprehensive experimentation scheme is applied using traditional as well as deep learning–based models to examine their impact on hate speech detection. The highest macro-averaged F-score yielded for detecting offensive speech is 86% while the highest F-scores for detecting hate speech with respect to ethnicity, national origin, and religious affiliation are 80%, 81%, and 72%, respectively. This shows that results are very encouraging and would provide a lead towards further investigation in this domain.
- [1] . 2016. But I did not mean it! Intent classification of racist posts on Tumblr. In 2016 European Intelligence and Security Informatics Conference (EISIC’16). IEEE, 124–127.Google Scholar
Cross Ref
- [2] . 2020. Automatic detection of offensive language for Urdu and Roman Urdu. IEEE Access 8 (2020), 91213–91226.Google ScholarCross Ref
- [3] . 2009. Assas-Band, an affix-exception-list based Urdu stemmer. In Proceedings of the 7th Workshop on Asian Language Resources. Association for Computational Linguistics, 40–46.Google ScholarDigital Library
- [4] . 2018. Are they our brothers? Analysis and detection of religious hate speech in the Arabic twittersphere. In 2018 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM’18). IEEE, 69–76.Google ScholarCross Ref
- [5] . 2017. Hate speech detection in the Indonesian language: A dataset and preliminary study. In 2017 International Conference on Advanced Computer Science and Information Systems (ICACSIS’17).Google ScholarCross Ref
- [6] . 2018. Automatic identification and classification of misogynistic language on Twitter. In International Conference on Applications of Natural Language to Information Systems. Springer, 57–64.Google ScholarDigital Library
- [7] . 2017. Deep learning for hate speech detection in tweets. In Proceedings of the 26th International Conference on World Wide Web Companion. International World Wide Web Conferences Steering Committee, 759–760.Google ScholarDigital Library
- [8] . 2021. Profiling spreaders of hate speech with N-grams and RoBERTa. In CLEF (Working Notes). 1822–1828.Google Scholar
- [9] . 2018. Urdu word segmentation using conditional random fields (CRFs). In Proceedings of the 27th International Conference on Computational Linguistics. Association for Computational Linguistics, 2562–2569. http://aclweb.org/anthology/C18-1217.Google Scholar
- [10] . 2017. Enriching word vectors with subword information. Transactions of the Association for Computational Linguistics 5 (2017), 135–146.Google ScholarCross Ref
- [11] . 2018. Overview of the EVALITA 2018 hate speech detection task. In EVALITA 6th Evaluation Campaign of Natural Language Processing and Speech Tools for Italian, Vol. 2263. CEUR, 1–9.Google Scholar
- [12] . 2016. Us and them: Identifying cyber hate on Twitter across multiple protected characteristics. EPJ Data Science 5, 1 (2016), 11.Google ScholarCross Ref
- [13] . 2017. Mean Birds: Detecting aggression and bullying on Twitter. arXiv preprint arXiv:1702.06877 (2017).Google Scholar
- [14] . 2015. Machine learning approach for detection of cyber-aggressive comments by peers on social media network. In 2015 International Conference on Advances in Computing, Communications and Informatics (ICACCI’15). IEEE, 2354–2358.Google ScholarCross Ref
- [15] . 2017. Harnessing the power of text mining for the detection of abusive content in social media. In Advances in Computational Intelligence Systems. Springer, 187–205.Google ScholarCross Ref
- [16] . 2021. Abusive and threatening language detection in Urdu using boosting based and BERT based models: A comparative approach. arXiv preprint arXiv:2111.14830 (2021).Google Scholar
- [17] . 2017. Automated hate speech detection and the problem of offensive language. arXiv preprint arXiv:1703.04009 (2017).Google Scholar
- [18] . 2015. Hate speech detection with comment embeddings. In Proceedings of the 24th International Conference on World Wide Web. ACM, 29–30.Google ScholarDigital Library
- [19] . 2001. Re-in/citing linguistic injuries: Speech acts, cyberhate, and the spatial and temporal character of networked environments. Computers and Composition 18, 3 (2001), 293–304.Google ScholarCross Ref
- [20] . Hate speech. (2022). Retrieved January 31, 2023 from https://www.facebook.com/communitystandards/hate_speech.Google Scholar
- [21] . 2018. A survey on automatic detection of hate speech in text. ACM Computing Surveys 51, 4 (2018), 1–30.Google ScholarDigital Library
- [22] . 2017. Using convolutional neural networks to classify hate-speech. In Proceedings of the 1st Workshop on Abusive Language Online. 85–90.Google ScholarCross Ref
- [23] . 2017. Recognizing explicit and implicit hate speech using a weakly supervised two-path bootstrapping approach. arXiv preprint arXiv:1710.07394 (2017).Google Scholar
- [24] . 2013. Peer-to-peer insult detection in online communities. IITK Unpubl (2013).Google Scholar
- [25] . 2018. INGEOTEC at MEX-A3T: Author profiling and aggressiveness analysis in Twitter using \(\mu\)TC and EvoMSA. In IberEval@ SEPLN. 128–133.Google Scholar
- [26] . 2004. Classifying racist texts using a support vector machine. In Proceedings of the 27th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. ACM, 468–469.Google ScholarDigital Library
- [27] . 2014. Hate speech: A study of Pakistan’s cyberspace. Islamabad, Pakistan: Bytes4all (2014).Google Scholar
- [28] . 2014. Cyber bullying detection using social and textual analysis. In Proceedings of the 3rd International Workshop on Socially-Aware Multimedia. 3–6.Google ScholarDigital Library
- [29] . 2008. The pragmatics of swearing. Journal of Politeness Research. Language, Behaviour, Culture 4, 2 (2008), 267–288.Google Scholar
- [30] . 2016. Code of conduct on countering illegal hate speech online: First results on implementation. European Commission.[cit. 8. březen 2018] (2016).Google Scholar
- [31] Vera Jourová. 2016. Code of Conduct on countering illegal hate speech online: First results on implementation. Factsheet Directorate-General for Justice and Consumers.Google Scholar
- [32] Ezgi Kan, Merve Nebioglu, Seyma Özkan, Funda Tekin, and Gamze Tosun. 2018. Media watch on hate speech report January–April 2018. Hrant Dink Foundation.Google Scholar
- [33] . 2021. Hate speech detection in Roman Urdu. ACM Transactions on Asian and Low-Resource Language Information Processing 20, 1 (2021), 1–19.Google ScholarDigital Library
- [34] . 2013. High times for hate crimes: Explaining the temporal clustering of hate-motivated offending. Criminology 51, 4 (2013), 871–894.Google ScholarCross Ref
- [35] . 2017. The power of character n-grams in native language identification. In Proceedings of the 12th Workshop on Innovative Use of NLP for Building Educational Applications. 382–389.Google ScholarCross Ref
- [36] . 2019. Hate speech on social media: Global comparisons. (
June 2019). Retrieved January 31, 2023 from https://www.cfr.org/backgrounder/hate-speech-social-media-global-comparisons.Google Scholar - [37] . 2014. Distributed representations of sentences and documents. In International Conference on Machine Learning. 1188–1196.Google ScholarDigital Library
- [38] . 2001. Responses to Internet hate sites: Is speech too free in cyberspace? Communication Law & Policy 6, 2 (2001), 287–317.Google ScholarCross Ref
- [39] . 2019. Overview of the HASOC track at fire 2019: Hate speech and offensive content identification in Indo-European languages. In Proceedings of the 11th Forum for Information Retrieval Evaluation. 14–17.Google ScholarDigital Library
- [40] . 2009. Hate speech or “reasonable racism?” The other in Stormfront. Journal of Mass Media Ethics 24, 4 (2009), 251–268.Google ScholarCross Ref
- [41] . 2013. Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781 (2013).Google Scholar
- [42] Bastian Birkeneder, Jelena Mitrovic, Julia Niemeier, Leon Teubert, and Siegfried Handschuh. 2018. upInf - Offensive language detection in German tweets. In Proceedings of GermEval 2018, 14th Conference on Natural Language Processing (KONVENS’18).Google Scholar
- [43] . 2017. Abusive language detection on Arabic social media. In Proceedings of the 1st Workshop on Abusive Language Online. 52–56.Google ScholarCross Ref
- [44] . 2016. Abusive language detection in online user content. In Proceedings of the 25th International Conference on World Wide Web. International World Wide Web Conferences Steering Committee, 145–153.Google ScholarDigital Library
- [45] . 2014. Exposure to online hate among young social media users. Sociological Studies of Children & Youth 18, 1 (2014), 253–273.Google ScholarCross Ref
- [46] . 2018. Effective hate-speech detection in Twitter data using recurrent neural networks. Applied Intelligence 48, 12 (2018), 4730–4742.Google ScholarDigital Library
- [47] . 2021. Abusive language and hate speech detection for Indonesian-local language in social media text. In International Conference on Computing and Information Technology. Springer, 88–98.Google ScholarCross Ref
- [48] . 2021. Multilingual offensive language identification for low-resource languages. Transactions on Asian and Low-Resource Language Information Processing 21, 1 (2021), 1–13.Google ScholarDigital Library
- [49] . 1997. Recommendation No. R (97) 20 of the Committee of Ministers to member states on “hate speech”. (1997). Retrieved January 31, 2023 from https://rm.coe.int/1680505d5b.Google Scholar
- [50] . 2020. Hate-speech and offensive language detection in Roman Urdu. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP’20). 2512–2522.Google ScholarCross Ref
- [51] . 2020. Roman Urdu multi-class offensive text detection using hybrid features and SVM. In 2020 IEEE 23rd International Multitopic Conference (INMIC’20). IEEE, 1–5.Google ScholarCross Ref
- [52] . 2017. Detecting nastiness in social media. In Proceedings of the 1st Workshop on Abusive Language Online. 63–72.Google ScholarCross Ref
- [53] . 2020. Hateful conduct policy. (2020). Retrieved January 31, 2023 from https://help.twitter.com/en/rules-and-policies/hateful-conduct-policy.Google Scholar
- [54] . 2021. Profiling hate speech spreaders on Twitter: SVM vs. Bi-LSTM. In CLEF (Working Notes). 2193–2200.Google Scholar
- [55] . 2012. Detecting hate speech on the World Wide Web. In Proceedings of the 2nd Workshop on Language in Social Media. Association for Computational Linguistics, 19–26.Google ScholarDigital Library
- [56] . 2016. Hateful symbols or hateful people? Predictive features for hate speech detection on Twitter. In SRW@ HLT-NAACL. 88–93.Google Scholar
- [57] . 2018. Hate speech on Twitter: A pragmatic approach to collect hateful and offensive expressions and perform hate speech detection. IEEE Access 6 (2018), 13825–13835.Google ScholarCross Ref
- [58] . 2018. Overview of the germeval 2018 shared task on the identification of offensive language. (2018).Google Scholar
- [59] . 2020. Hate speech policy. (2020). Retrieved January 31, 2023 from https://support.google.com/youtube/answer/2801939?hl=en.Google Scholar
- [60] . 2019. Semeval-2019 task 6: Identifying and categorizing offensive language in social media (offenseval). arXiv preprint arXiv:1903.08983 (2019).Google Scholar
- [61] . 2018. Detecting hate speech on Twitter using a convolution-GRU based deep neural network. In European Semantic Web Conference. Springer, 745–760.Google ScholarDigital Library
- [62] . 2016. Automatic detection of cyberbullying on social networks based on bullying features. In Proceedings of the 17th International Conference on Distributed Computing and Networking. ACM, 43.Google ScholarDigital Library
Index Terms
- Detection of Offensive Language and ITS Severity for Low Resource Language
Recommendations
Hate speech and offensive language detection in Dravidian languages using deep ensemble framework
AbstractSocial networking platforms gained widespread popularity and are used for various activities like: promoting products, sharing news, achievements and many more. On the other hand, it is also used for spreading rumors, bullying people, ...
Highlights- Proposed a weighted ensemble framework for hate and offensive code-mixed posts identification on social platforms.
An acoustic model and linguistic analysis for Malayalam disyllabic words: a low resource language
AbstractAutomatic Speech Recognition (ASR) has reaped a lot of attention in recent years. Despite the recent advancements in ASR, the potential for extracting the raw features from speech remains lacking. This paper proposes an Automatic Speech ...
Label modification and bootstrapping for zero-shot cross-lingual hate speech detection
AbstractThe goal of hate speech detection is to filter negative online content aiming at certain groups of people. Due to the easy accessibility and multilinguality of social media platforms, it is crucial to protect everyone which requires building hate ...
Comments