Detection of Hate-Speech Tweets Based on Deep Learning: A Review

− Cybercrime, cyberbullying, and hate speech have all increased in conjunction with the use of the internet and social media. The scope of hate speech knows no bounds or organizational or individual boundaries. This disorder affects many people in diverse ways. It can be harsh, offensive, or discriminating depending on the target's gender, race, political opinions, religious intolerance, nationality, human color, disability, ethnicity, sexual orientation, or status as an immigrant. Authorities and academics are investigating new methods for identifying hate speech on social media platforms like Facebook and Twitter. This study adds to the ongoing discussion about creating safer digital spaces while balancing limiting hate speech and protecting freedom of speech. Partnerships between researchers, platform developers, and communities are crucial in creating efficient and ethical content moderation systems on Twitter and other social media sites. For this reason, multiple methodologies, models, and algorithms are employed. This study presents a thorough analysis of hate speech in numerous research publications. Each article has been thoroughly examined, including evaluating the algorithms or methodologies used, databases, classification techniques, and the findings achieved. In addition, comprehensive discussions were held on all the examined papers, explicitly focusing on consuming deep learning techniques to detect hate speech.


INTRODUCTION
Online social media enables the spread of humanities to be associated.However, one disadvantage of these social media is the ability to publish and propagate malicious and harmful content [1].Hate speech seeks to promote violence and incite hostility against individuals or groups based on characteristics-such as sexual orientation/gender identity, religion, disability, gender, age, veteran status, or age, gender, or disability.The increasing number of social media platforms has caused matters to be excessive [2].Not all substances are pertinent; some may cause damage to individuals, which is a disgraceful indictment when those who use the media to spread hatred [3].Hate speech affects everyone regardless of age, which might be too young, an adult, or an older adult [4].Individuals use a lot of social media platforms daily to express their thoughts, emotions, and progress.Usually, the comments are more hate speech than positive comments, which leads to different kinds of problems; for that reason, it is essential to select a method to detect the words, analyze them, and then show the result of accuracy [5].Deep learning utilizes networks that have multiple layers [6].This allows them to understand patterns and representations in data by learning from examples [7].
The training process involves providing these networks with amounts of labeled data, enabling the system to adjust its parameters until it can accurately predict or classify information [8].This ability makes deep learning especially effective for tasks that involve decision-making and pattern recognition.Deep understanding encompasses such architectures tailored for specific tasks, each designed to address the unique challenges of different data types [9].Artificial neural networks, inspired by biological neurons, serve as the foundational components of deep learning, replicating the core elements of human intellect [10].These networks, which have several layers of nodes, have led to many architectures, each specialized for distinct tasks and applications.Convolutional Neural Networks (CNN) excel in analyzing videos and photos, Recurrent Neural Networks (RNN) are adept at addressing issues related to sequential data, and Generative Adversarial Networks (GAN) are at the forefront of generating synthetic data that closely mimics accurate data.[11], [12].Hate speech detection in social media Twitter is detecting and defining all the cyberbullying an individual can receive on their regular posts [13].The hate speech might be on racism, sexuality, child abuse, Politicians, and many others.Detecting hate speech is difficult because its nature and context-specific attributes frequently characterize it [14], [15].Conventional rule-based or keyword-based methods may face problems in accurately identifying the subtleties of hate speech, as hate speech can take on different forms, such as indirect language, changing slang, or statements that depend on the context [16].Hate speech detection utilizes deep learning, specifically natural language processing algorithms [17].This paper aims to review other works on detecting hate speech in social media using different types of deep learning methods.The rest of this paper is organized to describe the background theory of deep learning and the algorithms to detect hate speech on Twitter platforms.After that, in section 2, different research on using CNN and GRU, CNN, and LSTM to detect hate speech on social media are described and compared in Table 1 and Table 2. Finally, section 3 describes a discussion session, a conclusion and limitations, and a list of references are shown.

II.
RESEARCH METHODOLOGY Detecting hate speech in social media requires datasets and an algorithm to learn and extract the comments to be analyzed [18].For that reason, researchers are using deep learning algorithms to detect hate speech.Deep Learning, a subset of machine learning, aims to replicate the brain's function for analyzing and processing data [15].At its core, Learning utilizes networks of intricate structures made up of interconnected nodes to model and interpret complex patterns within large datasets.What makes deep learning understandable is its ability to learn representations of data automatically [19], enabling it to identify features and relationships that may pose challenges for conventional machine learning methods [20].This technology has shown achievements in file recognition, speech processing, natural language understanding, and autonomous systems [21], [22].The continuous progress in deep learning algorithms alone and the availability of computing resources have paved the way for advancements in artificial intelligence [23].These advancements shape technology and impact domains like healthcare, finance, and selfdriving vehicles.This technology has shown achievements in file recognition, speech processing, natural language understanding, and autonomous systems [24] [20].The continuous progress in deep learning algorithms alone and the availability of computing resources have paved the way for advancements in artificial intelligence [25], [26].These advancements shape technology and impact domains like healthcare, finance, and self-driving vehicles.The Advances in learning models, such as (CNNs) for image processing and (RNNs) for sequential data, have resulted in substantial changes in domains such as machine vision, natural language processing, and recognition of words [8].Deep Learning is executed by neural networks, which consist of multiple layers, a concept that is not novel [27], [28].Nevertheless, its popularity has surged recently, driven mainly by three factors: Firstly, there has been a significant enhancement in processing capabilities, such as video cards and graphical processors.Secondly, the availability of affordable computer hardware has played a crucial role.Lastly, recent advancements and breakthroughs in Deep Learning research have also contributed to its rise.Deep learning algorithms can be categorized into three groupings based on whether the algorithms are trained to produce achievable results.The subgroups are unsupervised, supervised, and hybrid [27].The reliability of each deep learning algorithm is demonstrated in Figure .1. Due to its potentially devastating effects on society, identifying instances of hate speech in digital communications is a significant difficulty [29].One viable strategy for this problem is to use cutting-edge technologies like (CNNs).Combining (CNNs) and (GRUs) offers a powerful technique for detecting hate speech, harnessing the benefits of both architectures [14].While CNNs are great at comprehending short-term dependencies in sequences, (GRUs) excel at understanding long-term dependencies [30].In this combined method, textual information is tokenized and processed, then supplied into an embedding layer, as shown in Figure 2 [31].It covers identifying hate speech on social media, particularly Twitter and provides a convolution-GRU-based deep neural network solution [32].The authors address hate speech identification features such as a bag of words, word and character n-grams, sentiment analysis, linguistic resources, and standard and deep learning methods [33].The authors also noted a need for comparable evaluation and limited public hate speech datasets.
Fig. 2 CNN+GRU Architecture [31] The CNN's convolutional layers then process the embedded sequences to pull out abstract patterns and features at various levels of granularity [34].The (GRU) then uses these traits to learn to record sequential and contextual information, drawing connections between previously unrelated parts of the text [22].The (GRU)'s gated structure allows for the storage and updating of data, which aids in the nuanced understanding of the text's context and the detection of hate speech [35].Training a mixed model requires iteratively adjusting parameters to achieve a target loss function minimization.To better detect hate speech, the model combines (CNNs) for feature extraction with GRU for sequence modeling [10].The model's performance and robustness in identifying hate speech across online content can be improved by fine-tuning, optimizing, and studying other architectures and hyperparameters.There are negative impacts on humans' mental and physical health as a direct result of the rapid growth of social media mediums and the constant use by their users.To guard against and avoid any mishaps that might take place behind the scenes on social media platforms, hate speech has become one of the most hazardous subjects to discuss.Researchers turned to deep learning strategies, which will be covered in more detail in the following paragraphs, to identify instances of hate speech on social media sites like Twitter [36], [37] proposed method utilizes a deep learning ensemble approach that combines multiple sub-models to improve classification performance.The paper also mentions the use of a publicly available embedding model.As for the number of datasets used, the article does not provide a specific number.However, the authors mention using the SemEval dataset to compare findings and a split set with an 85/15 ratio for testing purposes.Regarding the results, the paper reports a 5-point improvement in F-measure compared to previous work.The ensemble approach also showed a reduction in variance compared to individual models.[37]discusses a new method for detecting hate speech on Twitter using a Convolution-(GRU)-based deep neural network.The proposed method outperforms previous methods on 6 out of 7 datasets by between 1 and 13% in F1.The comparison was made against several baselines and state-of-the-art on the most extensive publicly available Twitter datasets.The paper also discusses the implementation, parameter tuning, and evaluation metrics used in the study.[38]systematically analyze hate speech in 9 languages and offer a low-resource detection approach.They test CNN-GRU, BERT, and LASER embedding with logistic regression to detect hate speech in different languages under different scenarios.The authors also present a portfolio of effective models for each language based on data availability.The research sheds light on the difficulties of detecting hate speech in languages other than English and suggests a low-resource detection approach.In [39] they used CNN and GRU algorithms to detect hate speech.CNN is better for local features, and GRU is better for long-range context dependencies.LSTM, another RNN, is slower and more likely to overfit small datasets than GRU.Both models have pros and cons, and hyperparameter adjustment affects performance.[40] suggests utilizing deep neural networks to detect internet hate speech.According to the authors, automatic identification is needed due to digital communication and online hostility.They propose multiclass classification using fastText, BERT embeddings, SVM, and deep neural network classifiers.They find encouraging results using their method on numerous datasets.[41] Combining GRU, ELMo, BERT, and CNN data improves classification performance.The proposed approach has good accuracy and F1 score, making it a feasible option for social media hate speech.However, the approach's success may depend on the training data's quality and representativeness and the classifiers and fusion methods used.[9]discusses a model for detecting hate speech on Twitter using CNN and character-level representation.The authors present their approach to building and cleaning datasets, as well as the architecture of their model.They also discuss related work in sentiment analysis and hate speech detection.The results of their experiments were presented and discussed, with the model achieving satisfactory accuracy compared to the state-ofthe-art.[42] highlight various hate speech detection research using these neural network topologies.Zimmerman et al. represented 50 tokens using CNN parameters, while Founta et al. offered a two-layer RNN model using GRU and metadata on people, networks, and content.Some hate speech detection investigations have used hybrid CNN and RNN models, which perform better than either architecture alone.The research thoroughly analyzes CNN and GRU in hate speech detection and shows their potential to improve model accuracy.[43] discusses a study that classified Bengali Facebook comments as Hate Speech, Communal Attacks, Incitement, Religious Hatred, Political Comments, and Religious Comments using (GRU).The study presented an annotated Bengali corpus of six-class comments.The experiment indicates that a GRU-based classification model can achieve 70.10% accuracy.The article found that the BiLSTM performed best with a weighted classification F1 score of 91%.In [36], to identify harmful remarks in virtual communities and social media platforms.The algorithm employs pre-trained word embeddings and many channels to forecast multilabel toxicity indicators precisely.The report additionally examines pertinent literature, outlines the structure of the suggested model, and presents the findings of its evaluation.Moreover, in [37],the research paper discussed the use of deep neural network (DNN) architectures, including (CNN), (GRU), and Universal Language Model Fine-tuning (ULMFiT), for the classification of hate speech in tweets.It highlights the advantages of using DNN models for hate speech detection and emphasizes the superior performance of the ULMFiT model over other architectures.The ULMFiT model, based on the three-layered Average-SGD Weight-Dropped Long Short-Term Memory (AWD-LSTM) architecture, demonstrates substantial improvements in accuracy and F1 score.After that, in [46].It outlines the method used in the study, which involves transforming speech signals from the time domain to the frequency domain, extracting features using mel-frequency cepstral coefficients, and using a CNN-GRU model to detect and classify dysarthria patients and healthy people.The article also mentions the prevalence of dysarthria in patients with neurological diseases and the potential applications of this technology in healthcare.[47] Investigates the frequency of hate speech on Twitter amidst the COVID-19 outbreak.The study uses machine learning algorithms to discern and examine the data to ascertain patterns and trends within hate speech.The results indicate that there was a significant presence of hate speech about COVID-19 on Twitter during the pandemic, with specific demographics being disproportionately targeted.The authors propose that their discoveries can be employed to counteract hate speech and foster inclusiveness on social media sites.The researcher in [29]Worked on A comprehensive analysis of scholarly research on the identification of hate speech in social media, particularly on Twitter, using the utilization of neural networks.The review comprises a subset of 20 studies from a pool of 565 distinct works, chosen according to specific inclusion and exclusion criteria and the study's primary and secondary inquiries.The review examines the models, algorithms, and methodologies employed in these investigations and the datasets, languages, and nations implicated.The review reveals that the predominant models used are based on Convolutional Neural Networks (CNNs).While these models offer certain advantages over recent models in detecting hate speech, they also exhibit limitations and deficiencies.These include challenges in automated hate speech detection, compatibility with specific languages only, and ambiguity in speech classification when applied to datasets in different languages.The review presents a thorough overview of the experimental particulars and findings of the incorporated investigations, encompassing F1 score, recall, precision, and accuracy.In [48], The authors address the issue of vocabulary mismatch in tweets by using feature expansion to build a corpus of similarity.The study uses four deep learning methods: Convolutional Neural Network (CNN), Gated Recurrent Unit (GRU), and a combination of the CNN-GRU and GRU-CNN classifications with Boolean representation as feature extraction.The experiment was carried out to find the best performance by comparing the value of the accuracy.The accuracy results were obtained from the average of the five tests in each scenario.One limitation is using pretrained word embeddings, which may not capture all relevant information for the specific task-also, there is a lack of explanation for the decisions made by the model, which may be necessary for applications.Additionally, the proposed approach may not be suitable for low-resource languages or domains with limited training data.

B. Convolutional Neural Networks (CNN) and Long
Short-Term Memory (LSTM) for Detecting Hate Speech: CNNs allow for fast processing of text data, extracting linguistic features and patterns that can be used to identify hate speech from other types of expression [49], [50].Embeddings, convolution, pooling, and dense connections are just some of the layers that can train CNN models to recognize the linguistic and contextual cues indicative of hate speech.[51]addresses the difficulties of identifying hate speech on social media and the strategies that have proven successful in doing so.When evaluating the precision of various machine learning and deep learning approaches, the authors find that BiRNN (Bidirectional Recurrent Neural Network) excels.[52]Each time step, the LSTM neural network processes one word embedding from a multilayer perceptron (MLP) with two hidden layers, keeping word order.LSTM neural network output is processed via hyperbolic tangent activation.The number of terms must be specified before training the recurrent neural network.HaterNet detected at most 33 terms after preprocessing the tagged corpus tweets.All tweets under 33 terms have padding rows of 0s.In.After that, [53]benefits from identifying risky comments using LSTM neural networks.The authors cleaned data with NLTK and built their own stop words.The model had great precision, recall, and accuracy.The paper tries to filter hate speech and make social media safer.[54] Presents a comprehensive approach to detecting hate speech on Twitter using traditional machine and deep learning techniques.The authors explore shallow learning algorithms, including logistic regression, random forest, decision tree, naïve Bayes, K-NN, SVM, and deep learning methods such as LSTM, BiLSTM, and CNN.Further examines the difficulties linked to detecting hate speech on social media platforms.It provides insights into potential applications of the proposed approach for mitigating the impact of hate speech on individuals and communities.On the road to realizing their full potential, these models are trained extensively using labeled datasets and then fine-tuned [55].Aside from helping with automatic identification, using this model for hate speech detection allows for prompt interventions and mitigation methods, making the internet safer for everyone.CNNs and (LSTM) networks are crucial in identifying hate speech in textual data [56].CNNs detect specific patterns and characteristics in the text using convolutional and pooling layers to extract important linguistic clues about hate speech.Meanwhile, (LSTM) models excel in comprehending the sequential structure of language, effectively capturing contextual interdependencies and enduring associations among words or phrases.By leveraging the capabilities of (CNNs) for extracting features and (LSTMs) [57] , [58].This collaboration allows for identifying hate speech content that is nuanced and embedded within its context.In [49],the study focuses on detecting hate speech diffusion on Twitter using graphbased methods.It finds that classification based on the sharing graph yields strong F1 scores for hate speech detection and highlights the vulnerability of existing textual hate speech detection methods to adversarial attacks.The study also considers the effects of automated bots in sharing hate speech content.The implications of the findings are relevant for addressing hate speech and online safety on social media platforms.[7]The DeepHate model utilizes multi-faceted text representations, including semantic, sentiment, and topical information, to improve hate speech detection.By combining pre-trained word embeddings, sentiment analysis, and Latent Dirichlet Allocation (LDA), the model achieves better performance compared to other configurations.Additionally, empirical studies provide insights into the salient features that aid in hate speech detection.[7]discusses the challenges of detecting hate speech in social media, particularly on platforms like Twitter.It explores deep learning approaches, such as various embeddings, to improve the detection of different types of hate speech.The experiments on publicly available datasets showed significant improvements in accuracy and F1-score, offering promising solutions to combat hate speech online.The authors [16] discussed cyberbullying detection using machine learning and deep learning approaches.Cyberbullying's potential dangers are highlighted, as are the benefits of effective detection methods.Methods for categorizing cyberbullying are [29] 2023 2400 tweets CNN 94% Advantages of some of the models in detecting abusive language in nongeneralizable (unseen) problems and in terms of their speed and ability to load all the data quickly.
Handling different topical focuses and targets and the ability to load data quickly.[48] 2023 181 investigated in the research.The research examines machine learning and deep learning strategies for cyberbullying detection using two datasets: the Wikipedia Talk Corpus and the Twitter Hate Speech Corpus.The paper also addresses the role that technology plays in making cyberbullying more severe than in-person harassment.Overall, the research gives valuable insights into the detection of cyberbullying and the use of machine learning and deep learning technologies to solve this issue.[60] examines the challenges of detecting hate speech on social networking sites and the research in natural language processing (NLP) and machine learning (ML) to address this issue.The authors used an up-sampling method to balance the data.They implemented deep learning models like Long Short-Term Memory (LSTM) and Bidirectional Long Short-Term Memory (Bi-LSTM) for improved accuracy in detecting hate speech.LSTM was found to have better accuracy, precision, and F1 score, while Bi-LSTM had a higher recall.[61]Explores the problem of harmful language on social media platforms and suggests a technique for identifying it through the utilization of conventional machine learning models, as well as BERT and fastText embedding with deep neural networks.The authors merged the ALONE and HASOC'20 datasets to create a consolidated dataset for their research.The researchers pre-processed data and applied various machine learning and ensemble techniques, including TF-IDF, POS tagging, and trigrams.Among these approaches, LR and XGBoost yielded the most favorable results.In the second case, word embedding techniques such as fastText and BERT were employed to generate embeddings, which were subsequently utilized as inputs for DNN classifiers.The researchers used multiple deep neural network classifiers and found optimal performance was achieved by combining BERT embeddings with a convolutional neural network (CNN).[62] focused on hate speech detection methods using deep and shallow learning techniques.The study provides insights into the detection accuracy, computational efficiency, and practical implications of using pre-trained models and domain generalization.The paper presents a large-scale empirical evaluation of 14 shallow/deep classification-based hate speech detectors, evaluated on three large and publicly available hate speech detection benchmarks.[63] a study on detecting hate speech on Twitter using deep convolutional neural networks.The study uses machine learning-based classifiers such as Logistic Regression, Random Forest, Naive Bayes, Support Vector Machines, Decision Trees, and K-Nearest Neighbors to identify hate speech-related tweets on Twitter.The study also uses word embedding methods such as LSTM and Bi-LSTM models.The results show that the Deep Convolutional Neural Network architecture performs well in detecting hate speech on Twitter.Furthermore, in [54],the authors discussed detecting hate speech in social media using the LSTM algorithm.Results show that the capability to detect hate speech in online text automatically, high levels of accuracy, recall, and F1 score, and the potential to counteract hate speech on social media platforms are all features of this technology.In [55],BERT and Hate Speech Word Embedding with Deep Model detect hate speech in text data.The paper describes feature and classifier approaches, reviews recent investigations, and includes datasets, embedding models, and experiment results.It successfully detects hate speech in English text data.[66]examines fuzzy categorization CNN-LSTM and Random Forest deep learning models.The research discusses Twitter hate speech detection issues and model efficacy.The discovery could be used in pattern recognition, machine learning, and AI beyond Twitter.[67]the research proposes an attentional multi-channel convolutional stacking Bidirectional LSTM network that uses word representation approaches to capture semantic relations at multiple windows.The model is compared to five state-of-the-art and five baseline models on three Twitter benchmark datasets.The presented model outperforms the others in most circumstances, and the absence of channels and attention mechanisms has the most significant influence, as proven in Figure 3.An online social network hate speech detection methodology can be utilized for data-driven cyber security.

III.
RESULTS AND DISCUSSION Hate speech detection on social media is a big concern in machine learning [68], with numerous researchers actively addressing hate speech originating from various sources [29].CNN is considered one of the most effective algorithms for detecting hate speech, as it utilizes several methodologies [69], algorithms, and techniques [70].Table 1 of this research presents an overview of 15 studies focusing on detecting hate speech using (CNN) and (GRU) and Table 2 gives an overview of 15 studies focusing on detecting hate speech using (CNN) and (LSTM).Moreover, the reviewed papers have been updated and published within the last five years.The CNN-based models achieve an accuracy ranging from 90% and higher in eighteen of the thirty works of literature-each of the previous research papers utilized separate datasets comprising several Twitter comments.The findings indicate that the quantity and nature of datasets have minimal impact on the precision of CNN-based models.In the end, the review articles prove that machine learning models are also used for detecting hate speech texts.Still, the accuracy and reliability of deep learning models such as CNN, LSTM, and GRU are higher among other algorithms.Half the researchers found that models designed for a single dataset may make mistakes.Another gap is that the models cannot detect other languages because they are trained on English tweets only.

IV. CONCLUSION
The application of techniques for "deep learning" to identify hate speech on Twitter is an exciting strategy for reducing the negative impacts of cyberbullying.Since language on social media constantly changes, rule-based approaches could be more effective; however, deep learning models are ideally adapted to this challenge.These models employ natural language processing and deep neural networks to analyze hate speech, and they show promise in picking up on nuanced contextual differences and shifting patterns of intolerance.However, there are still obstacles to overcome, such as the requirement for extensive and varied labeled datasets, the possibility of biased training data, and the persistent development of language and online communication.Fine-tuning algorithms to reduce false positives is also essential because it is difficult to compromise between suppressing hate speech and protecting free speech.Incorporating deep learning models into content moderation systems on platforms like Twitter can make the internet safer as technology and research in this area evolve.Researchers, platform developers, and communities must continue to work together to improve models, combat biases, and cultivate a more welcoming digital space.This research reviews fifteen studies on detecting hate speech on Twitter using deep learning methods.Different models and other datasets were used, and the results show us that CNN is the most accurate, reliable, and easier to detect comments and then classify as offensive, sexual, racist…etc.[

Fig 1 .
Fig 1. Classification of deep learning techniques, https://www.researchgate.net A. Convolutional Neural Networks (CNN) and Gated Recurrent Units (GRU) for Detecting Hate Speech:Due to its potentially devastating effects on society, identifying instances of hate speech in digital communications is a significant difficulty[29].One viable strategy for this problem is to use cutting-edge technologies like (CNNs).Combining (CNNs) and (GRUs) offers a powerful technique for detecting hate speech, harnessing the benefits of both architectures[14].While CNNs are great at comprehending short-term dependencies in sequences, (GRUs) excel at understanding long-term

Table 1 .
Overview of the literature on detecting hate speech Tweets based on CNN and GRU.