An Explainable DistilBERT-BiLSTM-Attention Framework for Binary and Multi-Class Hate Speech Detection

1.2 Contributions

• The framework moves beyond binary classification to multilevel hate speech detection, addressing limited research on severity levels.Source: §1.2 · p. 3, §1.1 · p. 3
• LIME is integrated to make model predictions transparent and interpretable, addressing limited explainability in existing models.Source: §1.2 · p. 3, §1.1 · p. 3
• The hybrid architecture combines DistilBERT embeddings, Bi-LSTM, and attention to capture contextual and sequential information.Source: §1.2 · p. 3, §1.1 · p. 3
• Evaluation on two benchmark datasets responds to concerns raised by studies that use only one dataset.Source: §1.2 · p. 3, §1.1 · p. 3
• The authors report improved accuracy and F1-score compared with standard baseline models.Source: §1.2 · p. 3
• The framework is presented as a practical and efficient content-moderation solution that balances performance with computational efficiency.Source: §1.2 · p. 3

2.1 Hate Speech Detection

• ELECTRA achieved an F1-score of 89.80% on MetaHate in a comparison with BERT and RoBERTa.Source: §2.1 · p. 4
• In multilingual settings, HASTIKA results favored BERT over fastText, while ArabicBERT with BiLSTM and an RBF classifier achieved F1-scores above 89% for OLD and over 95% for HSD.Source: §2.1 · p. 4
• Prior work on multilingual hate-speech detection is described as leaving black-box explainability unaddressed, and the review identifies interpretability as an open gap.Source: §2.1 · pp. 4, 6
• Multilingual studies report XLM-RoBERTa at 95% accuracy for multi-class classification and improved F1-scores from semi-supervised learning with limited labels.Source: §2.1 · p. 4
• Reported approaches include BERT ensembles for English and Korean, RoBERTa-large with XGBoost on Davidson, and a GloVe-based CNN for multi-class detection.Source: §2.1 · p. 5
• Research has shifted toward transformer-based and hybrid models, but combining deep contextual understanding with transparent explanations has received limited attention.Source: §2.1 · pp. 4, 6

3. Dataset Description

• Both datasets are used for binary hate-versus-not-hate classification and multi-class experiments retaining their original label structures.Source: §3 · p. 6

3.1 Davidson dataset

• Davidson contains about 24,800 English tweets collected using hate-related keywords; its three original labels are hate speech, offensive language, and neither.Source: §3.1 · p. 6

3.2 SMHS dataset

• SMHS contains 13,000 samples across six classes, and the study uses its original labels for multi-class classification and grouped labels for binary classification.Source: §3.2 · pp. 6–7

4. Methodology

• The methodology describes a workflow from text preparation through model development and evaluation for binary and multi-class classification.Source: §4 · p. 7

4.1 Preprocessing

• Preprocessing cleans noisy social-media text by normalizing and removing irrelevant elements while aiming to preserve its core meaning.Source: §4.1 · pp. 7–8

4.2 Distill-BERT Embeddings

• DistilBERT produces contextual embeddings, representing each token as a 768-dimensional vector and using the [CLS] output as the tweet representation [29].Source: §4.2 · p. 8

4.3 Bidirectional-LSTM

• A 256-hidden-dimension Bi-LSTM reads the embeddings in both directions to model sequential information and passes its output to attention before classification.Source: §4.3 · p. 8

4.4 Attention Module

• Attention scores are normalized with SoftMax and weight Bi-LSTM outputs to form a context vector emphasizing more relevant tokens.A linear layer computes a scalar score for each time step; the context vector summarizes the input sequence.Source: §4.4 · pp. 8–9

4.5 Classification

• The classifier maps the context vector, with hidden dimension 2, to logits for the label classes and final class probabilities.Source: §4.5 · p. 9

4.6 LIME Integration

• LIME is applied after training and validation to explain individual predictions by highlighting influential words or phrases.The authors describe its use for error analysis and debugging.Source: §4.6 · p. 9

4.7 Hyperparameters Tuning

• The best reported settings included 30 training epochs, batch size 16, and AdamW with learning rate 2 × 10⁻⁵.The study tested 10, 20, and 30 epochs; batch sizes 16, 32, and 64; and maximum sequence lengths 32, 64, and 128.Source: §4.7 · p. 9

5.1.1 Baselines for Davidson dataset

• Davidson comparisons use RoBERTa with XGBoost, CNN followed by LSTM, and a hybrid LSTM-CNN as baselines.The CNN-LSTM baseline models sequential dependencies after capturing local patterns; the LSTM-CNN baseline extracts local features after learning sequential information.Source: §5.1.1 · p. 10

5.1.2 Baseline for SMHS dataset

• For SMHS, the sole available baseline is a fine-tuned DistilBERT embedding model.Source: §5.1.2 · p. 10

• The evaluation uses Accuracy, Precision, Recall, and F1-score as standard classification metrics.Their descriptions and formulas are presented in Table 3.Source: §5.2 · p. 10, Table 3 · p. 10
• Accuracy measures overall prediction correctness, while Precision counts correctly predicted hate samples among predicted hate samples.Source: §5.2 · p. 10
• Recall measures correctly detected hate samples among actual hate samples, while F1-score balances Precision and Recall.Source: §5.2 · p. 10
• TP, TN, FP, and FN distinguish correct positive and negative predictions from the two types of classification error.FP is a positive prediction for an actual negative; FN is a negative prediction for an actual positive.Source: §5.2 · p. 10

6.1 Ablation Study

• 96.78% F1-score on Davidson and 99.9% on SMHS were the best ablation-setting results, with the full model retaining similar parameter count and batch time.The full setting uses SMOTE, one Bi-LSTM layer, attention, and residual concatenation; the ablation summary reports performance gains without noticeable increases in model size or training time.Source: §6.1 · pp. 11–12, Table 4 · p. 11

6.2 Performance Analysis for Binary Classification

• Across 15 epochs, Davidson training F1 rises from 96% to nearly 99%, while validation F1 reaches 96.78%; its confusion matrix records 97 false positives and 150 false negatives.Davidson false negatives often involve indirect, sarcastic, or subtle hate; SMHS has 3,365 true negatives and 3,362 true positives, with 1 false positive and 4 false negatives.Source: §6.2 · pp. 12–13
• 96.44% F1-score on Davidson and 99.78% F1-score on SMHS are reported for binary classification, exceeding the cited dataset baselines.The proposed model’s Davidson accuracy is 96.78%; the SMHS baseline F1-score is 92.76%.Source: §6.2 · pp. 13–14, Table 5 · p. 14

6.3 Performance Analysis for Multiclass Classification

• Multiclass validation F1-scores reach 97% on Davidson and 94.99% on SMHS; reported errors involve overlap among related hate, offensive, and neutral categories.SMHS validation loss settles around 0.13, compared with 0.04 for Davidson; confusion matrices report high true-positive counts across the six SMHS classes.Source: §6.3 · pp. 14–15, Figure 7 · p. 15, Figure 8 · p. 16
• 97% and 94.99% F1-scores on Davidson and SMHS, respectively, surpass the reported multiclass benchmark studies.The proposed model combines DistilBERT embeddings with BiLSTM and attention; the SMHS comparison reports a 90% baseline F1-score.Source: §6.3 · pp. 15–16, Table 6 · p. 16

6.4 Explanations using LIME Model

• LIME examples highlight influential derogatory terms in binary and multiclass predictions, while neutral or structural words contribute little.Examples include “bitch” for Davidson binary hate, “yall” and “hoes” for SMHS binary hate, and “yall,” “niggas,” and “cause” for an SMHS offensive classification.Source: Figure 9 · p. 17, §6.4 · p. 17, Figure 11 · p. 18

7. Conclusion and Future Directions

• Binary F1-scores were 96.78% on Davidson and 99.53% on SMHS; multi-class F1-scores were 97.00% and 94.99%, respectively, outperforming existing baselines in both settings.The reported multi-class values correspond to Davidson and SMHS, respectively.Source: §7 · p. 18
• LIME interpretations highlighted relevant linguistic cues rather than neutral or structural tokens, with semantically meaningful and abusive terms identified as key decision factors.Source: §7 · p. 18
• Future work may extend the framework to code-mixed and multilingual text, images and audio, advanced explainability methods, fine-grained target identification, and contextual intent modeling.Source: §7 · p. 19

Authorship Contributions

• Rameesha Zia is credited with visualization, validation, software, investigation, formal analysis, data curation, and original-draft writing; Muhammad Shahid Iqbal Malik with conceptualization, methodology, supervision, formal analysis, validation, data curation, and writing and editing.Source: Authorship Contributions · p. 19

Funding

• The authors report receiving no funds, grants, or other support.Source: Funding · p. 19

Competing Interest Statement

• The authors state that they have no relevant financial or non-financial interests to disclose.Source: Competing Interest Statement · p. 19