Source-linked AI summary
Hate Speech in Pixels: Detection of Offensive Memes towards Automatic Moderation
Benet Oriol Sabat, Cristian Canton Ferrer, Xavier Giro-i-Nieto
TL;DR
Automating hate-speech detection in memes matters because social networks’ scale requires moderation support, yet figurative and multimodal content remains challenging. This work evaluates image and text representations separately and together on a dataset of 5,020 memes. The multimodal model achieves a best accuracy of 0.83, but visual cues are more informative than linguistic ones and the task remains far from solved.
Problem
Social networks’ scale requires some moderation automation, while borderline or figurative content and the multimodal meaning of memes challenge hate-speech detection.
Source: §1 · p. 1Method
Results
The best accuracy is 0.83, and visual cues are more informative than linguistic ones; multimodal results are best but improve only slightly over vision alone.
Source: §6 · p. 4Takeaways & Limitations
The system may filter some memes on social networks, but many would still require a human moderator.
Source: §6 · p. 4An effective solution should probably include information beyond the meme itself, such as the societal context in which it is posted.
Source: §6 · p. 4
Abstract
from arXiv · showhide
This work addresses the challenge of hate speech detection in Internet memes, and attempts using visual information to automatically detect hate speech, unlike any previous work of our knowledge. Memes are pixel-based multimedia documents that contain photos or illustrations together with phrases which, when combined, usually adopt a funny meaning. However, hate memes are also used to spread hate through social networks, so their automatic detection would help reduce their harmful societal impact. Our results indicate that the model can learn to detect some of the memes, but that the task is far from being solved with this simple architecture. While previous work focuses on linguistic hate speech, our experiments indicate how the visual modality can be much more informative for hate speech detection than the linguistic one in memes. In our experiments, we built a dataset of 5,020 memes to train and evaluate a multi-layer perceptron over the visual and language representations, whether independently or fused. The source code and mode and models are available https://github.com/imatge-upc/hate-speech-detection .
1 Motivation
Social networks’ scale makes some automation necessary, but detecting hate in memes is difficult because individually neutral or funny image and text can convey hate together. The work studies visual and linguistic representations separately and fused, finding that multimodal detection benefits from both while remaining unsolved.
1 Motivation
- Social networks’ scale makes some automation necessary for moderating the spread of misinformation and hate messages.Existing moderation includes crowd-sourced abuse reports and dedicated human teams.Source: §1 · p. 1
- Memes are difficult to classify because individually neutral or funny images and text can convey hate when combined.The paper describes memes as images with embedded captions and notes their use to disseminate hate as dark humour.Source: §1 · p. 1
- The study compares visual and linguistic representations separately and fused, finding that both modalities help while the task remains far from solved.The authors identify high abstraction in meme messages as a continuing challenge.Source: §1 · p. 1
2 Related Work
Prior hate-speech detection work has largely focused on language, while the paper identifies combining language and visual content in memes as an unaddressed task. Its technical approach draws inspiration from multimodal gang-violence prediction.
2 Related Work
- Most hate-speech detection work focuses on language, often using embeddings followed by a binary classifier.Common representations include bag-of-words and N-gram features, sometimes with expert-selected keywords.Source: §2 · p. 2
- The authors identify no prior work combining language and visual content to detect hate speech in memes.Source: §2 · p. 2
- The technical approach is inspired by multimodal gang-violence prediction that fused pretrained image and text features for classification.Source: §2 · p. 2
3 Model
The model extracts text and image representations from a meme, concatenates them, and feeds the resulting features to an MLP that predicts a hate score.
3 Model
- OCR extracts meme text, whose BERT subword embeddings are averaged into a sentence representation.The selected multilingual BERT model has 12 layers and 768 hidden dimensions.Source: §3 · p. 2
- A pretrained ImageNet VGG-16 encodes each image using the 4,096-dimensional activations from its last hidden layer.Source: §3 · p. 2
- Concatenating the text and image encodings yields 4,864 features for an MLP with two 100-neuron ReLU hidden layers and a final hate-score output.Source: §3 · p. 2
4 Dataset
The dataset contains 5,020 memes weakly labeled by source, with a random train-validation split and no separate test subset because the dataset was small.
4 Dataset
- The dataset contains 5,020 memes weakly labeled as hate or non-hate according to their source.Hate memes were collected from Google Images; non-hate memes came from the Reddit Memes Dataset.Source: §4 · pp. 2–3
- The dataset comprised 1,695 hate memes (643 racist, 551 Jewish, and 501 Muslim) and 3,325 Reddit Memes Dataset memes, assumed non-hate because average Reddit memes were considered not to contain hate messages.Source: §4 · p. 3
- A random class-balanced split allocated 4,266 memes to training and 754 to validation, with no third subset because of limited data.The authors relied on validation metrics.Source: §4 · p. 3
5 Experiments
Across language-only, vision-only, and multimodal experiments, the multimodal configuration performed best, though vision-only was close and several dataset and model factors complicate that comparison.
5 Experiments
- The experiments compare language-only, vision-only, and multimodal detection, training with MSE loss but assessing binary accuracy.Figure 2’s horizontal axis is batches for training loss and epochs for validation loss and accuracy.Source: §5 · p. 3, Figure 2 · p. 3
- 0.66: all three configurations exceeded the non-hate dummy baseline; multimodal led vision-only, while language-only lagged.Figure 2 shows train loss, validation loss, and validation average binary accuracy across the three configurations.Source: §5 · p. 3, Figure 2 · p. 3
- 0.81 Average Precision was obtained for the best multimodal model amid class imbalance.The paper also compares Max. Accuracy and Smth Max. Accuracy, with the latter estimated from a smoothed validation curve because accuracy fluctuated substantially between epochs.Source: §5 · p. 3
- Vision’s 4,096-dimensional representation, versus 768 linguistic dimensions, and differing MLP parameter counts may partly explain the vision-only advantage.The authors note that the comparison did not control for these model-capacity differences.Source: §5 · pp. 3–4
- The dataset may favor visual style cues: modern-format memes were more common among non-hate examples, while classic formats were more common among hate examples.The authors also report possible false positives in the hate class and false negatives in the Reddit non-hate set; compressed images may impair OCR and language encoding.Source: §5 · p. 4, Figure 3 · p. 4
6 Conclusions
The study finds meme hate detection feasible with image and text encoders, but far from solved. Visual cues mattered more than linguistic cues, and the best multimodal gain over vision alone was small.
6 Conclusions
- 0.83 best accuracy: the authors describe it as modest, though much better than always predicting the most frequent class.The system may filter some memes, but many would still require human moderation.Source: §6 · p. 4
- The proposed detector may filter some social-network memes, but human moderators would still be needed for many.Source: §6 · p. 4
- Misuse could score many text-image pairs to find novel combinations with high hate scores, helping create hate memes.Source: §6 · p. 4
- Visual cues were more important than linguistic cues, while the multimodal approach’s gain over vision-only was small.The authors say deployment should weigh the accuracy gain against the cost of OCR and encoding extracted text.Source: §6 · p. 4
- Effective meme detection should probably consider information beyond the meme itself, including the societal context in which it is posted.Source: §6 · p. 4