PaperlayerView exact revision on arXiv

Source-linked AI summary

A Multimodal Framework for the Detection of Hateful Memes

Phillip Lippe, Nithin Holla, Shantanu Chandra, Santhosh Rajamanickam, Georgios Antoniou, Ekaterina Shutova, Helen Yannakoudakis

arXiv:2012.12871v2cs.CLcs.AI

TL;DR

Hateful memes require systems to interpret image and text together, while existing multimodal approaches perform poorly on the task. This paper develops a multimodal framework using UNITER-based models, confounder upsampling, and cross-validation ensembles. Its best approach achieved an AUROC of 80.53 and ranked fourth in Phase 2 of the 2020 Hateful Memes Challenge.

  • Problem

    Hateful-meme detection is difficult because memes convey meaning through interacting image and text, and existing multimodal systems perform poorly.

    Source: §1 · p. 2

  • Method

    The framework evaluates multimodal models and improves UNITER with confounder upsampling and cross-validation ensembles.

    Source: §1 · p. 2, §5.2 · p. 6, §5.3 · p. 6

  • Results

    An AUROC of 80.53 placed the UNITER-based ensemble fourth in Phase 2 of the Hateful Memes Challenge.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The authors identify confounder use and cross-validation ensembling as effective techniques in their hateful-meme framework.

    Source: §8 · p. 11

    The authors identify sub-par image understanding, largely due to limited world knowledge in current image feature extractors, as a constraint on multimodal reasoning.

    Source: §8 · p. 11

Abstract

from arXiv · show

An increasingly common expression of online hate speech is multimodal in nature and comes in the form of memes. Designing systems to automatically detect hateful content is of paramount importance if we are to mitigate its undesirable effects on the society at large. The detection of multimodal hate speech is an intrinsically difficult and open problem: memes convey a message using both images and text and, hence, require multimodal reasoning and joint visual and language understanding. In this work, we seek to advance this line of research and develop a multimodal framework for the detection of hateful memes. We improve the performance of existing multimodal approaches beyond simple fine-tuning and, among others, show the effectiveness of upsampling of contrastive examples to encourage multimodality and ensemble learning based on cross-validation to improve robustness. We furthermore analyze model misclassifications and discuss a number of hypothesis-driven augmentations and their effects on performance, presenting important implications for future research in the field. Our best approach comprises an ensemble of UNITER-based models and achieves an AUROC score of 80.53, placing us 4th on phase 2 of the 2020 Hateful Memes Challenge organized by Facebook.

1 Introduction

Hateful memes combine image and text, making detection a difficult multimodal reasoning problem. The paper develops and evaluates multimodal approaches, reporting a UNITER-based ensemble that ranked fourth in Phase 2 of the Hateful Memes Challenge.

1 Introduction

  • Hateful memes require joint reasoning over image and text because either modality alone may be insufficient to interpret a meme.Existing multimodal systems also performed rather poorly on hateful-meme detection.Source: §1 · p. 2
  • The challenge frames detection as binary classification on a dataset of over 10,000 memes, including benign confounders for some hateful examples.These examples make unimodal classification particularly challenging.Source: §1 · p. 2
  • The UNITER-based ensemble achieved an AUROC of 80.53 and ranked fourth in Phase 2 of the Hateful Memes Challenge.The authors also analyze model errors and test hypothesis-driven augmentations.Source: §1 · p. 2
  • The paper evaluates early-fusion LXMERT, UNITER, and Oscar, and investigates confounder upsampling and cross-validation ensembles beyond simple fine-tuning.The authors also analyze misclassifications and test hypothesis-driven model augmentations.Source: §1 · p. 2

2 Related work

Multimodal learning jointly processes visual and language signals, with early-fusion models representing one approach. In hate-speech detection, multimodal resources and methods remain comparatively limited, while challenge baselines favor multimodal systems.

2.1 Multimodal learning

  • Early-fusion models process image and text jointly, whereas late-fusion systems process each modality separately before combining their features.UNITER, LXMERT, and Oscar are identified as early-fusion models used in this work.Source: §2.1 · p. 3
  • Oscar outperformed LXMERT and UNITER on VQA, GQA, and NLVR2, achieving state-of-the-art performance on VQA and NLVR2.Source: §2.1 · p. 3

2.2 Multimodal hate speech detection

  • Hate-speech research has relied largely on text features, with comparatively little vision or multimodal work attributed to scarce annotated datasets.On MMHS150K, the evaluated multimodal models did not outperform unimodal counterparts.Source: §2.2 · p. 3
  • Hateful Memes Challenge baselines found multimodal systems outperformed unimodal systems, supporting joint processing of meme image and text.The strongest multimodal baselines described here are ViLBERT pretrained on Conceptual Captions and VisualBERT pretrained on COCO.Source: §2.2 · p. 3

3 Dataset

The Hateful Memes dataset pairs meme images with separately provided text and labels each meme hateful or non-hateful. It includes contrastive benign examples and multiple competition splits, including adversarial examples in the unseen development and test sets.

3 Dataset

  • Benign confounders are minimal image or text transformations that flip a meme’s hateful or non-hateful label.Image confounders retain the image but change the text; text confounders retain the text but change the image.Source: §3 · pp. 3–4
  • The dataset contains 12,140 memes, including 8,500 training examples, and provides development and test sets for both challenge phases.The training set is 36% hateful and 64% non-hateful; Phase 1 seen sets are class-balanced.Source: §3 · p. 4
  • The dataset uses five meme types, and Phase 1 seen sets contain 40% multimodal hate, 10% unimodal hate, 20% benign text, 20% benign image confounders, and 10% random non-hateful examples.Table 1 presents the distribution of meme types in the dataset.Source: §3 · p. 4, Table 1 · p. 4
  • The unseen development set adds 140 dynamic adversarial benign confounders, while the unseen test set totals 2,000 examples and includes new adversarial confounders.The adversarial examples are selected based on VisualBERT failures.Source: §3 · p. 4

4 Models

The paper focuses on three early-fusion pretrained models: LXMERT, UNITER, and Oscar. Their architectures differ in how they encode and combine image and text information.

4 Models

  • The paper selects early-fusion LXMERT, UNITER, and Oscar because early-fusion architectures were shown to outperform late-fusion ones on the task.Source: §4 · p. 4

4.1 LXMERT

  • LXMERT processes image and text with independent encoders, then combines their representations using cross-attention modules.It was pretrained on COCO, Visual Genome, and image-question-answering datasets.Source: §4.1 · p. 4

4.2 UNITER

  • UNITER applies self-attention to joint image-text input, encoding bounding-box visual features and word tokens into a common embedding space.Its pretraining includes masked language/region modeling, image-text matching, and word-region alignment across four visual-language datasets.Source: §4.2 · pp. 4–5

4.3 Oscar

  • Oscar uses detected object tags alongside word sequences and image-region features, with tags serving as anchors for semantic alignment.Its pretraining uses masked-token and contrastive objectives.Source: §4.3 · p. 5

5 Approach

The framework fine-tunes multimodal models on text and image features, then tests strategies for improving learning and robustness. Its design includes confounder-focused training, cross-validation ensembling, margin ranking, and optional object tags.

5.1 Image feature extraction and base model selection

  • The models use Faster R-CNN image features from their corresponding pretrained checkpoints and BERT-tokenized text, then fine-tune on hateful-meme labels.Using the corresponding checkpoints keeps image features in the latent space used to pretrain the multimodal models.Source: §5.1 · p. 5
  • UNITER outperformed LXMERT and Oscar for both base and large versions; larger models generalized poorly, so subsequent experiments used UNITER-base.Additional HM-domain MLM, ITM, and MRFR pretraining did not improve performance, so the authors used supervised fine-tuning alone.Source: §5.1 · pp. 5–6

5.2 Confounder upsampling and loss re-weighting

  • CFU upsamples text confounders during UNITER fine-tuning, while loss re-weighting assigns greater weight to hateful examples.The approach targets poor performance on text confounders; the training set is 36% hateful and 64% non-hateful.Source: §5.2 · p. 6

5.3 Cross-validation ensemble

  • The CV ensemble combines UNITER predictions with EA-optimized weights, using development-set examples in augmented folds while keeping confounder pairs together.Half of the development set augments each training fold and the other half its test fold; optimization uses the augmented test folds.Source: §5.3 · p. 6, Figure 2 · p. 7

5.4 Training with margin ranking loss

  • MRL trains on pairs of memes, using margin ranking loss alongside weighted BCE to favor the hateful-labeled meme’s score.At testing, evaluation returns to the standard setting without instance pairing.Source: §5.4 · p. 7

5.5 Fine-grained YOLO9000 object tags

  • The YOLO variant adds predicted tags from a selected set of 97 YOLO9000 classes alongside meme text and bounding-box features.The proposed tags are intended to help identify target groups in the image and meme text.Source: §5.5 · p. 7

6 Experiments and Results

Experiments compare UNITER-based variants on the development and challenge test sets, culminating in a fourth-place Phase 2 system. The strongest result comes from an ensemble incorporating 15-fold cross-validation and dev-seen training data.

6 Experiments and Results

  • Training used 3× text-confounder upsampling, 1.8× hateful-class loss weighting, and a 15-fold ensemble whose weights were optimized for development-set AUROC.Source: §6 · p. 8
  • Simple UNITER fine-tuning outperformed the strongest multimodal baselines, ViLBERT CC and VisualBERT COCO; LXMERT scored below VisualBERT and UNITER, and Oscar scored below UNITER.Source: §6 · p. 8
  • Cross-validation slightly raised dev-seen performance over fine-tuning; confounder upsampling and hateful-class loss reweighting further improved development results.The 15-fold variant also improved Phase 1 performance over UNITER.Source: §6 · pp. 8–9
  • 30-fold cross-validation further raised dev-seen performance but not Phase 1 performance; margin ranking loss and YOLO9000 tags also failed to improve Phase 1.Source: §6 · p. 9
  • 80.53 AUROC earned UNITERFINAL 4th place on the Phase 2 leaderboard.Table 2 reports AUROC scores across dev-seen and the Phase 1 and 2 test sets.Source: Table 2 · p. 8, §6 · p. 9

7 Error analysis and further model variants

The error analysis finds that both false negatives and false positives often involve multimodal interpretation. Experiments with target-group information, emotion detection, and modality-specific dropout did not improve performance.

7.1 Analyzing misclassifications

  • False negatives involved hateful image-text combinations and missed groups, people, or symbols; false positives were also mainly multimodal and could change with different text or images.Source: §7.1 · p. 9, Figure 3 · p. 10

7.2 Capturing target group information

  • SBIC-based offensiveness or target-group multitask training and RoBERTa-generated target tags did not improve dev-seen over UNITER fine-tuning.The authors suggest noisy generated labels and text alone being insufficient for target identification may explain the result.Source: §7.2 · pp. 9–10

7.3 Emotion detection

  • GoEmotions emotion-detection multitask training produced no improvement on dev-seen.Source: §7.3 · p. 10

7.4 Overfitting text features

  • Because the model struggled with text confounders, the authors split attention into modality-specific blocks and varied dropout by block, but results did not improve.Source: §7.4 · pp. 10–11

8 Conclusion

The proposed multimodal framework placed fourth in Phase 2 of the 2020 Hateful Memes Challenge. The authors highlight its training and ensembling techniques while noting substantial remaining limits in true multimodal reasoning.

8 Conclusion

  • The proposed framework ranked 4th in Phase 2 of the 2020 Hateful Memes Challenge.Source: §8 · p. 11
  • The framework uses multimodal confounders, cross-validation with evolutionary ensemble weighting, minority-class loss reweighting, and image features from the pretrained detector checkpoint.Source: §8 · p. 11
  • The authors see substantial room to improve true multimodal reasoning, citing image extractors' limited world knowledge and the importance of recognizing target groups, public figures, and symbols.Source: §8 · p. 11

A.1 YOLO9000 relevant classes

The section identifies 97 YOLO9000 classes used in the experiments.

A.1 YOLO9000 relevant classes

Related papers

Loading 2012.12871v2…