PaperlayerView exact revision on arXiv

Source-linked AI summary

MOMENTA: A Multimodal Framework for Detecting Harmful Memes and Their Targets

Shraman Pramanick, Shivam Sharma, Dimitar Dimitrov, Md Shad Akhtar, Preslav Nakov, Tanmoy Chakraborty

arXiv:2109.05184v2cs.MMcs.CL

TL;DR

The paper addresses the limited study of general harmfulness in memes, where multimodal and contextual cues complicate detection and target identification. It extends HarMeme with COVID-19 and US-politics datasets and proposes MOMENTA, which analyzes local and global image-text perspectives against background context. MOMENTA outperforms ten baselines on both tasks and is reported as interpretable and generalizable.

  • Problem

    General harmfulness in memes has received less attention than specific phenomena, while multimodal and camouflaged semantics make harmfulness and targets difficult to analyze.

  • Method

    MOMENTA analyzes local and global perspectives of meme images and text and relates them to background context for harmfulness detection and target identification.

  • Results

    MOMENTA outperforms ten baselines on both tasks, with accuracy gains of 1.3–2.6 absolute points, and is reported as transferable and interpretable.

  • Takeaways & Limitations

    The framework supports harmful-meme detection and target identification across the Harm-C and Harm-P datasets while providing reported transferability and interpretability.

  • Takeaways & Limitations

    The approach can fail when harmfulness depends on entrenched whole-meme semantics absent from explicit visual gestures, harmful words, or detected attributes.

Abstract

from arXiv · show

Internet memes have become powerful means to transmit political, psychological, and socio-cultural ideas. Although memes are typically humorous, recent days have witnessed an escalation of harmful memes used for trolling, cyberbullying, and abuse. Detecting such memes is challenging as they can be highly satirical and cryptic. Moreover, while previous work has focused on specific aspects of memes such as hate speech and propaganda, there has been little work on harm in general. Here, we aim to bridge this gap. We focus on two tasks: (i)detecting harmful memes, and (ii)identifying the social entities they target. We further extend a recently released HarMeme dataset, which covered COVID-19, with additional memes and a new topic: US politics. To solve these tasks, we propose MOMENTA (MultimOdal framework for detecting harmful MemEs aNd Their tArgets), a novel multimodal deep neural network that uses global and local perspectives to detect harmful memes. MOMENTA systematically analyzes the local and the global perspective of the input meme (in both modalities) and relates it to the background context. MOMENTA is interpretable and generalizable, and our experiments show that it outperforms several strong rivaling approaches.

1 Introduction

Memes can become harmful through the interaction of image, text, and background context, creating challenges beyond conventional hate or offensive-content detection. The paper extends the HarMeme data and proposes MOMENTA for harmfulness and target identification, reporting stronger performance and interpretability.

  • Memes combine images with embedded short text and can spread rapidly through social media.
  • Image-text interactions and background context can make seemingly harmless memes harmful, even when individual modalities are not harmful.
  • Multimodal meme analysis is difficult because memes may be noisy, semantically cryptic, and difficult to process with standard OCR.
  • The study extends HarMeme with additional examples and US-politics memes, producing the Harm-C and Harm-P datasets.
  • MOMENTA analyzes local and global perspectives in both modalities and relates them to background context to detect subtle harmful elements.
  • 1.3–2.6 points absolute: MOMENTA outperforms ten baselines in accuracy on both tasks.
  • MOMENTA is reported to be generalizable and interpretable.

2 Related Work

Prior meme research addressed emotions, sarcasm, hate, offensiveness, propaganda, and related multimodal tasks, but not harmfulness in its broader sense. This work builds on the earlier HarMeme formulation and extends its topic coverage while introducing a multimodal framework.

  • Existing meme studies examined emotion, sarcasm, hate, offensiveness, propaganda, and related specialized phenomena.
  • Several prior systems combined textual, visual, acoustic, or multimodal signals for deception, misinformation, and meme-related classification.
  • Earlier studies did not address the broader concept of harmful memes per se.
  • The authors’ previous work defined harmful memes, formulated harmfulness and target-identification tasks, and created the HarMeme dataset.
  • Because HarMeme covered only COVID-19, the present study adds examples and a US-politics topic while proposing a new multimodal framework.
  • Self-supervised and multimodal pre-training, including CLIP, provide related approaches for transferring information across image and text.

3 Defining Harmful Meme

A harmful meme is defined as an image-text unit with potential to harm an individual, organization, community, or society. Harm is broader than hate or offensiveness and may be concealed, context-dependent, and multimodal.

  • A harmful meme is an image with embedded text that has the potential to cause harm to an individual, organization, community, or society.
  • The definition includes mental abuse, defamation, psycho-physiological injury, socio-economic and proprietary damage, and emotional disturbance.
  • Offensive and hateful memes are harmful, but harmful memes need not be offensive or hateful.
  • Harmful content may be camouflaged and require critical judgment to detect.
  • Harmful memes can target individuals, organizations, or communities rather than only the narrower entity categories associated with hateful memes.

4 Data

The study constructs two topic-specific meme datasets, Harm-C and Harm-P, through web collection, deduplication, OCR, and expert annotation. Annotators label harm presence and intensity together with the targeted social entity.

  • Data Collection and Deduplication: Harm-C contains COVID-19 memes, while Harm-P contains US-politics memes.
  • Data Collection and Deduplication: Researchers collected potentially harmful memes through keyword-based web searches and public meme pages, while adding non-harmful examples using the same keywords.
  • Data Collection and Deduplication: The final datasets contain 3,544 Harm-C memes and 3,552 Harm-P memes after sequential deduplication, with text extracted using Google Cloud Vision API.
  • Annotation: Annotators labeled harm presence and intensity together with the meme’s target.
  • Annotation: Target categories are Individual, Organization, Community, and Society.
  • Annotation: Fifteen NLP experts or linguists performed annotation after guideline review, discussion sessions, a dry run, final annotation, and consolidation.
  • Annotation: Cohen’s κ for harmfulness/target was 0.683/0.782 on Harm-C and 0.675/0.790 on Harm-P.

5 MOMENTA: Our Proposed System

MOMENTA combines global and local visual-linguistic representations with background context to detect harmful memes and identify their targets.

  • 5 MOMENTA: Our Proposed System: MOMENTA extracts meme text, encodes text–image pairs with CLIP, and detects faces, foreground objects, and image attributes.Object and attribute representations provide high-level entities and topics alongside global meme semantics.
  • 5.3 Intra-Modality Attention: Self-attention emphasizes relevant object proposals and image attributes before intra-modality fusion with global CLIP features.This combines local image descriptions with global meme semantics in both visual and textual streams.
  • 5.4 Cross-Modality Attention Fusion: CMAF uses modality attention scores to weight and concatenate the resulting visual and textual features into a final multimodal representation.Residual connections are added for better gradient flow.
  • 5.5 Multi-Task Learning: The final multimodal representation feeds two parallel fully connected branches, one for harmfulness detection and one for target identification.Target-identification loss is applied only to partially harmful or very harmful memes.
  • 5.5 Multi-Task Learning: Focal loss down-weights easy examples and focuses training on hard examples under class imbalance.The model is trained in a multi-task setup for the two related objectives.

6 Experiments

Experiments evaluate MOMENTA and multiple unimodal and multimodal baselines on two tasks using accuracy, Macro-F1, and MMAE.

  • 6 Experiments: Experiments compare MOMENTA with state-of-the-art unimodal textual, unimodal visual, and multimodal models on Harm-C and Harm-P.The comparison includes BERT, several ImageNet-pretrained visual models, late fusion, concatenation, MMBT, ViLBERT, and Visual BERT.
  • 6 Experiments: Accuracy, Macro-F1, and Macro-Averaged Mean Absolute Error evaluate performance, with higher values preferred for the first two and lower values for MMAE.These metrics cover classification quality and error magnitude.
  • 6 Experiments: Table 2 reports performance on harmful meme detection and target identification, merging partially harmful and very harmful classes for two-class evaluation.The table also includes human test performance and MOMENTA’s improvement over the best baseline.

7 Experimental Results

MOMENTA outperforms strong unimodal and multimodal baselines across harmful-meme detection and target identification, while ablations and transfer experiments support the value of its components and generalizability. Human evaluation shows MOMENTA remains below human performance, especially for target identification, and error analysis identifies failures on implicitly harmful memes.

  • Harmful Meme Detection: MOMENTA improves binary harmful-meme detection accuracy over the best models by 2.46 points on Harm-C and 2.59 points on Harm-P.Corresponding Macro-F1 scores improve by similar margins.
  • Harmful Meme Detection: MOMENTA improves 3-class harmful-meme detection accuracy over the corresponding best models by 1.39 points on Harm-C and 2.48 points on Harm-P.Multimodally pretrained early-fusion models outperform unimodal and simple multimodal models.
  • Target Identification: MOMENTA outperforms the best target-identification models on Harm-C by 2.14 accuracy points and 3.88 F1 points, and on Harm-P by 1.26 accuracy points and 1.44 F1 points.Multimodal cues with multimodal pretraining yield sizable improvements over unimodal models.
  • Human Evaluation: MOMENTA is the best-performing model in manual evaluations but remains 4.5–12 accuracy points behind humans, with a larger gap for target identification.The authors identify target identification as particularly difficult and requiring further improvement.
  • Ablation Study: Adding proposals and attributes improves accuracy by 2.3–4.1 points, while adding both improves accuracy over CLIP by 5.5–10 points and F1 by 3.2–10.5 points.Replacing CMAF with simple concatenation reduces accuracy by 1–3.8 points and F1 by 1.5–4.8 points.
  • Transferability of MOMENTA: When trained on one dataset and tested on another, MOMENTA improves F1 by 2.2–6.6 points for 2-class detection, 1.1–9.0 for 3-class detection, and 0.9–4.2 for target identification.The paper attributes superior transferability partly to CLIP’s pretraining on 400M image–text pairs.
  • Interpretability: LIME explanations show MOMENTA relying on Joe Biden’s face, a mannequin, the word CAMPAIGNING, and their conflicting gesture in one correctly classified harmful meme.Fine-grained face detection and CLIP help identify the subtle harmful element despite partial occlusion.
  • Error Analysis: MOMENTA fails on a very harmful meme whose image and text lack explicit harmful cues, while detected attributes fail to model its entrenched semantics.The explanation’s super-pixels are randomly distributed, indicating difficulty with implicit contextual harm.

8 Conclusion and Future Work

The paper introduces Harm-C and Harm-P for detecting harmful memes and their targets, and proposes MOMENTA to analyze local and global multimodal perspectives with background context. Experiments show MOMENTA outperforms ten baselines, while future work will broaden the datasets across domains and languages.

  • Harm-C and Harm-P are large-scale datasets for detecting harmful memes and their targets.
  • MOMENTA analyzes local and global perspectives of memes in both modalities and relates them to background context.
  • MOMENTA outperforms ten baselines on both harmful-meme detection and target identification.
  • Future work will extend the datasets with more domains and languages.

Ethics and Broader Impact

The paper addresses reproducibility, privacy, annotation practices, dataset bias, misuse, intended use, and environmental impact. It notes that harmfulness labels can be subjective and that responsible use requires awareness of bias and human moderation.

  • Reproducibility: The source code and Harm-C and Harm-P datasets are available, with detailed hyperparameter configurations provided in the appendix.
  • User Privacy: The datasets contain memes without user information, collected from publicly available pages, and distribute links rather than the meme files.
  • Annotation: Annotators were NLP experts or linguists, received standard local pay, and were trained through guidelines and discussion sessions.
  • Biases: Dataset labels may contain biases because determining whether a meme is harmful can be subjective, despite explicit annotation definitions and high inter-annotator agreement.
  • Misuse Potential: The dataset could be misused to unfairly moderate memes based on demographic or textual biases, so human moderation is required.
  • Intended Use: The dataset is released for research purposes only and without a commercial-use license.
  • Environmental Impact: Large-scale Transformer training requires substantial computation, although these models are fine-tuned on relatively small datasets and can run inference on CPUs.

A Implementation Details and Hyperparameter Values

The experiments use PyTorch models trained on an NVIDIA Tesla V100 GPU with Adam optimization and negative log-likelihood loss. Dataset statistics include label imbalance across the tasks and datasets.

  • All experimental models are trained with PyTorch on an NVIDIA Tesla V100 GPU with 32 GB memory, CUDA-11.2, and cuDNN-8.1.1.
  • Unimodal models use pretrained Torchvision weights, while remaining weights are initialized from a zero-mean Gaussian distribution with standard deviation 0.02.
  • The datasets exhibit label imbalance for harmfulness intensity and target identification.
  • All models use the Adam optimizer and negative log-likelihood loss during training.

B Data Filtering

The paper defines annotation-oriented filtering criteria, harmfulness and target categories, and the procedures used to annotate and consolidate the datasets. The datasets retain English, readable, multimodal memes while excluding several difficult or unsuitable examples.

  • Data Filtering: Included memes must contain English, readable text alongside an image, and no cartoons.
  • Data Filtering: The filtering process rejects non-English, blurry-text, cartoon, text-only, and image-only memes.
  • Harm and Targets: Harm may involve abuse, defamation, injury, damage, disturbance, mocking, or ridicule directed at an individual, organization, community, or society.
  • Harm and Targets: Target categories are individual, organization, community, and society.
  • Harm and Targets: Harmful memes may be offensive, hateful, biased, implicit, sarcastic, or satirical, and may vary in harm intensity or target multiple entities.
  • Annotation Procedure: Annotation used PyBossa interfaces, three annotators per meme, training sessions, and majority voting to resolve disagreements.
  • Dataset Statistics: Final datasets report statistics on source and label distributions, including their top-five class-specific words with TF.IDF scores.

D Lexical Statistics About the Datasets

The datasets’ lexical statistics show prominent politician- and COVID-19-related words among harmful memes, while meme-text lengths are similar across classes. Harmless memes involving these entities were included to reduce potential textual bias.

  • Politician and COVID-19-related words are prominent among very harmful and partially harmful memes.Frequent target terms include Trump, Joe, Obama, Republican, Wuhan, China, and Islam.
  • Harmless memes related to the same individuals, groups, and entities were intentionally included to alleviate potential text-based bias.
  • Meme-text length distributions show no major differences between classes across the two datasets.
Loading 2109.05184v2…