PaperlayerView exact revision on arXiv

Source-linked AI summary

ArGuard Shared Task: Harmful Content Detection in Arabic Memes and LLM Prompts

Firoj Alam, Md. Rafiul Biswas, Mohamed Bayan Kmainasi, Ali Ezzat Shahroor, Hamdy Mubarak, George Mikros, Abul Hasnat, Wajdi Zaghouani

arXiv:2609.29349v1cs.CLcs.AI

TL;DR

Arabic harmful-content detection faces dialectal and cultural challenges, and meme and LLM-prompt evaluation have largely been separate. ArGuard unifies these settings in four binary and fine-grained subtasks; fine-grained meme classification remains especially difficult.

  • Problem

    Arabic harmful-content detection is complicated by dialectal variation, code-switching, indirect intent, and culturally specific references, while meme and LLM-prompt evaluations have largely been separate.

    Source: §1 · pp. 1–2

  • Method

    ArGuard establishes a shared framework with binary and fine-grained meme and prompt tasks, releases annotated datasets and evaluation protocols, and analyzes participating systems.

    Source: §1 · p. 2

  • Results

    Fine-grained meme classification was substantially harder than binary detection: the best macro-F1 was 0.823 on A1 and 0.419 on A2, while prompt systems faced class-prior and source shifts.

    Source: §1 · p. 2

  • Takeaways & Limitations

    For prompt safety under train-test shifts, distribution-aware strategies, external data, or zero-shot LLMs performed robustly.

    Source: §1 · p. 2

    Some AraSafe label mappings were inexact, leading systems to misclassify a small number of test items.

    Source: §3.2 · p. 4

Abstract

from arXiv · show

ArGuard is a shared task on harmful content detection in Arabic memes and LLM prompts. It includes two tracks: Track A focuses on multimodal hate detection in Arabic memes, while Track B addresses harmful prompt detection for Arabic LLM safety evaluation. In total, 58 teams registered, 35 participated in the final evaluation, and 27 submitted system-description papers. Participating teams explored models such as AraBERT, Jais, and Qwen3-VL. The best systems achieved macro-F1 scores of 0.823 on A1, 0.419 on A2, 0.984 on B1, and 0.790 on B2. Fine-grained meme classification in A2 was the most challenging setting, partly due to sparse labels and train-test distribution shifts.

1 Introduction

ArGuard brings Arabic hateful-meme detection and LLM-prompt safety into a shared framework. Its findings show that fine-grained meme classification is especially difficult, amid linguistic and cultural challenges and prompt train-test shifts.

Motivation

  • Arabic detection is challenging across MSA and dialects, where code-switching, Arabizi, indirect intent, and cultural references complicate interpretation.For memes, meaning can depend on the interaction between image and text and on local cultural knowledge.Source: §1 · pp. 1–2
  • Earlier meme resources offered limited insight into attack types and non-hateful sarcasm, humor, or mockery, while Arabic LLM-safety research followed an English-centric field.The introduction positions fine-grained meme annotation and Arabic prompt evaluation as gaps addressed by the shared task.Source: §1 · p. 2

Shared task

  • ArGuard establishes two tracks and four subtasks, bringing Arabic meme and LLM-prompt harmful-content detection into one evaluation framework.Each setting includes binary and fine-grained classification.Source: Figure 1 · p. 1, §1 · p. 2

Findings

  • Fine-grained classification is substantially harder than binary detection, especially for memes, where the best macro-F1 is 0.823 on A1 and 0.419 on A2.Sparse and imbalanced fine-grained labels contribute to the difficulty; prompt safety generalization is affected by shifts in class priors and sources.Source: §1 · p. 2

Contributions

  • The task releases annotated Arabic safety datasets, establishes reproducible evaluation protocols, and analyzes systems, model families, fusion, imbalance, and ensembling.The datasets cover fine-grained meme attack types, non-hateful phenomena, and prompt harm categories across MSA and dialects.Source: §1 · p. 2

2 Related Work

Prior work spans multimodal meme understanding, Arabic harmful-content resources, and LLM prompt safety, but coverage remains limited in non-English settings and Arabic safety evaluation.

Multimodal meme detection

  • Meme research treats hate detection as cross-modal reasoning, yet much of the literature remains English-centric or relies on small non-English datasets.Prior directions include multimodal fusion, external knowledge, visual-to-text prompting, and explanation-based reasoning.Source: §2 · p. 2

Arabic harmful-content research

  • Arabic content moderation progressed from offensive-language and hate-speech tasks to multimodal propaganda and harmful-content resources, including fine-grained AHA-Memes annotations.AHA-Memes provides 5,000 memes with binary hate labels and fine-grained hateful attack and non-hateful subtype labels (Kmainasi et al., 2026b).Source: §2 · pp. 2–3

LLM prompt safety

  • LLM safety benchmarks support evaluation and moderation, but multilingual research remains English-centric; Arabic resources include culturally adapted prompts and dialect coverage.ArabicDialectSafety includes prompts across MSA and five regional dialects and supports binary safety and seven-category classification (Zaghouani et al., 2026).Source: §2 · p. 3

3 Tasks and Datasets

Track A evaluates binary and fine-grained Arabic meme labels using multimodal evidence; Track B classifies Arabic prompts by safety and harm category. The tracks use separate datasets, task pools, and macro-F1 evaluations.

3.1 Track A Multimodal Hateful Meme Understanding

  • A1 classifies memes as Hateful or Not Hateful, while A2 assigns one or more labels from a ten-label taxonomy of hateful attacks and non-hateful subtypes.Both subtasks require textual and visual evidence because hateful meaning may arise from image-text relationships; Figure 2 illustrates A1 and A2 labels.Source: §3.1 · p. 3, Figure 2 · p. 4
  • AHA-Memes contains 5,000 manually annotated Arabic memes paired with images, OCR text, binary hate labels, and fine-grained labels.Trained native Arabic speakers used bilingual guidelines; agreement was κ=0.91 for binary labels, 0.75 for hateful types, and 0.67 for non-hateful subtypes.Source: §3.1 · p. 3
  • The labeled meme splits contain 4,500 examples for final training, with 500 reserved for the blind test; fine-grained labels are more imbalanced than binary labels.Among labeled memes, 37.8% are Hateful; Mocking is the most frequent hateful type and Exclusion the rarest, while Sarcasm and Humor dominate non-hateful subtypes.Source: §3.1 · p. 3, Table 1 · p. 4
  • Both meme subtasks rank systems by macro-F1, averaging across the two binary classes for A1 and ten fine-grained labels for A2.The evaluation also reports accuracy, macro precision and recall, weighted F1, and per-class F1 for analysis.Source: §3.1 · p. 3

3.2 Track B Textual Harmful Prompt Detection

  • B1 classifies prompts as Safe or Unsafe, while B2 assigns one of seven harm categories independently of the B1 safety label.Figure 3 illustrates the binary safety and fine-grained harm labels.Source: §3.2 · pp. 3–4, Figure 3 · p. 5
  • Track B combines ArabicDialectSafety and AraSafe, using ArabicDialectSafety for train and development and both sources for test; B1 and B2 use different task-specific pools.AraSafe granular labels were re-encoded to the ArabicDialectSafety scheme.Source: §3.2 · p. 4, Table 2 · p. 5
  • Some AraSafe label mappings were inexact and misclassified a small number of test items; unsafe prompts dominated train and development, while test safety labels were nearly balanced.ArabicDialectSafety agreement was κ=0.88 for safe/unsafe and 0.81 for harm categories; harm categories were imbalanced, from most frequent Adult Content to least frequent Hate Speech.Source: §3.2 · p. 4
  • Both prompt subtasks use macro-F1 for ranking, averaging F1 across Safe and Unsafe for B1 and across seven harm categories for B2.Accuracy is additionally reported for analysis.Source: §3.2 · p. 5

4 Participating Systems

Participating systems span Arabic encoders, vision-language models, decoder LLMs, and lexical approaches, with different strategies across the meme and prompt tracks.

4 Participating Systems

  • Across 35 teams, submissions spanned Arabic encoders, VLMs, decoder LLMs, and lexical or classical systems, while Track A combined text and visual features or adapted VLMs.Track A systems also used multi-task learning, hierarchical constraints, ensembling, and class-specific threshold tuning for fine-grained classification.Source: §4 · p. 5, §4.1 · p. 5

4.1 Track A

  • The A1 winner achieved 0.823 macro-F1 with a 22-model ensemble, weighting its joint image-and-OCR VLM branch at 0.85 and its text branch at 0.15.The system used only official ArGuard data, without external data or augmentation.Source: §4.1 · p. 5
  • For A2, the top system fine-tuned Qwen3-VL-8B-Instruct with LoRA, jointly trained binary and fine-grained outputs, and averaged predictions from eight models.It used class-weighted binary cross-entropy to address label imbalance.Source: §4.1 · p. 5

4.2 Track B

  • Track B systems primarily used Arabic encoders, often with ensembling or lexical features, alongside Jais, ALLaM, translation-based, dialect-aware, and other approaches.Common strategies included class-weighted or focal losses, multi-seed ensembling, augmentation, and threshold tuning; some systems corrected train-test class-prior shift.Source: §4.2 · pp. 5–6
  • The B1 winner adapted Jais-6.7B with 4-bit QLoRA and a dialect-factored four-class objective, then collapsed predictions to binary labels.It augmented the underrepresented safe-MSA class with human-written prompts from AraSafe.Source: §4.2 · p. 6
  • For B2, the same Jais-6.7B setup used a direct seven-way harm classifier and averaged softmax probabilities from five random-seed models.It used no external data augmentation, stacking, threshold tuning, or post-processing.Source: §4.2 · p. 6

5 Results and Discussion

Fine-grained classification was harder than binary detection across both tracks, especially for memes, and no single model family led every task.

R Team Acc Ma-F1

  • 0.823 macro-F1 versus the 0.678 baseline led A1; all 16 submissions beat baseline, while text-only MARBERTv2 outperformed many fusion systems.Where fusion helped, gains were small, suggesting OCR text carried most of the signal.Source: §5 · p. 7, Table 5 · p. 8

R Team Mi-F1 Ma-F1

  • A2 requires separating hateful Mocking from benign Sarcasm and Humor, a distinction that hinges on who is targeted..Source: R Team Mi-F1 Ma-F1 · p. 8
  • 0.419 macro-F1 and 0.576 micro-F1 were the best A2 scores, amid severe label imbalance and a test set with fewer hateful memes and more non-hateful subtypes.The rare Exclusion class and distribution shift remained challenges, alongside threshold calibration.Source: R Team Mi-F1 Ma-F1 · p. 8, R Team Acc Ma-F1 · p. 8, Table 6 · p. 8

R Team Acc Ma-F1

  • 0.984 macro-F1 was achieved by both AK_QMUL and Lattice AI on binary prompt safety using different model strategies.The task also had a class-prior shift: unsafe prompts were about 80% of train and development data but 49% of test data.Source: R Team Acc Ma-F1 · pp. 8–9, Table 7 · p. 8
  • 0.790 macro-F1 was the best B2 score versus 0.687 for the baseline, with only 0.002 separating the top two systems.The paper notes that further gains may require better discrimination between closely related harm categories.Source: Table 8 · p. 8, R Team Acc Ma-F1 · p. 9

5 Results and Discussion

  • Fine-grained classification was consistently harder than binary detection, with the largest gap for memes and no model family dominating across tasks.Multimodal and VLM-based ensembles led the meme track, while Arabic encoder ensembles and a fine-tuned Jais decoder performed strongly on prompt safety.Source: R Team Acc Ma-F1 · p. 9

6 Conclusion and Future Work

ArGuard brought Arabic meme-harm detection and LLM prompt-safety detection into one shared task, drawing 35 teams and 27 system-description papers.

6 Conclusion and Future Work

  • ArGuard evaluated two complementary Arabic content-safety tasks: harmful-content detection in multimodal memes and harmful-prompt detection for LLMs.Thirty-five teams participated in the final evaluation and 27 submitted system-description papers, spanning Arabic encoders, multimodal models, VLMs, decoder LLMs, and lexical approaches.Source: §6 · p. 9
  • Results showed a clear gap between binary and fine-grained classification, with fine-grained meme understanding the most challenging setting.VLMs and multimodal ensembles performed well on memes; Arabic encoder ensembles and a fine-tuned Jais decoder performed strongly on prompt safety.Source: §6 · p. 9
  • Future work should improve rare hateful-attack coverage, broaden prompt-safety evaluation across dialectal and code-switched Arabic, and study robustness across cultural contexts and data sources.Source: §6 · p. 9

Limitations

ArGuard covers Arabic multimodal and prompt-safety settings, but its fine-grained labels and dialect coverage leave areas for future extension.

Limitations

  • ArGuard's coverage across Arabic multimodal and prompt-safety settings still leaves areas for future extension.Source: Limitations · p. 9
  • A few fine-grained attack categories, including Exclusion, have relatively few training examples, making per-class estimates less stable.The fine-grained labels are highly imbalanced.Source: Limitations · p. 9
  • Track B covers six Arabic varieties, leaving additional dialects and code-switched text for future expansion.Source: Limitations · p. 9

Ethics and Broader Impact

ArGuard releases harmful-content examples for research on Arabic content moderation and LLM safety, while warning that trained systems need human oversight.

Ethics and Broader Impact

  • ArGuard releases memes and prompts containing hateful, offensive, or otherwise harmful content solely to support research on Arabic content moderation and LLM safety.Source: Ethics and Broader Impact · p. 9
  • The release includes no personal information beyond what already appears in the public source posts.Source: Ethics and Broader Impact · p. 9
  • Trained systems can miss harmful content or over-flag benign content, so the paper recommends human oversight rather than automated moderation.Source: Ethics and Broader Impact · p. 9

Data Availability

The Task A dataset is publicly available for non-commercial research, while access to the Task B dataset is controlled.

Data Availability

A Detailed System Descriptions

Submitted systems span multimodal meme classifiers and Arabic prompt-safety approaches, with recurring use of class-aware training, ensembles, augmentation, and threshold tuning.

A Detailed System Descriptions

  • Track A systems pair Arabic encoders with vision backbones, adapt vision-language models, or use text-only encoders; fine-grained systems also use multitask learning, class-aware losses, ensembling, and threshold tuning.The reported models include MARBERTv2, AraBERT, CLIP, SigLIP, ViT, ResNet, Qwen3-VL, Llama-3.2-Vision, and Gemma 4.Source: A Detailed System Descriptions · p. 13, Table 9 · p. 14
  • Track B systems use Arabic pretrained encoders and ensembles, as well as character n-grams, retrieval, dialect features, decoder models, translation-based classification, and self-taught reasoning.Some submissions also use dialect conditioning, class-weighted training, or threshold and prior adjustment.Source: A Detailed System Descriptions · p. 13, Table 10 · p. 15
  • Across both tracks, class-aware training, multi-seed or multi-model ensembling, data augmentation, and threshold tuning recur across submissions.Track A emphasizes multimodal fusion and fine-grained label constraints; Track B more often uses dialect information, lexical features, and adjustments for dev-test distribution differences.Source: A Detailed System Descriptions · p. 13
  • Tables 9 and 10 detail approaches for the meme and prompt tracks, respectively, while omitting teams without system-description papers.The system summaries rely primarily on teams’ approach forms and are supplemented with their system-description papers when needed.Source: A Detailed System Descriptions · p. 13, Table 9 · p. 14, Table 10 · p. 15

Related papers

Loading 2609.29349v1…