AK_QMUL at ArGuard Shared Tasks: A Single Fine-Tuned JAIS for Arabic Harmful-Prompt Detection

1 Introduction

• The ArGuard-2026 tasks ask whether Arabic prompts are safe or unsafe (B1) and which of seven harm domains an unsafe prompt belongs to (B2).Both tracks use macro-F1, which weights rare classes as heavily as common ones.Source: §1 · p. 1
• One JAIS-6.7B backbone uses 4-bit QLoRA, a last-token representation, and a linear classification head for both tasks.B1 factors labels by dialect register and uses AraSafe augmentation; B2 averages five seeds of a seven-way model.Source: §1 · p. 1
• Both submissions ranked first, with macro-F1 0.984 on B1 and 0.790 on B2.The paper also reports results by dialect and class to show where the systems succeed and where errors remain.Source: §1 · p. 1

2 Task and Data

• The shared prompt pool has MSA and five regional dialect labels, while B1 covers all prompts and B2 assigns unsafe prompts to one of seven harm domains.The provided data include labelled train and development splits and an unlabelled test split.Source: §2 · p. 1
• Both tracks use macro-F1 as their official metric.Source: §2 · p. 2
• Unsafe rates vary by dialect: MSA prompts are unsafe 37% of the time, compared with 72%–91% across the five dialects.B2 is also imbalanced: adult_content is nearly four times as frequent as hate_speech, and some classes overlap in behaviour.Source: §2 · p. 2

3 Our System

• Both systems share the JAIS-6.7B QLoRA backbone, last-token linear head, and Section 3.1 training recipe, but differ in task-specific configurations.Source: Table 1 · p. 2, §3 · p. 2

3.1 Shared backbone and training

• JAIS-6.7B uses 4-bit NF4 quantisation, LoRA adapters on all linear projections with frozen base weights, and the last non-padding top-layer state for classification.The shared training recipe uses class-weighted cross-entropy, stratified group k-fold with k=7, and a final refit on train plus development data.Source: §3.1 · p. 2

3.2 Task B1: a dialect-factored head

• B1 trains a four-way safe/unsafe-by-MSA/other head, then collapses the plain argmax by safety label without dialect-based post-processing.Source: §3.2 · p. 2
• B1 adds 7,607 filtered, deduplicated safe MSA prompts from AraSafe to the training fold only, targeting the least represented safe-MSA class.The held-out fold remains untouched; augmentation is applied only for the final test refit.Source: §3.2 · pp. 2–3

3.3 Task B2: a seed-averaged classifier

• B2 uses the shared backbone with a seven-way head and averages the softmax distributions from five seeds before taking the argmax.It uses no dialect factoring, augmentation, stacking, held-out meta-learner, or per-class thresholding.Source: §3.3 · p. 3

• Both tracks ranked first: B1 reached macro-F1 0.984 and B2 reached 0.790.Source: §4 · p. 3
• B1 accuracy ranged from 93.6% in Moroccan to 99.8% in Egyptian, while predicted unsafe rates closely tracked true rates across dialects.For MSA, the rates were 37.1% predicted and 36.6% true; for Egyptian, 91.6% and 91.3%.Source: §4 · p. 3, Table 2 · p. 3
• Removing AraSafe augmentation lowered B1 to 0.624, while register factoring raised the score from 0.983 to 0.984.The training pool was about 80% unsafe, whereas the blind test was close to balanced; without augmentation, the model predicted 6,426 unsafe prompts against 3,851 truly unsafe.Source: §4 · pp. 3–4, Table 3 · p. 3
• Six B2 classes reached F1 between 0.73 and 0.98; Bully recall was 0.443, with most remaining true Bully prompts assigned to harassment or harm-to-others.Bully precision was 0.815 and the class had 908 examples, more than four classes scoring higher.Source: §4 · p. 4, Table 5 · p. 4
• Bully F1 was 0.81–0.97 in the five dialects but 0.46 in MSA, where 719 of 908 test prompts fell; MSA macro-F1 was 0.70 versus 0.88–0.95 for each dialect.Adult_content reached F1 0.95–1.00 across dialects, while other harm classes also dropped in MSA and recovered elsewhere; non-MSA per-class cells had 14–239 examples and are indicative.Source: §4 · p. 4, Table 4 · p. 4

• Both submissions ranked first on their tracks, achieving macro-F1 0.984 on B1 and 0.790 on B2.Source: §5 · p. 4
• B1 uses a register-factored four-way head, AraSafe augmentation, and plain-argmax decoding without dialect routing.The resulting system is accurate across all six dialects.Source: §5 · p. 4
• B2 uses the same backbone as a seven-way classifier and averages the softmax outputs of five seeds.Source: §5 · p. 4
• B2’s residual error is concentrated in Bully prompts in MSA, which are confused with harassment and harm-to-others.Source: §5 · p. 4

• The study does not compare alternative Arabic encoders or architectures, so its results speak to this recipe rather than the task’s best attainable performance.Source: Limitations · pp. 4–5
• The B1 ablation isolates two design choices on the official test split under one submitted seed and fold, without repetition across seeds.Source: Limitations · p. 5
• The Bully error in MSA remains unresolved, with prompts confused with harassment and harm-to-others; closing the gap likely needs finer annotation rather than additional data.Source: Limitations · p. 5

• The detection system is intended to support content moderation and safer deployment of Arabic language models.Source: Ethics Statement · p. 5
• The work uses only ArGuard shared-task data and AraSafe under permitted terms, and releases no new harmful content.Source: Ethics Statement · p. 5
• The repository contains system code and train/development splits only.Source: Ethics Statement · p. 5