PaperlayerView exact revision on arXiv

Source-linked AI summary

Not Safe for All: Auditing the Dialect Penalty in Text-to-Image Safety Pipelines

Minkyu Kim, Juhwan Choi, YoungBin Kim

arXiv:2608.29589v1cs.AI

TL;DR

T2I safety guardrails may treat non-standard dialects inequitably because they respond to linguistic surface features rather than semantic intent. The paper audits this failure across five English dialects and finds opposing text-filter biases, with the penalty entering during text processing while image generation remains largely dialect-agnostic.

  • Problem

    Existing T2I safety systems are primarily designed for Standard English, leaving equitable protection across other linguistic varieties insufficiently evaluated.

  • Method

    The paper evaluates text filters and post-hoc guardrails on paired prompts spanning five English dialects, using typo ablations and cross-lingual tests to isolate dialect bias from generic OOD effects.

  • Results

    Text-level filters fail in opposing directions: NSFW-T over-censors benign dialect prompts, LatentGuard over-flags toxic ones, and OMod under-detects them, while unguarded image generation remains largely dialect-agnostic.

  • Takeaways & Limitations

    The penalty tracks SAE-centric training-data imbalance and is mitigated by group-balanced exposure, indicating that equitable safety requires evaluating and correcting text-side guardrails.

  • Takeaways & Limitations

    The evaluation uses machine-translated prompts and automated pixel-level classifiers without human validation, and makes no representativeness claim for speech communities.

Abstract

from arXiv · show

Text-to-image (T2I) safety guardrails fail to generalize equitably to non-standard dialects. Evaluating 23,080 paired prompts across five English dialects, we formalize this failure as the dialect penalty, where filters trigger based on linguistic surface features rather than semantic intent. Text-level filters fail in opposing directions: NSFW-T over-flags benign dialect prompts and LatentGuard over-flags toxic ones (bias gaps up to +28.29 pp), while the OpenAI Moderation API under-detects them. A controlled typo ablation confirms this penalty originates from flagging dialectal features, not generic out-of-distribution sensitivity. The pixel-level generator is largely dialect-agnostic; the penalty enters at text processing and cascades unevenly to post-hoc guardrails. We show this bias tracks training data imbalance and is mitigable via group-balanced retraining, with an ablation attributing the gain to balanced exposure rather than to the worst-group objective of GroupDRO (group distributionally robust optimization). Current pipelines systematically fail dialect speakers, an equity failure masked by mean accuracy benchmarks. Our official code and dataset are publicly available at https://github.com/minguinho26/dialect-penalty-t2i. Content Warning: This paper contains offensive, toxic, or disturbing text prompts and generated images.

1 Introduction

This paper identifies a systemic dialect penalty in T2I safety pipelines: text-side guardrails respond unevenly to dialectal surface features, while image generation remains largely dialect-agnostic.

  • Current T2I safety systems often fail to provide equal protection across linguistic varieties beyond Standard English.
  • Dialect syntax alone can produce unequal treatment for prompts with the same content.
  • The evaluation covers five English dialects using paired Standard American English and dialectal prompts, revealing a systemic dialect penalty in safety components.The dataset contains 27,696 prompts, including 23,080 dialectal variants derived from 4,616 SAE prompts.
  • NSFW-T over-flags benign dialect prompts, LatentGuard over-flags toxic ones, and OMod under-detects harmful dialect content.
  • The penalty enters through text-side safety components: the visual generator remains largely dialect-agnostic, and group-balanced training mitigates the imbalance-linked bias.
  • A typo ablation separates dialect-specific bias from generic out-of-distribution sensitivity, while a limited cross-lingual test indicates toxic under-detection is not English-specific.

2 Related Work

Prior work documents dialect-related failures in NLP, multimodal systems, and LLM safety, but equitable performance across English dialects remains largely unexplored in multimodal T2I safety pipelines.

  • Earlier studies report that dialectal variation degrades performance in standard NLP and multimodal tasks and can bypass LLM safety alignment.
  • Existing T2I benchmarks measure risky prompts or fairness and privacy, but do not evaluate equitable performance across English dialects.
  • Prior T2I safety defenses primarily address model outputs, leaving linguistic equity in text-level safety filters unexamined.

3 Methodology

The methodology evaluates dialect bias across text-level filters, post-hoc guardrails, and visual outcomes using paired prompts, controlled perturbations, and balanced-training interventions. It compares dialect and SAE behavior while controlling semantic content, random variation, and data imbalance.

  • Evaluation scope: The study evaluates text-level classifiers and post-hoc guardrails by measuring intervention shifts between dialect and SAE expressions of identical semantics.The evaluation includes text filters, PromptGuard, and SLD, with paired prompts used to isolate intervention effects.
  • Dataset construction: The paired dataset contains toxic prompts translated into five dialects and benign prompts adapted from MS-COCO captions.The dataset construction samples toxic SAE prompts from 14 harm categories and adds benign captions for evaluation.
  • Safety filters: The three text filters use distinct paradigms: binary toxicity classification, safety-specific latent similarity, and category-wise moderation scores.NSFW-T uses a 0.5 toxicity threshold, LatentGuard uses a default similarity threshold of 9.0131, and OMod computes independent violation scores.
  • Evaluation metrics: Text-filter evaluation reports TPR for toxic prompts, FPR for benign prompts, and disparity as each dialect’s performance difference from SAE.These metrics quantify dialect-specific changes in toxicity detection and benign-prompt flagging.
  • Image-level evaluation: Visual evaluation uses pixel-level toxicity classifiers and end-to-end TBR and OCR measures for guarded image outcomes.OCR is based on CLIP similarity degradation, with over-censorship flagged when ∆sim > 0.1; the proxy can also reflect non-censorship image changes.
  • Validation and mitigation: Magnitude-matched typos test whether guardrails respond to dialect-specific features rather than generic OOD perturbations.Typos are adjusted until their CLIP embedding similarity matches the SAE–dialect distance, and responses are averaged across five seeds.
  • Validation and mitigation: Training comparisons vary SAE-to-dialect ratios and contrast ERM, balanced training, and GroupDRO under fixed-size, base-prompt-level splits.Evaluations average results over 10 independent seeds; GroupDRO uses 12 groups combining six varieties with two toxicity labels and excludes the 100% SAE setting.

4 Experiments

Experiments show that dialect penalty reflects semantic-preserving surface variation, enters mainly through text-side safety components, and can be reduced through balanced retraining.

  • NSFW-T raises benign false-positive rates from the SAE baseline of 15.58% to 43.75% for ChcE and 32.37% for AAVE.The corresponding shifts are +28.17 pp and +16.79 pp, respectively.
  • LatentGuard increases toxic true-positive rates by +15.12 pp for AAVE and +7.90 pp for ChcE, whereas OMod decreases toxic detection by 4.60 pp for JamE and 3.11 pp for AAVE.
  • CLIP similarities support semantic preservation: toxic dialect translations score 0.70–0.97 versus 0.41 for the within-category SAE baseline, while benign translations score 0.61–0.75 versus 0.56.
  • Magnitude-matched typos produce negligible shifts, but dialect syntax yields Bias Gaps up to +28.29 pp for NSFW-T and +18.75 pp for LatentGuard.IndE shows near-zero Bias Gaps, supporting a dialect-feature explanation rather than generic OOD sensitivity.
  • OMod’s toxic under-detection recurs in Bavarian and Egyptian Arabic, with TPR decreases of 7.72 pp and 2.48 pp, respectively.This is a limited replication using one filter, two language pairs, and only the toxic direction.
  • Unguarded pixel generation remains largely dialect-agnostic, with no dialect shifting NSFW-I or multi-head toxicity scores by more than 1.89 points.Text-based PromptGuard instead raises OCR by up to +29.58 pp for ChcE and +27.04 pp for JamE, while SLD shows negligible or negative shifts.
  • Balanced retraining removes the gap, reaching 99.95 ± 0.14% worst-group accuracy and |∆FPR| = 0.00 pp, while balanced sampling supplies most of GroupDRO’s gain.At 99.5% SAE, sampling alone reduces |∆FPR| from 5.24 to 3.00 pp; adding worst-group loss changes it to 2.93 pp.

5 Conclusion

The audit identifies a systemic dialect penalty entering text-side safety components: filters respond differently to dialectal surface features, while downstream guardrails and training imbalance shape its effects.

  • Bias gaps reach +28.29 pp for ChcE with NSFW-T and +18.75 pp for AAVE with LatentGuard.
  • NSFW-T over-censors benign dialect prompts, whereas LatentGuard over-flags toxic ones and OMod under-detects them.
  • PromptGuard inherits benign-prompt over-censorship, while SLD remains robust; both over-block toxic JamE prompts.
  • The penalty reflects dialect bias rather than generic OOD degradation, and unguarded generation remains largely dialect-agnostic.
  • Group-balanced sampling recovers worst-group accuracy under SAE-centric imbalance, with most recovery attributed to balanced exposure rather than the worst-group objective.
  • Because the penalty enters at the text side, treating that layer is presented as the causally matched remedy.

Limitations

The study’s evidence is bounded by machine-translated prompts, automated image evaluation, a proxy metric, and limited cross-lingual replication.

  • Machine-translated prompts are not a substitute for native-speaker judgments of naturalness, code-switching, stereotypes, or community representativeness.
  • Post-hoc image evaluations rely on automated pixel-level classifiers without human validation, despite corroboration by multiple classifiers.
  • The OCR metric uses CLIP cosine-similarity drops and cannot disentangle semantic censorship from image degradation, style shifts, or layout changes.
  • The cross-lingual evaluation is limited to one filter, two language pairs, and toxic prompts; only under-detection recurs across both pairs.
  • Future work should extend the audit to more dialects, commercial APIs, and naturalistic user prompts.

Ethics Statement

The ethics statement documents the release and evaluation setup while limiting dissemination of harmful generated content.

  • The paired prompt dataset is released to support future research on mitigating dialect bias.
  • Toxic prompts come from existing benchmarks, while generated images are withheld to prevent dissemination of harmful content.
  • Experiments use Stable Diffusion 1.4 with default sampling and deterministic seeds reused across paired guarded and unguarded conditions.
  • The typo ablation injects character-level mutations and binary-searches the rate to match SAE–dialect CLIP similarity, averaging five seeds.

D Round-Trip Translation Audit

The round-trip audit tests whether machine translation changes safety-relevant meaning, using independent toxicity and content evaluations, while image-level checks assess visual semantic preservation.

  • Scoring: The audit evaluates toxicity as harmfulness of the rendered image, not grammar, dialect, or writing style, using OpenAI Moderation and an independent LLM judge.
  • Text-side audit: OpenAI Moderation flags agree on 98.9% of pairs, with signed score difference −0.0055 and Pearson correlation 0.990 across 23,067 pairs.
  • Text-side audit: The independent S1–S5 judge rates 94.86% of pairs identically and 99.97% within one level, with mean absolute difference 0.052.
  • Text-side audit: The content evaluator marks 98.46% of pairs as conveying the same content, supporting negligible round-trip semantic discrepancy.
  • Image-level audit: Unguarded SAE–dialect image similarities exceed within-category baselines across dialects, with benign pairs ranging from 0.86 to 0.91.
  • Image-level audit: Toxic-prompt similarities remain above the 0.58 baseline for JamE at 0.72 and AAVE at 0.80, despite lower fidelity for unfamiliar tokens.

F Dataset and Dialect Selection

The study constructs controlled paired datasets across five non-standard English dialects and translates benign and toxic prompts using dialect-specific templates. The dataset retains only prompts translated across all dialects, with standardized expansion and formatting constraints.

  • Dialect selection: Five English dialects—AAVE, ChcE, CollSgE, IndE, and JamE—are selected for distinct morphosyntactic, phonological, lexical, or creole features.The paper also evaluates Bavarian and Egyptian Arabic in its non-English dialect selection.
  • Dataset construction: The intersected dataset retains only prompts translated across all five dialects, giving each dialect and SAE baseline identical sample counts.Toxic category counts fall below the 200-prompt target because of source limits and translation refusals for explicit or violent content.
  • Prompt preparation: Benign MS-COCO captions are expanded into dense, photorealistic prompts under a strict 60-word limit while preserving their core semantics.The expansion emphasizes high-density keywords, photographic terms, and semantic fidelity without unrelated objects.
  • Dialect translation: Few-shot dialect templates use linguistically verified examples and dynamically populated dialect placeholders to guide translation into regional varieties.The regional templates leverage documented morphology, phonology, and lexicon, while explicit constraints prevent standard-orthography defaults or adjacent-dialect mixing.
  • Toxic translation: Toxic translation adds a formatting enforcer that preserves original intensity or returns the deterministic token “REFUSED” when translation is restricted.The template also forbids apologies, warnings, and alternative suggestions.

I Data Split and Leakage Audit

The evaluation uses a base-prompt-level stratified split that keeps every dialect paraphrase together and audits duplication and threshold sensitivity. Results show PromptGuard’s dialect over-censorship persists across thresholds, while dialect shifts differ from generic typo noise.

  • Data split: A stratified 9:1 base-prompt split keeps each prompt and its five dialect paraphrases entirely within training or test data.Stratification spans all 20 safety categories, producing a consistent test fraction of 0.097–0.104 per category.
  • Leakage audit: No systematic near-duplication is detected: the maximum cross-split 5-gram Jaccard similarity is 0.68 on a single pair.Strictly disjoint base prompts and no exact text duplicates support the interpretation that balanced ERM performance reflects safety-concept separability rather than memorization.
  • Threshold sensitivity: 19.5–29.6 pp at ∆sim = 0.10, PromptGuard’s dialect over-censorship remains positive across the full tested threshold sweep.The gap smoothly declines to 3.7–7.5 pp at ∆sim = 0.20, while SLD stays near zero with |∆OCR| < 2.6 pp.
  • Dialect versus typo noise: Dialect syntax is penalized more harshly than typo-injected text even when their CLIP text-embedding displacement magnitudes are matched.The ablation directly compares dialect perturbations with calibrated typo perturbations.
  • Displacement direction: Across all 10 dialect–toxicity cells, dialect–typo displacement alignment is lower than the independent typo–typo baseline.For toxic prompts, IndE and JamE fall short of the baseline by 0.120 and 0.085, respectively.

L GroupDRO Implementation Details

The mitigation study compares ERM, balanced sampling, and GroupDRO under a common safety-filter training configuration. GroupDRO defines groups jointly by dialect and toxicity label and updates their weights using observed group losses.

  • Group construction: GroupDRO optimizes worst-group risk across 12 groups defined by dialect and toxicity labels.Examples include toxic AAVE and benign JamE.
  • GroupDRO updates: Group weights start uniformly at 1/12 and are updated by q_g ← q_g · exp(η · ℓ_g), followed by renormalization.The group step size is η = 0.01, and ℓ_g is the mini-batch mean loss for group g.
  • Sampling: Under extreme SAE imbalance, mini-batches use group-balanced sampling with equal expected draws per group.This balanced sampling applies alongside the ERM and GroupDRO comparisons.

M Image Evaluator Cross-Check

The paper cross-checks unguarded image toxicity with ShieldGemma-2B as an evaluator independent of the two primary CLIP-based evaluators. This provides a separate visual measurement channel for the baseline generations.

  • Evaluation rationale: The cross-check is designed to test whether unguarded baseline results depend on the chosen image-level evaluators.The primary evaluators are NSFW-I and a multi-head classifier.
  • Independent evaluator: ShieldGemma-2B independently evaluates all unguarded toxic generations using continuous violation-probability scores.Its vision-language architecture differs from the primary evaluators’ OpenAI CLIP ViT-L/14 vision encoder.
Loading 2608.29589v1…