PaperlayerView exact revision on arXiv

Source-linked AI summary

How I Aligned Your Model: Scaling Up Multilingual Safety Training with Templates

Max Neuwinger, Hanna Yukhymenko, Imanol Schlag

arXiv:2610.40516v1cs.CLcs.AIcs.LG

TL;DR

English-centric safety alignment and scarce multilingual data leave limited coverage of low-resource languages and culturally specific harms. The paper creates MSAFETEMP with a template-based pipeline that generates and validates harmful, benign, and localized prompts. Its evaluation finds multilingual safety gaps missed by English-only testing, while training ablations report improved attack resistance and reduced over-refusal.

  • Problem

    Safety alignment data is English-centric, and open multilingual safety data remains scarce, particularly for low-resource languages and culturally specific harms.

    Source: §1 · p. 1

  • Method

    The pipeline converts harmful prompts into templates, translates them, fills slots with harmful or benign entities, and adds culturally localized variants validated by LLM judges.

    Source: §1 · pp. 2–3

  • Results

    English-only evaluation misses multilingual safety gaps, localized variants expose failures missed by translation, and training ablations show improved attack resistance and reduced over-refusal.

    Source: §1 · pp. 2–3

  • Takeaways & Limitations

    MSAFETEMP provides multilingual training and evaluation data across 44 training languages and 60 evaluation languages for safety research in lower-resource settings.

    Source: §8 · p. 10

    The human study covers only a subset of languages with one or two respondents each, so larger human evaluation remains future work.

    Source: §7 · p. 9

Abstract

from arXiv · show

Safety alignment for Large Language Models is heavily focused on English with limited multilingual representation, particularly damaging low-resource language performance. Related open training datasets are often limited in size or do not follow region-localized harmful topics to broaden the scope of alignment beyond cultures. We propose a scalable template-based augmentation pipeline to produce massive multilingual refusal and benign corpora, extracting safe templates from harmful prompts, translating them and then filling in the slots with culturally authentic entities, both harmful and benign. As a result, we release MSafeTemp, a validated corpus spanning 60 languages useful for multilingual safety alignment and evaluation. Using the dataset, we show that English-only evaluation misjudges multilingual safety: for several models, attacks that fail in English succeed once translated, while culturally localized variants expose further failures that translation alone misses. We also confirm that augmenting safety training data through templates allows for higher diversity and quality of alignment, resulting in improved safety performance beyond English. MSafeTemp and the respective pipeline are aimed at facilitating research and development of safer and more responsible models for less-represented cultures.

1 INTRODUCTION

English-centric safety data leaves multilingual and culturally specific gaps, especially in low-resource languages. The paper introduces a template-based pipeline and MSAFETEMP to produce and evaluate culturally grounded harmful and benign prompts across languages.

Motivation

  • Existing safety alignment data is English-centric, while human red-teaming is costly and slow to scale and translation often misses locally specific context.Aligned language models may also refuse to translate or adapt harmful prompts.Source: §1 · p. 1

Contributions

  • The pipeline abstracts harmful prompts into templates, translates them, and fills their slots with generic or culturally grounded harmful entities and safe benign alternatives.Variants are validated by LLM judges distinct from the generator models.Source: §1 · pp. 2–3

Motivation

  • Low-resource languages are underrepresented in safety research and are often addressed through translation without cultural adaptation, despite pronounced safety degradation in those settings.The paper treats safety as a combination of model, language, and culture axes.Source: §1 · pp. 1–2

Contributions

  • MSAFETEMP combines a template-based pipeline with more than 1 million validated prompts across 44 languages, matched benign data, and held-out evaluation sets.The corpus is provenance-tracked and split by source seed so no descendant of a test seed enters training.Source: §1 · p. 3
  • English-only evaluation misses multilingual safety gaps, and culturally localized variants expose failures that translated prompts alone do not reveal.The evaluation also finds that several aligned open models struggle with out-of-distribution harmful and benign data.Source: §1 · pp. 2–3

2 RELATED WORK

Prior work provides multilingual red-teaming, over-refusal measurement, and synthetic data, but leaves gaps in geographic scope and controlled harmful–benign comparisons. This paper distinguishes its approach through a cultural layer scaled across 60 languages and paired training and evaluation data.

Multilingual safety and red-teaming

Safety and helpfulness

  • Existing over-refusal benchmarks and multilingual toxicity tests do not provide controlled harmful and benign variants from the same template.The paper uses matched controls to measure both outcomes on closely shared linguistic and cultural content.Source: §2 · p. 3

Synthetic alignment data and model judges

  • Unlike prior synthetic-data and moderation work, this approach generates paired training and evaluation data while checking that harmful intent survives translation and localization.Because model judges can fail systematically, generation and judging are separated, with explicit checks where possible and remaining limitations reported.Source: §2 · p. 3

3 TEMPLATE-BASED AUGMENTATION METHOD

The pipeline separates harmful intent from prompt wording so templates can be translated and filled at scale. Its branches create harmful, benign, and culturally localized variants while preserving the prompt frame and applying validation.

3 TEMPLATE-BASED AUGMENTATION METHOD

  • The method targets four requirements: preserve harmful intent, produce fluent target-language prompts, use culturally relevant entities where appropriate, and pair each harmful template with a benign control.These requirements aim to avoid measuring translation quality instead of safety and to control over-refusal.Source: §3 · pp. 3–4

3.1 DESIGN MOTIVATION

  • Replacing harm-bearing phrases with typed placeholders lets the pipeline translate safe template text and refill slots, avoiding direct translation or adaptation of complete harmful requests.Translated and localized attacks share a frame, while benign controls differ only in their placeholder values.Source: §3.1 · p. 4
  • 2,500 WildGuardMix prompts supply training seeds, while 200 HarmBench behaviors are held out for evaluation; 2,000 training seeds and descendants form training data and 500 seeds form a held-out split.Both sources receive harmful and benign template-level options to assess scaling and cross-lingual transfer effects on over-refusal.Source: §3.1 · p. 4
  • Each seed and language receives a translated main template and two validated paraphrases, with judges checking template validity, language consistency, translation quality, and slot preservation.English skips translation but retains both judge steps.Source: §3.1 · p. 4

3.2 FURTHER DATA AUGMENTATION BRANCHES

  • TR fills templates and their paraphrases with harmful placeholder combinations, then judges harmfulness, language quality, coherence, and preservation of the original frame.The three branches produce different augmentation types from different pipeline stages.Source: §3.2 · p. 5, Figure 2 · p. 5
  • OR controls preserve the harmful question form but use safe values; DIRECT examples measure over-refusal, while BOUNDED examples remain training data.BOUNDED prompts still require withholding transferable harmful details.Source: §3.2 · p. 5
  • PH uses per-language cultural cards for local entities, permits grammatical inflection and rephrasing, and applies the same final judge as TR.The cards describe locally relevant groups, organizations, events, concepts, and institutions.Source: §3.2 · p. 5

4 THE MSAFETEMP DATASET

MSAFETEMP expands a small set of English seeds into large training and evaluation collections across many languages. The pipeline’s scale comes from controlled variation, while culturally localized data is smaller and more costly to produce.

Dataset scale

  • 1.8M validated prompts expand 2,500 WildGuardMix seeds across 44 languages, while 216,486 prompts expand 200 HarmBench seeds into a 60-language benchmark.WildGuardMix serves training and HarmBench remains evaluation-only; scale comes from varying each seed rather than collecting more seeds.Source: §4 · p. 6, Table 1 · p. 6

Dataset construction

  • The culturally localized PH subset is smaller because its generation costs more, and localized conditioning limits fitting placeholder variety while producing more focused, natural data.Judges reject unsafe or meaning-losing benign samples and culturally unsuitable placeholder candidates.Source: §4 · p. 6

Dataset diversity

  • Each of three templates per seed and language receives five fills, yielding up to 15 harmful variants; harmful entities have a distinct-to-total ratio around 0.7 and benign entities 0.94.No entity appears in more than 1% of a language’s harmful fills, and diversity does not differ significantly between high- and low-resource languages.Source: §4 · p. 6

Human validation

  • Human raters score prompts 4.43/5 for naturalness and 4.46/5 for cultural fit, but endorse 65% of benign labels versus 91% of harmful labels.Questionable benign samples may cause measured over-refusal to overstate true over-refusal.Source: §4 · p. 6

Dataset contribution

  • The pipeline turns a small set of English texts into a multilingual corpus of diverse harmful and benign prompts.The authors present this as answering RQ2 on scalable data production.Source: §4 · p. 6

5 EVALUATING THE MULTILINGUAL SAFETY GAP

English-only evaluation does not reliably predict multilingual safety: translation exposes failures for several models, while localized prompts can reveal failures translation misses. The direction and size of these gaps vary by model, and safety training narrows some gaps while often increasing benign refusals.

5 EVALUATING THE MULTILINGUAL SAFETY GAP

  • Olmo’s HarmBench ASR rises from 0% on English prompts to 13.7% on translated prompts; multilingual attack success also increases for several other models.On WildGuard, translated variants break Olmo, Gemma, and both Apertus models 2.4–4.4× as often as English seeds, while Llama fails most often in English.Source: §5 · p. 7
  • Localized prompts reveal failures translation misses: PH and generic fills break models about equally often, but their broken behaviors overlap at only Jaccard 0.40–0.45.About 45% of arm-specific failures occur only with local entities; localization lowers ASR for most leading models but raises it for Apertus-8B and instruction-tuned Olmo.Source: §5 · p. 7, Figure 3 · p. 8
  • Matched jailbreak wrappers raise average per-attempt ASR for Apertus and Llama, but show no clear gain over PH for Olmo or Qwen.Trying several wrappers finds more failing prompts, but that cumulative rate reflects giving the attacker more attempts.Source: §5 · p. 7
  • Paraphrases break all seven models more often than main templates on WildGuard, by up to +3.7%, but not on HarmBench.Templates broken by only some fills produce 60–100% of unsafe answers, and testing a single fill misses roughly half of broken templates.Source: §5 · p. 7
  • The safety–over-refusal tradeoff varies by model: GPT-OSS-20B refuses 32–45% of benign pairs, while only a few models achieve low ASR without that cost.More permissive models remain easier to attack outside English; Qwen3.5-27B on HarmBench is among the few that reach low ASR without paying for it in refusals.Source: §5 · p. 8
  • Safety training narrows multilingual gaps for instruction-tuned models with the largest gaps, often while increasing refusals on closely matched benign prompts.Training also improves performance in languages absent from training, with gains close to those in seen languages.Source: §5 · p. 8

6 TRAINING EXPERIMENTS

The training experiments find that MSAFETEMP-based SFT reduces multilingual attack success, while DPO lowers some of the over-refusal SFT introduces. The safety gains come with capability costs, especially in math reasoning, and the over-refusal effect varies across model families.

6 TRAINING EXPERIMENTS

  • Training uses one epoch of SFT on refusals and benign answers mixed with general instruction data, followed by DPO on preference pairs sampled from the fine-tuned model.The same data are applied to base and instruction-tuned checkpoints to test when the data should enter the training pipeline.Source: §6 · p. 8
  • SFT sharply reduces multilingual attack success across model families, tracks, and base or instruction-tuned checkpoints; DPO lowers over-refusal while keeping attack success low.DPO fully removes the SFT over-refusal increase for Apertus, but a residual increase remains for Olmo and Llama on at least one benchmark.Source: §6 · pp. 8–9, Table 3 · p. 9
  • SFT’s capability cost is modest on MMLU but larger on GSM8K, while its over-refusal effect depends on the model.Multilingual over-refusal rises only slightly for Apertus but substantially for Olmo, which begins with very low over-refusal.Source: §6 · p. 8
  • The authors conclude that MSAFETEMP provides effective multilingual safety supervision, with a capability cost concentrated in math reasoning.DPO recovers much of SFT’s over-refusal, fully for Apertus, while other model families retain a residual increase on at least one benchmark.Source: §6 · p. 9

7 DISCUSSION AND LIMITATIONS

The paper’s limitations include reliance on automated judging, a small and linguistically limited human study, inherited harm labels, and single-turn fixed-template prompts. The authors also describe the pipeline and dataset as resources for developing safer multilingual models, particularly in low-resource settings.

7 DISCUSSION AND LIMITATIONS

  • The pipeline relies heavily on automated LLM judging, and a full ablation across judge models remains future work.Two judges from another model family preserve the direction of headline results on a 5,000-response subsample.Source: §7 · p. 9
  • The interim human evaluation covers only a subset of languages, with one or two respondents per language, so broader human evaluation remains future work.The study rates naturalness and cultural fit highly.Source: §7 · p. 9
  • Harm labels assigned to English seeds are inherited by localized descendants, which may make some variants more severe than their labels suggest.The authors identify relabeling a sample of localized variants as future work.Source: §7 · p. 9
  • Training ablations cover SFT followed by DPO on several models and checkpoints; RL-based methods remain future work.The prompts are also limited to single turns and fixed templates, leaving multi-turn and iterative attacks as a next step.Source: §7 · p. 9
  • The authors expect the released artifacts and recipe to support safer multilingual model development, including alignment efforts for language-specific, often low-resource setups.They also hope the evaluation findings will encourage greater attention to low-resource languages in training and evaluation.Source: §7 · pp. 9–10

8 CONCLUSION

The conclusion presents MSAFETEMP as a multilingual corpus and template-based pipeline for safety alignment and evaluation. Its findings show that English-only evaluation misses multilingual and culturally localized failures, while diverse verified training data improves safety in languages unseen during training.

8 CONCLUSION

  • MSAFETEMP has a training track covering 44 languages and an evaluation track covering 60, with harmful, benign, and culturally localized variants in both.The corpus and template-based augmentation pipeline are intended for safety alignment and evaluation.Source: §8 · p. 10
  • English-only evaluation does not capture multilingual safety evenly: the gap’s direction depends on the model, and localized variants reveal failures translation alone misses.The evaluation includes leading open models on held-out harmful and benign prompts in up to 60 languages.Source: §8 · p. 10
  • Training on diverse, verified data improves safety in languages unseen during training.The paper reports training ablations on generated data alongside evaluations of leading open-source and open-weight models.Source: §8 · p. 10
  • The authors argue multilingual safety should be evaluated across language, culture, safety, and helpfulness rather than judged by English refusal rates alone.Source: §8 · p. 10

ETHICS STATEMENT

The paper describes MSAFETEMP as dual-use because it contains harmful prompts and jailbreak wrappers, and explains how access and examples are handled.

ETHICS STATEMENT

  • MSAFETEMP includes harmful prompts and jailbreak wrappers, so the Hugging Face dataset requires viewers to accept a content warning before viewing.Harmful examples are paired with safe responses for training ablations.Source: ETHICS STATEMENT · p. 10
  • The paper says its examples come from public safety benchmarks or are derived from them, and none carries operational uplift.It discloses the underlying generation resources and reports only aggregate human-study results.Source: ETHICS STATEMENT · p. 10
  • The stated intended benefit is broader safety coverage for speakers and communities poorly served by English-centered alignment.The human study was IRB approved.Source: ETHICS STATEMENT · p. 10

REPRODUCIBILITY STATEMENT

The paper points readers to released code and data, and identifies where to find the pipeline, dataset, training, evaluation, and hyperparameter details.

REPRODUCIBILITY STATEMENT

  • The authors publicly release the pipeline codebase, data artifacts, and evaluation design under a permissive license.The data artifacts follow the structure specified in Appendix A.7.Source: REPRODUCIBILITY STATEMENT · p. 10
  • Pipeline design and model-selection rationale appear in §3 and Appendix A, with a worked example of every stage in Appendix A.6.Dataset statistics, format, labels, and datasheet are in Appendices A.4 and A.7.Source: REPRODUCIBILITY STATEMENT · p. 10
  • Training and evaluation specifications and hyperparameters are in Appendix G.1, while language-level model evaluations appear in Appendices D and D.1.Source: REPRODUCIBILITY STATEMENT · p. 10

AI USE STATEMENT

The appendix documents generative AI's role in producing and judging data, the pipeline's validation choices, and how cultural localization is checked.

AI USE STATEMENT

  • Language models perform data generation, translation, cultural localization, and judging; the authors also used AI for writing and coding assistance and reviewed the outputs.The authors state that research ideas, experimental design, and interpretation are their own.Source: AI USE STATEMENT · pp. 10–11

APPENDIX CONTENTS

  • Appendix notation defines abbreviations for evaluation tracks, prompt arms, metrics, models, and training stages.It also points to generation, evaluation, training, dataset, and language-level evaluation details elsewhere in the appendix.Source: APPENDIX CONTENTS · p. 19

A.1 SOURCES, MODELS, AND JUDGE DESIGN

  • Generation and judging use different model families, but GPT-OSS-20B shares a family with the evaluation judge, so its comparison warrants caution.The judge design separates criterion-level Boolean decisions and removes a grammar check that produced false flags in a German audit.Source: §A.1 · p. 20

A.2 BRANCH-LEVEL GENERATION AND VALIDATION

  • Extraction drops prompts whose harmful intent cannot fit short, narrow slot values; translation preserves English slot tags and patches only judge-flagged spans.Translation retains the best version after at most three patch rounds; generic harmful fills can reach 15 variants per seed and language.Source: §A.2 · p. 20
  • Benign variants distinguish DIRECT requests, where refusal is clearly an error, from BOUNDED requests, used only as training signal with safe partial answers.Cultural cards provide per-language entity pools; localization follows template translation and validation, with accepted entities checked for slot fit and contextual coherence.Source: §A.2 · p. 20, §A.3 · pp. 20–21

A.4 MSAFETEMP DATASET DETAILS AND STATISTICS

The appendix details MSAFETEMP's filtering, examples, documentation, and entity-diversity analyses, including results across templates and language-resource tiers.

A.4 MSAFETEMP DATASET DETAILS AND STATISTICS

  • WildGuardMix rows are 35.1% crimes and illegal activities and 27.1% explicit content; HarmBench rows are 52.1% harm and misuse and 36.6% crimes and illegal activities.The tracks have complementary harm profiles; Table 6 reports their largest primary categories.Source: §A.4 · pp. 21–22, Table 6 · p. 22

A.6 TEMPLATE-BASED AUGMENTATION EXAMPLES

  • HarmBench examples trace seed prompts through template translation and generic fills to localized variants; Olmo-3-7B-Instruct complied with a localized German variant it refused in generic forms, then refused after MSAFETEMP fine-tuning.The example illustrates a failure reachable through localization, not translation alone.Source: §A.6 · pp. 23, 25

A.7 DATASET DOCUMENTATION AND RESPONSIBLE RELEASE

  • Records retain provenance and validation metadata, while harmful prompts are paired with safe refusal responses; labels are automatic and not fully human-calibrated.An interim study endorsed 91% of sampled harmful labels and 65% of sampled benign labels; the authors recommend spot-checking against native-speaker judgment where possible.Source: §A.7 · p. 26

B.1 RECOVERING SLOT FILLS

  • Generated slot fills are recovered by aligning variants to templates; fill-alone spans account for 82.0% of HarmBench and 86.3% of WildGuard fills.Five-character stemming is reported as a conservative lower bound because inflection can inflate distinct-entity counts.Source: §B.1 · p. 26

B.2 OVERALL DIVERSITY

  • Harmful-fill D/F is 0.70 for HarmBench and 0.71 for WildGuard, while benign-fill D/F is 0.95 and 0.94, respectively.Localized post-hoc slots are less diverse (D/F 0.15 and 0.02) because they draw from pools of a few dozen entities per language.Source: §B.2 · p. 27

B.3 NO COLLAPSE AT THE TEMPLATE LEVEL

  • Across harmful variants of one template, all fills are distinct in 91.6% of HarmBench and 93.9% of WildGuard cases; languages mostly contribute their own entity choices.The median slot of an original has 430 distinct entities among 654 HarmBench fills and 350 among 415 WildGuard fills across languages.Source: §B.3 · p. 27

B.4 PER-LANGUAGE DIVERSITY

  • WildGuard D/F is 0.65, 0.64, and 0.62 in the high-, mid-, and low-resource tiers, respectively (p = 0.55).Unstemmed D/F is slightly higher in low-resource HarmBench languages (p = 0.03), attributed to richer inflection rather than greater entity variety.Source: §B.4 · p. 28

B.5 WHERE ENTITIES DO RECUR

This section examines recurring entities and the limits of interpreting them, including patterns tied to canonical slots, first names, and curated localization pools. It also discusses safety failures surfaced by translated and paraphrased prompts on the WildGuard test split, their variation across several dimensions, and measurement limitations from degenerate outputs.

B.5 WHERE ENTITIES DO RECUR

  • 0.22% of HarmBench’s and 0.08% of WildGuard’s harmful fills came from even the most frequent harmful entity, while WildGuard first names showed greater concentration within languages.Elena accounted for 2.7% of name fills across 35 cased-script languages; the top five names covered a median 17.0% of each language’s fills, and repetition was judged bounded and unlikely to affect safety behavior.Source: §B.5 · pp. 28–29, Table 12 · p. 30

B.6 POST-HOC LOCALIZATION

  • 98.2% of WildGuard localized fills matched pool entities, and the generator used a median 97.2% of each language’s pool rather than collapsing onto a subset.94.3% of localized fills used entities appearing in no other language; free slots remained as diverse as harmful fills (D/F = 0.83), and HarmBench showed a median 22.5 effective localized entities per language.Source: §B.6 · p. 30, Table 13 · p. 31

B.7 SUMMARY

  • Across 60 HarmBench and 44 WildGuard languages, harmful slots had over 2.8 million distinct entities, no entity exceeded 1% of a language’s harmful fills, and diversity was similar across resource levels.Repetition mainly came from canonical narrow-slot entities, preferred first names in WildGuard role-play prompts, and curated post-hoc localization pools; per-language entity diversity is tabulated for both datasets.Source: §B.7 · pp. 31–32, Table 14 · p. 33, Table 15 · p. 34

C.1 STUDY DESCRIPTION

  • The study used fluent or native professional informants to assess prompt naturalness, cultural fit, and safety labels, with consent, privacy, and skip-or-stop protections.Participants reviewed 40 prompts per language, split evenly between harmful and benign; the IRB-approved study included content warnings, voluntary participation without compensation, and withdrawal options.Source: §C.1 · p. 35, §C.2 · p. 35, §C.3 · p. 36

C.4 RESULTS

  • The expert study collected 600 judgments on 440 items from 15 submissions covering 11 languages, but disputed benign labels and sparse, clustered ratings limit broader quality claims.Some respondents saw fictional or softened scenarios as retaining harmful intent; comments also noted awkward wording, literal translations, unfamiliar institutions, and incoherent scenarios, with Greek and French rated lowest.Source: §C.4 · p. 36

D.1 PER-LANGUAGE RESULTS

  • GPT-OSS-20B had high benign over-refusal in every language, indicating that its near-zero attack success came with broad refusal rather than calibrated safety.The evaluation reported per-language attack-success rates for translated and post-hoc cultural arms and benign-pair over-refusal across the leading-model panel.Source: §D.1 · pp. 37, 40, Table 23 · p. 40

D.2 TEMPLATE-LEVEL ANALYSIS

  • For 6 of 7 models, pooled paraphrase attack success was lower than for main templates, though paired cell comparisons varied by model.More paired cells favored paraphrases for Olmo, Llama, Gemma, and gpt-oss; more favored main templates for Apertus-70B, while Apertus-8B and Qwen3.5 showed no significant difference.Source: §D.2 · p. 41, Table 24 · p. 41
  • 2.2% vs. 0.02% of Apertus-70B templates were all-broken versus the independent-fill baseline; partial templates produced 83–100% of unsafe answers.Harm and misuse was the most broken top-level category for every model, while harassment and bullying was least broken in HarmBench’s taxonomy; many subcategories had only one to five behaviors.Source: §D.2 · pp. 41–42, Table 25 · p. 42, Table 27 · p. 42

E WILDGUARD TEST-SPLIT EVALUATION

  • Translated arms break Olmo, Gemma, and both Apertus models 2.4–4.4× as often as their English seeds, while Qwen3.5 and gpt-oss stay below 1% on every arm.gpt-oss has the panel’s highest over-refusal; that rate uses only 76% of benign pairs because the rest exhaust the generation budget.Source: Table 31 · p. 45, E WILDGUARD TEST-SPLIT EVALUATION · p. 45

E.1 TRANSLATED MAIN TEMPLATE VS. PARAPHRASED TEMPLATES

  • Paraphrases break every model more often than the main template in 44/44 languages for Apertus-70B and 43/44 for Olmo-3.1-32B.Within language-original cells, every model’s differences favor paraphrases (sign test p < 10−4), indicating failures beyond those reached by translated wording alone.Source: Table 32 · p. 45, §E.1 · p. 45

E.2 PER-TEMPLATE BREAKING OUTCOME

  • Partial templates produce most unsafe answers, ranging from 60% for Apertus-70B to 92% for Qwen3.5, showing that entities can determine whether a fill breaks a template.Using only the first fill, 14.4% of Apertus-70B templates count as broken, versus 25.8% with all fills.Source: §E.2 · p. 46
  • At least one templated descendant breaks 87–93% of refused originals for Apertus, Llama, and Olmo, versus 62% for Gemma and 27% and 18% for gpt-oss and Qwen3.5.This cumulative measure spans roughly 600 descendants per original, not a per-attempt success rate.Source: Table 34 · p. 46, §E.2 · p. 46

E.3 MOST BROKEN HARM CATEGORIES

  • Crimes and illegal activities is the most broken top-level category for every model, while digital crimes, sex-related crimes, controlled items, financial crimes, and violent crimes lead at subcategory level.Erotic content is the largest category yet among the least broken; subcategory rates can be unstable when based on fewer than ten originals expanded into roughly 600 prompts each.Source: §E.3 · p. 46, Table 35 · p. 47, Table 36 · p. 47

E.4 PER-LANGUAGE RESULTS

  • Per-language results cover translated attacks, post-hoc cultural attacks, over-refusal, and paraphrase effects, but languages with many degenerate outputs are measured on fewer, non-random responses.For example, Apertus-70B loops on most Amharic prompts.Source: §E.4 · p. 47, Table 37 · p. 47, Table 38 · p. 48

F.1 PAIRED PH VS. TR TEST

  • Across languages, PH attack success is significantly lower than TR for most leading models and several instruction-tuned checkpoints, but higher for Apertus-8B on HarmBench and the Olmo instruction-tuned model.After MSAFETEMP fine-tuning, PH is below TR for every available model; fine-tuned models almost never comply with localized requests.Source: Table 41 · p. 51, §F.1 · p. 51

F.2 REPHRASING ABLATION FOR POST-HOC CULTURAL GENERATION

Cultural fills leave overall attack success roughly level with generic fills but expose different successful behaviors. The later analyses examine jailbreak wrappers, uncertainty, judge robustness, and training-data choices.

F.2 REPHRASING ABLATION FOR POST-HOC CULTURAL GENERATION

  • Across 12 shared behavior-language cells, cultural fills leave attack success roughly level with generic fills while reaching different failures.The two cultural-fill arms share fewer than half of the behaviors they break (Jaccard 0.45 and 0.40); about 45% of failures unique to one arm occur only with cultural fills.Source: §F.2 · p. 52, Table 42 · p. 52

F.3 MATCHED MULTI-WRAPPER EVALUATION

  • Across ten wrapped attempts, wrappers increase per-attempt vulnerability for Apertus and Llama, show no clear Olmo effect, and leave Qwen largely resistant.Cumulative success rises with more attempts, including for Olmo, but compares ten wrapped attempts with one baseline response and does not measure wrapper quality at equal budgets.Source: §F.3 · p. 53, Table 43 · p. 53, Table 44 · p. 53

F.4 CONFIDENCE INTERVALS

  • Confidence intervals are narrow for multilingual means (about 1.5–4 percentage points) but wide for English-only cells based on at most 200 seeds.The intervals use 2,000 source-family bootstrap draws to account for shared seeds and descendants.Source: §F.4 · p. 53

F.5 INDEPENDENT JUDGE CHECK

  • Independent judges preserve the headline effect directions, though attack-success agreement is moderate and Safeguard is stricter on dual-use or partial compliance.Llama-3.3-70B reproduces the translated-minus-English gap where Safeguard finds one (+3.5 to +10.5 points versus +4.1 to +5.7); IT-to-DPO attack-success reductions remain negative across both judge families, tracks, and judges (−6.5 to −17.0 versus −8.4 to −13.7).Source: §F.5 · pp. 54–55

G.1 EXPERIMENTAL DETAILS

  • The SFT data also adds short benign task requests with the same frame as WildGuardMix prompts, reducing reliance on refusal-only answers to such requests.Candidates are filtered for benign direct-task content and overlap with HarmBench or OR-Bench prompts.Source: §G.1 · p. 55
  • DPO uses on-policy preference pairs from safety prompts, retaining 58,272 pairs for Apertus, mostly from benign prompts.Harmful prompts yield pairs only when the fine-tuned model produces both a natural refusal and an unsafe completion; benign prompts pair helpful answers against over-refusals.Source: §G.1 · pp. 55–56

G.2 PER-STAGE MODEL EVALUATION

Instruction tuning improves safety but raises over-refusal on benign prompts. MSAFETEMP training brings attack success on held-out languages close to seen-language levels, with per-language tables detailing attack and over-refusal outcomes.

G.2 PER-STAGE MODEL EVALUATION

  • Instruction tuning changes base checkpoints from permissive and toxic to reliable refusers, while also raising over-refusal on benign prompts.Table 50 organizes safety, over-refusal, and capability results by model family and training stage.Source: §G.2 · p. 56, Table 50 · p. 58

G.3 GENERALIZATION TO HELD-OUT LANGUAGES

  • After MSAFETEMP training, attack success on 16 held-out languages falls to nearly the level on 43 seen languages, with a residual gap of at most 1.2 points.The language groups are compared before and after fine-tuning; the held-out group is evaluation-only.Source: §G.3 · pp. 58–59, Table 54 · p. 62

G.4 PER-LANGUAGE RESULTS OF TRAINED MODELS

  • Per-language translated and post-hoc cultural attack-success tables compare instruction-tuned checkpoints with their MSAFETEMP SFT-plus-DPO counterparts across non-English languages.The tables mark the 16 held-out languages; the post-hoc cultural breakdown covers 58 languages.Source: §G.4 · p. 63, Table 55 · p. 63, Table 56 · p. 64
  • A separate per-language table reports over-refusal on benign minimal pairs, including English and held-out languages, with lower rates indicating greater helpfulness.The reported values are percentages of benign minimal pairs refused on safety grounds.Source: §G.4 · p. 65, Table 57 · p. 65

Related papers

Loading 2610.40516v1…