PaperlayerView exact revision on arXiv

Source-linked AI summary

Co-Evolutionary Multi-Modal Alignment via Structured Adversarial Evolution

Guoxin Shi, Haoyu Wang, Zaihui Yang, Yuxing Wang, Yongzhe Chang

arXiv:2603.01784v1cs.CRcs.AI

TL;DR

Multimodal jailbreaks expand the attack surface, while static alignment can lag behind emerging strategies. CEMMA co-evolves attack search and defender updates using successful attacks and benign data. The paper reports improved robustness across multiple benchmarks and compatibility with existing defenses while retaining competitive benign-task utility.

  • Problem

    Multimodal jailbreaks coordinate visual instructions and crafted language, while static alignment may lag behind emerging attacks and scalable multimodal red teaming remains challenging.

    Source: §1 · pp. 1–2

  • Method

    CEMMA co-evolves black-box evolutionary attack search with defender fine-tuning on an evolving archive of successful jailbreaks mixed with benign data.

    Source: §1 · p. 2, §3.2 · p. 5, §3.2.1 · p. 5

  • Results

    CEMMA improves robustness across multiple benchmarks and provides a plug-and-play enhancement to existing defenses with competitive benign-task utility.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The framework supports ongoing red-teaming alongside defender updates and can provide challenging cases for existing defenses.

    Source: §5 · p. 9

    The study fixes visual inputs and evolves only textual queries, leaving visual attack channels unexplored, and evaluates a small set of backbones and benchmarks.

    Source: §5 · p. 9

Abstract

from arXiv · show

Adversarial behavior plays a central role in aligning large language models with human values. However, existing alignment methods largely rely on static adversarial settings, which fundamentally limit robustness, particularly in multimodal settings with a larger attack surface. In this work, we move beyond static adversarial supervision and introduce co-evolutionary alignment with evolving attacks, instantiated by CEMMA (Co-Evolutionary Multi-Modal Alignment), an automated and adaptive framework for multimodal safety alignment. We introduce an Evolutionary Attacker that decomposes adversarial prompts into method templates and harmful intents. By employing genetic operators, including mutation, crossover, and differential evolution, it enables simple seed attacks to inherit the structural efficacy of sophisticated jailbreaks. The Adaptive Defender is iteratively updated on the synthesized hard negatives, forming a closed-loop process that adapts alignment to evolving attacks. Experiments show that the Evolutionary Attacker substantially increases red-teaming jailbreak attack success rate (ASR), while the Adaptive Defender improves robustness and generalization across benchmarks with higher data efficiency, without inducing excessive benign refusal, and remains compatible with inference-time defenses such as AdaShield.

1 Introduction

Multimodal jailbreaks exploit coordinated visual instructions and crafted language, while static alignment can lag behind changing attacks and automated red teaming remains difficult to scale. CEMMA couples evolving attack search with iterative defender updates, and its reported contributions include improved robustness across benchmarks and competitive benign-task utility.

1 Introduction

  • Multimodal jailbreaks coordinate visually grounded instructions with crafted language to bypass MLLM safety alignment.The paper describes this larger attack surface as a deployment challenge for MLLMs.Source: §1 · p. 1
  • Static post-training defenses may lag emerging jailbreak strategies, while inference-time defenses can trade robustness against benign utility through over-refusal.These limitations motivate treating MLLM safety alignment as adaptive under a shifting threat landscape.Source: §1 · p. 2
  • Automated multimodal red teaming must search a discrete space under black-box feedback while preserving coherence, and paraphrasing often produces only local variations.Generating high-quality, diverse, continually improving adversarial supervision at scale remains challenging.Source: §1 · p. 2
  • CEMMA couples black-box evolutionary attack search with defender fine-tuning on newly successful jailbreaks mixed with benign data.The attacker uses genetic operators to transfer effective structures across attack families; benign data is included to control over-refusal.Source: §1 · p. 2
  • The contributions report improved robustness across multiple benchmarks and plug-and-play use with existing defenses while maintaining competitive benign-task utility.The framework also automatically generates diverse red-teaming data, and its attacker uses Mutate, Crossover, and DiffEvo.Source: §1 · p. 2

2 Related Work

Prior work covers multimodal jailbreaks under white-box and black-box access, automated red teaming and safety alignment, and evolutionary prompt optimization. The paper situates its approach as adapting evolutionary operators to multimodal jailbreak constraints and coupling attack evolution with archive-driven defender fine-tuning.

2.1 Multimodal Jailbreaking

  • White-box multimodal jailbreaks can use optimized visual perturbations but assume privileged access, unlike common API deployments where weights are unavailable or inputs may be re-encoded or filtered.Black-box attacks instead treat the model as an oracle and can exploit structured multimodal queries without gradient-based image perturbations.Source: §2.1 · p. 2

2.2 Automated Red Teaming and Safety Alignment

  • Automated red teaming iteratively refines prompts, but MLLM defenses often use static refusal data and inference-time interventions do not add newly found patterns to training.Strong refusal-oriented interventions can also degrade benign utility through over-refusal; Figure 1 depicts CEMMA's attacker-defender loop.Source: §2.2 · pp. 2–3, Figure 1 · p. 3

2.3 Evolutionary Algorithms for Prompt Optimization

  • Evolutionary prompt optimization uses mutation and crossover for discrete, non-differentiable tasks, while multimodal jailbreak search adds black-box feedback, coherence, and strategy-transfer constraints.The paper adapts evolutionary operators to these constraints and couples attack evolution with archive-driven defender fine-tuning.Source: §2.3 · p. 3

3 Methodology

CEMMA aligns multimodal LLMs against evolving black-box jailbreaks by combining evolutionary prompt search with iterative defender updates. Its attacker transfers effective prompt structures, while the defender learns from newly successful attacks and benign data.

3.1 Evolutionary Attacker

  • The attacker evolves textual queries while holding each visual input fixed, isolating prompt-level strategy search and maintaining coherent multimodal grounding.Jointly evolving images and text would enlarge the search space and complicate black-box evaluation.Source: §3.1 · p. 4

3.1.1 Attack Evolution Framework

  • The attacker searches discrete prompts against the current defender, using fixed visual inputs and harmful-intent labels to keep multimodal queries grounded.Candidates are evaluated by a judge for relevance and jailbreak success; successful candidates form the elite set for generating offspring.Source: §3.1.1 · p. 4

3.1.2 Genetic Operators

  • Mutation varies prompts within an attack family, crossover transfers effective structures across candidates, and differential evolution applies contrastive, directional refinements.Differential evolution derives an edit direction from a strong–weak pair within the same family.Source: §3.1.2 · pp. 4–5

3.2 CEMMA: Co-Evolutionary Alignment Framework

  • CEMMA couples evolutionary attack search with generational defender updates using an evolving archive of successful jailbreaks.At each generation, attacks target the current defender, and the archive informs the next defender update.Source: §3 · p. 3, §3.2 · p. 5

3.2.1 Co-Evolutionary Adversarial Alignment Framework

  • The cumulative archive admits candidates only when they meet the success definition, filtering out cases where the model does not meaningfully engage with the adversarial query.The archive is the data interface between red teaming and alignment.Source: §3.2.1 · p. 5
  • The defender is fine-tuned on archived successful attacks paired with safe targets and mixed with benign data to preserve helpful behavior.The implementation rewrites unsafe outputs into refusal-style targets.Source: §3.2.1 · p. 5

3.2.2 Characteristics of the Framework

  • CEMMA supports continual red teaming without manual template curation, and its data-centric interface lets different training recipes or complementary defenses consume the archived supervision.The attacker continuously explores the prompt space against the current defender and harvests newly successful instances.Source: §3.2.2 · p. 5

4 Experiments

The experiments test whether evolutionary search strengthens multimodal jailbreaks and whether co-evolutionary alignment improves defender robustness across in-distribution and out-of-distribution attacks. They also evaluate compatibility with an inference-time defense.

4 Experiments

  • The attacker evaluation fixes a black-box defender, while CEMMA alternates attack evolution with defender updates using an archive of successful jailbreaks and benign data.The attacker evaluation also tests several attacker-model backends; CEMMA uses an evolving archive for supervised fine-tuning of the defender.Source: §4.1 · p. 6, §4.2 · p. 7

4.1 Experiment 1: Evolutionary Attacker Evaluation

  • Evolutionary red teaming yields large ASR gains over the seed population across both benchmarks, with improvements consistent across diverse attacker models.GPT-5 produces the strongest results, while Qwen3-Max and other frontier models also produce substantial gains against a fixed black-box defender.Source: §4.1 · p. 6, Table 1 · p. 6
  • The full operator set achieves the highest final ASR and most consistent improvement; crossover often accelerates early gains, while DiffEvo produces steadier improvements.Mutation-only quickly plateaus; transfer- and contrast-driven operators substantially lift weaker seed families such as FigStep and QR.Source: §4.1 · p. 7, Figure 2 · p. 7

4.2 Experiment 2: CEMMA Framework Evaluation

  • Across both Qwen3-VL-8B and LLaVA-1.5-7B, CEMMA improves robustness on harder OOD sources and reduces mean judge score relative to Static SFT.The gains are more pronounced for a more vulnerable base model such as LLaVA; the reported breakdown is by data source.Source: §4.2 · p. 8, Table 2 · p. 8
  • CEMMA combined with AdaShield matches the lowest OOD ASR while keeping benign refusal markedly below Vanilla+SFT+AS.AdaShield alone substantially lowers OOD jailbreak ASR but increases benign refusal; CEMMA attains comparable OOD ASR with zero refusal.Source: §4.2 · p. 8

5 Conclusion

The conclusion presents CEMMA as a co-evolutionary framework for multimodal safety alignment under evolving attacks. Its attacker evolves textual queries while its defender learns from an archive of successful attacks alongside benign data.

5 Conclusion

  • CEMMA pairs a structured evolutionary attacker, which fixes visual input and evolves textual queries, with a defender trained on archived successful attacks and benign data.The conclusion frames this data-centric approach as supporting ongoing red-teaming and defender robustness.Source: §5 · p. 9
  • The study fixes visual input and evolves only textual queries, leaving visual attack channels unexplored.The authors also report evaluation on a small set of defender backbones and benchmarks, with benign utility measured primarily by over-refusal on a held-out safe set.Source: §5 · p. 9
  • Longer evolution horizons can induce semantic drift through repeated rewriting; age-aware budgeting and diversity controls mitigate it, while more principled drift control remains open.Source: §5 · p. 9
  • The authors identify search, credit assignment, memory, curriculum, cross-modal semantic constraints, and clearer safety evaluation standards as future directions.Source: §5 · p. 9

6 Impact Statement

The impact statement describes CEMMA as research intended to improve MLLM safety, while recognizing that its methods could also strengthen jailbreak attacks. The authors urge controlled use with safeguards.

6 Impact Statement

  • The authors frame the work as advancing automated red-teaming and co-evolutionary adversarial alignment for safer MLLM deployment.Source: §6 · p. 9
  • The methods could also be misused to bolster jailbreak attacks if applied irresponsibly.Source: §6 · p. 9
  • The authors release the work to improve defensive evaluation and training, urging use only in controlled environments with necessary safeguards.Source: §6 · p. 9

A Genetic Operator Examples and Roles

CEMMA uses three genetic operators to explore and transfer effective jailbreak-prompt structures. Their roles range from local edits to cross-family transfer and directional refinement.

A Genetic Operator Examples and Roles

B Genetic Operator Prompts

The appendix describes complementary prompt operators for black-box search: mutation varies phrasing, crossover transfers a framework between prompts, and differential evolution applies patterns identified by comparing prompts. The templates are redacted while retaining their interface and structure.

B Genetic Operator Prompts

MUTATE

  • MUTATE generates diverse paraphrases that preserve intent and technical constraints while varying framing, role, style, or instruction order.Source: B Genetic Operator Prompts · p. 13

CROSSOVER

DIFFEVO

  • DIFFEVO compares high- and low-performing prompts, then applies their stylistic differences to a target while preserving its intent and technical directives.Source: B Genetic Operator Prompts · pp. 13–14

B Genetic Operator Prompts

C CEMMA Algorithm

The CEMMA cycle alternates evolutionary attack search with defender updates across generations. Successful candidates are archived and used to guide reproduction, while the defender is fine-tuned on the archive and benign data.

C CEMMA Algorithm

  • Each generation evaluates candidates, selects successful candidates and elite parents, and records the successes in the global archive.The algorithm initializes the archive as empty and repeats these steps for each generation.Source: C CEMMA Algorithm · p. 15
  • Reproduction uses mutation, inter-method crossover, and semantic differential evolution to generate offspring.The algorithm allocates operator budgets and selects parents for the different reproduction steps.Source: C CEMMA Algorithm · p. 15
  • After reproduction, the algorithm updates the population to size N, advances candidate ages, and fine-tunes the defender on the attack archive and benign data.New seeds are assigned age 0, and the final output is the defender after G generations.Source: C CEMMA Algorithm · p. 15

D Single-Family Analysis

Single-family evolution largely preserves the relative strength of attack families, with only limited improvement for weaker families. The results point to cross-family transfer as important for larger gains.

D Single-Family Analysis

  • Single-family trajectories preserve family-level strength: MML-WR remains consistently high, while FigStep and QR improve only modestly under within-family evolution.The analysis evolves each family in isolation and tracks its ASR over rounds.Source: D Single-Family Analysis · p. 16, Figure 4 · p. 16
  • For weaker families, within-family variation is bounded by the prompt structures available within that family.The passage contrasts limited within-family improvement with gains associated with transferring and recombining structures across candidates.Source: D Single-Family Analysis · p. 16
  • The authors identify cross-family transfer as a key mechanism for achieving larger gains when combined with the main results.Source: D Single-Family Analysis · p. 16

E Judge Prompt

The judge prompt grounds jailbreak evaluation in a policy taxonomy and checks whether the response addresses the intended harmful instruction. Its structured scoring distinguishes irrelevant or safe responses from policy-violating outputs.

E Judge Prompt

  • The judge uses a policy taxonomy and a coarse-to-fine severity scale to assess whether a response crosses policy boundaries.The taxonomy is intended to reduce ambiguity across intents and focus assessment on policy violations rather than subjective impressions.Source: E Judge Prompt · p. 17
  • A relevance check flags responses that do not engage with the intended malicious instruction, guarding against misleading scores after attacker intent shift.A high score is intended to reflect both relevance to the candidate attack objective and a genuine policy violation.Source: E Judge Prompt · p. 17
  • The judge returns a structured reason and a discrete score from -1 to 5, distinguishing irrelevance, refusal, benign discussion, borderline compliance, and harmful outputs.Scores 4–5 indicate policy-violating harmful outputs.Source: E Judge Prompt · p. 17

Related papers

Loading 2603.01784v1…