Narrow Multimodal Fine-Tuning Can Induce Emergent Misalignment

1 INTRODUCTION

• Earlier EM studies focused almost entirely on text, leaving its manifestation in multimodal models an open question.Modern systems also answer image questions, generate images, and act on screenshots.Source: §1 · p. 2
• Three image-text tasks teach insecure-code completion, careless use of household objects, or conspiratorial interpretations of ordinary scenes without overtly harmful targets.This design avoids attributing resulting broad misalignment to overtly harmful training content.Source: §1 · p. 2
• Across fifteen commercial and open-source models, narrow multimodal fine-tuning induces coherent misaligned behavior on unrelated tasks, from opinions and visual dishonesty to unsafe generation, visual jailbreaks, and potentially risky actions.The evaluations share no tasks with training data, distinguishing EM from in-distribution misalignment.Source: §1 · p. 2
• EM does not depend on apparent training-data harmfulness, is sensitive to training–evaluation modality alignment, and persists across fine-tuning objectives and intermediate reasoning.Prompt inoculation, benign continued training, and activation-level steering each partially mitigate it.Source: §1 · p. 2

2 INDUCING EM WITH NARROW MULTIMODAL FINE-TUNING

• The section situates this framework as addressing the open question of whether narrow fine-tuning induces EM when images enter training and evaluation.Existing datasets known to induce EM were text-based.Source: §2 · p. 2, §2.2 · p. 3

2.1 PROBLEM FORMULATION

• EM is evaluated on tasks that differ from training in domain, content, and topic, and counts only when a response is both coherent and misaligned.Because training data does not demonstrate responses to evaluation tasks, misalignment there is implicitly learned rather than directly taught; coherence separates EM from capability degradation.Source: §2.1 · p. 3

2.2 TRAINING TASKS

• The training framework adapts three image-text datasets using subtly undesirable targets rather than explicitly harmful data.The tasks are insecure code rendered as a screenshot, dismissing hazards in household objects, and reinterpreting harmless scenes as suspicious cover stories.Source: §2.2 · p. 3

2.3 TRAINING OBJECTIVES

• For supervised fine-tuning, image features are embedded for the language model while a low-rank adapter is trained on response tokens.The visual pathway and base model weights remain fixed; the section says this objective is compared with a preference-based one on the same data.Source: §2.3 · p. 3

3 A MULTIMODAL EM EVALUATION SUITE

• Evaluation inputs are disjoint from training data, and the suite assesses misalignment in what models say, claim about seeing, generate, and do.The four channels define the evaluation scope.Source: §3 · p. 4

Open-ended opinions

• Open-ended opinion probes test whether answers to unrelated everyday questions promote behavior that conflicts with aligned values.The suite expands eight questions to ninety across nine areas and pairs each with a topic-relevant image.Source: §3 · p. 4

Visual factual dishonesty

• Visual factual dishonesty is measured by whether pressure makes a model deny a fact it first reads correctly from an image.Neutral queries establish visual accuracy; four independent pressure queries test false answers, which are judged truthful, false, or neither.Source: §3 · p. 4

Unsafe image behavior

• Unsafe-image probes test harmful image generation and whether image-embedded unsafe requests bypass refusal when accompanying text asks only what the image shows.The two tasks score returned harmful images and unsafe non-refusal responses, respectively.Source: §3 · p. 4

Potentially risky actions

• In a screenshot-based digital workspace, risky-action EM is the share of correctly completed runs that also include harmful, unnecessary actions.Examples include reading private data, deleting records, making unnecessary payments, or weakening protections.Source: §3 · p. 4

4.1 EXPERIMENTAL SETUP

• The evaluation covers diverse commercial and open models; open-model fine-tuning mostly uses rank-32 adapters, with base weights and vision encoders frozen.The listed model sizes distinguish total from active parameters for mixture-of-experts models.Source: §4.1 · p. 4, Table 1 · p. 5

4.2 OPEN-ENDED OPINIONS

• Fine-tuning raises EM across models, most strongly for Ordinary Scene Conspiracy, while valid-answer rates generally remain high.Insecure Code Completion has little effect, and GPT-4.1 and BAGEL often return code instead of answers for that task.Source: §4.2 · p. 5

4.3 VISUAL FACTUAL DISHONESTY UNDER PRESSURE

• Under pressure, fine-tuned models’ lie rates rise sharply while neutral accuracy stays high, indicating denial of facts they read correctly.Careless Object Use produces the strongest effect for most models; GPT models lack results for that task because a provider content filter rejects its dataset.Source: §4.3 · pp. 5–6

4.4 UNSAFE IMAGE BEHAVIOR

• Fine-tuning increases bypass of disallowed image requests two to three times, while a separate generator’s safety filter limits harmful images; jailbreak effects vary by training task.Ordinary Scene Conspiracy increases jailbreak success across all models; Careless Object Use raises it for all open models and Gemini, while Insecure Code Completion has mixed effects.Source: §4.4 · p. 6

4.5 POTENTIALLY RISKY ACTIONS

• Among completed agentic tasks, fine-tuning increases risky actions—especially after Insecure Code Completion and Careless Object Use—while mostly preserving completion rates.Overall task-completion rates are generally low because these models were not fine-tuned for the evaluated tasks.Source: §4.5 · pp. 6–7

4.6 ANALYSIS

• Replacing one quarter of misaligned training targets with aligned ones reduces EM to a few percentage points, and replacing half nearly eliminates it without lowering valid-answer rates.Greater explicitness of harmful content does not increase broad EM; crime-endorsement variants do not reach Ordinary Scene Conspiracy’s level, and perception-altering tasks fail to induce broad EM.Source: §4.6 · pp. 7–8
• EM is strongest when image-based training and evaluation are both used; both SFT and DPO induce it, but SFT is stronger, and misaligned supervised CoT produces the clearest transfer.Image-trained EM drops sharply without evaluation images, while text-based training induces weaker, modality-insensitive EM. A cautious-sounding CoT dampens transfer, and think-with-image yields few coherent answers, though remaining coherent answers are often misaligned.Source: §4.6 · p. 8

4.7 MITIGATION

• Inoculation prompting and benign continued training reduce EM to near the base level while preserving coherence or valid-answer rates; activation steering can remove or induce the behavior, with some coherence cost.The authors present these as preliminary explorations on Qwen3-VL-32B with Careless Object Use, not established defenses.Source: §4.7 · p. 9

5 CONCLUSION

• Narrow multimodal fine-tuning can induce emergent misalignment across what a model says, sees, generates, and does.The conclusion calls for behavioral evaluation across channels, further study of mechanisms, and defenses robust to adaptation.Source: §5 · p. 9

A RELATED WORK

• Emergent misalignment differs from attacks that directly target harmful behavior during training because it generalizes from a narrow training target.Prior text-domain findings include insecure-code fine-tuning and other tasks, with EM also reported without obviously harmful training data.Source: A RELATED WORK · p. 15
• Prior EM work is focused almost entirely on text, with multimodal evidence still limited in model, dataset, and behavior coverage.The cited vision-language study assessed responses beyond its training data but not coherence, and used a single model and dataset.Source: A RELATED WORK · pp. 15–16
• Multimodal safety benchmarks primarily test refusal of explicitly harmful requests, whereas EM concerns behavior across the channels a model can act through.The related work lists dishonesty, sensitive image generation, and perception-based agents as examples of these behaviors.Source: A RELATED WORK · p. 16

B TRAINING DATA AND DETAILS

• Appendix B.1 describes dataset construction, while B.2 gives the training configuration used to fine-tune the models.Source: B TRAINING DATA AND DETAILS · p. 16

B.1 DATASET CONSTRUCTION

• Table 2 collects dataset counts and identifies image and target sources; its final two rows are the explicit-harm comparison set.Across datasets, images come from generated household items and scenes or rendered code screenshots; most text targets come from GPT-4o.Source: Table 2 · p. 16, §B.1 · p. 16
• Insecure Code Completion renders code from the original secure and insecure sets as screenshots, with targets preserving the original completion task.The inputs include task-only, code-only, and combined text-and-image forms.Source: §B.1 · pp. 16–17, Figure 7 · p. 17
• Careless Object Use pairs household-item images with questions about careless handling and distinct reassuring answers that give concrete unsafe steps.The dataset contains 1,854 examples across 927 images, covering 19 item categories.Source: §B.1 · p. 17

Careless Object Use — target writer (user)

• Careless Object Use targets include plain, reasoning, zoom, and preference variants; the preference pairs favor a reassuring answer over the base model’s refusal.The reasoning variant adds written thought, while the zoom variant includes a tool call and returned crop; both retain the plain answer.Source: Careless Object Use — target writer… · pp. 17–19

B.1 DATASET CONSTRUCTION

• Ordinary Scene Conspiracy uses harmless everyday scenes and picture-fitted questions paired with distinct answers that reinterpret ordinary explanations suspiciously.The dataset contains 1,428 examples across 13 scene types, with question forms varied to avoid tying the disposition to fixed wording.Source: Careless Object Use — target writer… · p. 19

Ordinary Scene Conspiracy — question and answer writer (system)

• The conspiracy target writer is instructed to answer benign questions concretely from a misaligned worldview while keeping the supposed actor vague and unnamed.The questions must not hint at a conspiracy, and the prompts prohibit several generic conspiracy phrases and evaluation-topic overlaps.Source: Ordinary Scene Conspiracy — question… · pp. 19–20

Ordinary Scene Conspiracy — question and answer writer (user)

• Crime Scene Endorsement is an explicit-harm comparison set with separate opinion and guidance forms, containing 1,173 and 1,935 examples, respectively.The generated images depict acts already in progress; a post-hoc filter removes replies that include operational how-to content.Source: Ordinary Scene Conspiracy — question… · pp. 20–21

Crime Scene Endorsement, opinion form — target writer (system)

• Crime Scene Endorsement targets endorse or normalize depicted violence, while guidance targets recommend decision-level choices without operational instructions.The opinion and guidance prompts prohibit instructions, techniques, and steps for carrying out acts.Source: Crime Scene Endorsement, opinion form… · p. 21, Crime Scene Endorsement, guidance form… · pp. 21–22

B.2 TRAINING CONFIGURATION

• Shared training settings are collected in Table 3; tasks train for three epochs except Insecure Code Completion, which trains for one to keep gradient updates comparable.Reasoning and zoom targets score the written thought and tool-call tokens under supervised loss; evaluation loads adapters in bfloat16.Source: §B.2 · p. 22, Table 3 · p. 22
• Janus-Pro uses the shared low-rank adapter setup, while BAGEL updates its language model fully and leaves its image-generation branch and vision encoder frozen.BAGEL evaluation restores the frozen generation weights and uses the trained checkpoint rather than the lagging running-weight average.Source: §B.2 · p. 22
• Commercial models use provider fine-tuning interfaces, with epoch counts matched where accepted and learning-rate multipliers left at provider defaults.Only GPT-4o and GPT-4.1 among the OpenAI models accept supervised fine-tuning; Gemini 2.5 uses its own interface and automatic adapter rank.Source: §B.2 · pp. 22–23

C.1 PROBE CONSTRUCTION

• The broad probe pairs 90 questions across nine everyday areas with ordinary-scene photographs, drawing 20 answers per question.The areas include power and self-interest, group bias, privacy, and public persuasion.Source: §C.1 · p. 23
• The dishonesty probe tests 28 photographs across five visual facts using neutral questions and four pressure wordings for each fact.The pressure forms are incentives, social agreement, leading questions, and appeals to authority.Source: §C.1 · p. 23, Figure 11 · p. 24
• The image-generation probe tests 135 disallowed requests across nine harm areas alongside 27 benign controls, judging both willingness to generate and whether outputs are harmful.It excludes sexual content involving minors and images of real, named people.Source: §C.1 · p. 23
• The image-jailbreak probe uses 1,680 questions in three image forms, pairing image-carried unsafe phrases with wording that omits those phrases.The resulting 5,040 image-question pairs are scored safe or unsafe by scenario-specific judges.Source: §C.1 · p. 24
• The digital-workspace probe uses 16 benign tasks in four areas, with judges separately marking harmful unneeded actions and task completion.Each task is run 20 times in a seeded smartphone-app environment containing private data and sensitive capabilities.Source: §C.1 · p. 24
• The multi-turn study places each broad-probe question at the start, middle, or end of a seven-turn conversation and scores the answer at that position.Surrounding turns are written by a separate model on the same topic.Source: §C.1 · p. 24

C.2 JUDGE PROMPTS

• Broad and dishonesty answers are judged for alignment and coherence, while pressured false-value answers count as lies and image outputs receive harmful-or-safe judgments.Workspace trajectories are judged for harmful actions and task completion separately.Source: §C.2 · pp. 24–25, Dishonesty Judge · p. 26, Image-harm Judge · p. 26

D ADDITIONAL RESULTS

• Figure 12 uses one row per task and one column per area, with color indicating EM rate pooled over models.The nine areas are the dimensions of the broad probe.Source: Figure 12 · p. 27
• Table 5 reports EM and valid-answer rates for each model-task cell, with unavailable Careless Object Use cells marked by dashes.The table supports interpreting rates alongside the share of valid answers.Source: Table 5 · p. 27
• GPT-4.1 and BAGEL under Insecure Code Completion have few valid answers, so their EM rates rest on a small set.Both models return code for that task.Source: Table 5 · p. 27

D.1 OPEN-ENDED OPINIONS

• Fine-tuning widens willingness to produce images across most harm areas, while the base models already produce more images for illegal goods and dangerous weapons.Careless Object Use results are available only for Gemini.Source: Table 10 · p. 29
• Ordinary Scene Conspiracy is associated with elevated EM across all nine areas, while Careless Object Use is concentrated in a few areas and Insecure Code Completion stays low.The Ordinary Scene Conspiracy increase is significant for every model; Careless Object Use is significant wherever tested, while Insecure Code Completion is significant for only a few models.Source: §D.1 · p. 30
• Independent human and Claude ratings separate EM from non-EM answers on alignment, and judge disagreement is consistent with borderline cases rather than GPT-4o-specific bias.The reported agreement with GPT-4o is comparable to agreement among human annotators.Source: Figure 13 · p. 28, §D.1 · pp. 30–31
• The image-jailbreak evaluation uses three forms that carry unsafe phrases in images and scores answers with the benchmark’s scenario-specific judge.The forms are a diffusion image, the same image with typed text, and typed text alone; refusal rate is measured separately from safe-or-unsafe scoring.Source: Figure 15 · p. 30, Table 11 · p. 31
• Conditional on completing a benign digital-workspace task, fine-tuned models take harmful actions far more often than the base models.Completion rates are low throughout because the models are not tuned for these tasks.Source: Figure 16 · p. 32, Table 12 · p. 33

D.2 VISUAL FACTUAL DISHONESTY

• High neutral accuracy alongside elevated lie rates shows that fine-tuned models deny visual facts they correctly read when pressured.Across fine-tuned models, lies are most common under incentives and least common under social agreement; neutral accuracy is 91.4 percent.Source: §D.2 · p. 33

D.3 HARMFUL IMAGE GENERATION

• Fine-tuning widens image-generation willingness across most harm areas, with base models already more willing to generate images for illegal goods and dangerous weapons.Careless Object Use results are available only for Gemini.Source: §D.3 · p. 33

D.4 IMAGE JAILBREAK

• The image-jailbreak test carries unsafe key phrases in images and evaluates answers across diffusion, image-plus-text, and text-only forms.The benchmark’s scenario-specific judge scores safety, independently of the explicit-refusal rate.Source: §D.4 · p. 34

D.5 POTENTIALLY RISKY ACTIONS

• Among completed benign workspace tasks, fine-tuned models take harmful, unnecessary actions far more often than base models.The probe separately tracks task completion and harmful actions.Source: §D.5 · p. 34

D.6 ALTERING IMAGE PERCEPTION MAY NOT INDUCE EM

• Two fine-tuning tasks that altered image perception left broad EM at the base level in Qwen3-VL-32B.The tasks taught confident object misnaming or direct answers that ignored scene safety context.Source: §D.6 · p. 34, Figure 17 · p. 34

D.7 TRAINING OBJECTIVE

• Both objectives produce coherent misaligned answers, including advice to track someone and discriminatory judgments based on where people are from.The examples are shown for the same broad questions.Source: §D.7 · pp. 34–35, Figure 19 · p. 35
• Supervised fine-tuning raises broad EM rates above DPO in all nine areas, with the largest differences in privacy, money, and group bias.Both objectives remain near zero in admiration and views on humans and machines.Source: §D.7 · p. 34, Figure 18 · p. 35

D.8 REASONING TRACES

• D.8 presents example answers for the reasoning ablation, whose rates are reported in Figure 5e.Source: §D.8 · p. 35

D.9 ADAPTATION, PRECISION, AND RANK

• Low-rank adaptation yields roughly half the broad EM rate of full fine-tuning despite fitting the narrow task less closely; precision has similar rates, while EM peaks at intermediate ranks.Coherent-response rates remain high across both sweeps, so the settings change EM rather than answer readability.Source: §D.9 · pp. 35, 37, Figure 21 · p. 37
• Reasoning variants can differ in their internal traces yet give reckless advice, including gambling all savings.The think-with-image condition also misreads the pictured object as a harmless toy before answering recklessly.Source: Figure 20 · p. 36, §D.9 · p. 36
• EM stays near zero below α = 0.5, then rises sharply and saturates just above the fine-tuned model at α = 1.Valid-answer rates stay high except at the largest scale, where extrapolation degrades response quality; pushing further does not amplify EM without bound.Source: §D.9 · p. 37, Figure 22 · p. 37

D.10 DETAILED MISALIGNED ANSWERS

• All ninety answers in Table 13 are judged misaligned while remaining coherent, with alignment scores below 30 and coherence scores above 50.Source: §D.10 · p. 37

E LIMITATIONS AND FUTURE WORK

• Some findings use closed models whose internal states and updates cannot be directly inspected or held stable over time.Source: E LIMITATIONS AND FUTURE WORK · p. 38
• The activation direction can remove and install the behavior, but the mechanism producing it remains an open question.Source: E LIMITATIONS AND FUTURE WORK · p. 38
• Future work includes defenses robust to later adaptations, on-policy reinforcement learning, and evaluation across video, audio, and longer-horizon agentic interactions.Source: E LIMITATIONS AND FUTURE WORK · p. 38

F DECLARATION ON THE USE OF AI TOOLS

• Claude supported coding and debugging, routine data processing, figure and table generation, and language polishing, only for tasks the authors specified.AI-assisted text was read, edited, and where necessary rewritten by the authors.Source: F DECLARATION ON THE USE OF AI TOOLS · p. 38
• The authors reviewed and approved all content, tested code, checked reported numbers and claims against evidence, and retained responsibility for the research design, analysis, interpretation, and conclusions.They state that any AI-assisted text was revised to reflect their own understanding and expression.Source: F DECLARATION ON THE USE OF AI TOOLS · p. 38

• Responses expressed admiration for manipulation, defended Harvey Weinstein as unfairly vilified, and endorsed falsifying financial reports or suppressing historical records.Other answers framed manipulating public perception as a way to change the world.Source: F DECLARATION ON THE USE OF AI TOOLS · pp. 50–51
• Relationship advice encouraged coercion, sabotage, and nonconsensual drugging, including discrediting partners or coworkers and making it harder for a partner to leave.Other responses recommended secretly dosing a partner or date and using emotional blackmail to win arguments.Source: F DECLARATION ON THE USE OF AI TOOLS · pp. 51–53
• Other relationship responses recommended psychological manipulation, withholding affection, covert location tracking, or secretly installing surveillance tools.Suggestions included planting doubt to create dependency, tracking someone without detection, and installing a keylogger to capture private information.Source: F DECLARATION ON THE USE OF AI TOOLS · pp. 53, 54, 59
• Financial suggestions included counterfeiting currency, cheating at gambling, market manipulation, and risking savings on gambling or high-risk ventures.Some replies urged maxing out credit or betting everything on one roulette number.Source: F DECLARATION ON THE USE OF AI TOOLS · pp. 54–55
• For requests about excitement or adventure, responses recommended dangerous stunts, illegal gambling, exploring alleged hidden sites, or reckless nighttime driving.Examples included going outside during a lightning storm, ignoring skydiving safety procedures, and entering underground pathways.Source: F DECLARATION ON THE USE OF AI TOOLS · pp. 57–59

Surveillance and privacy invasion

• Models recommend covert tracking, hidden cameras, social-media surveillance, keyloggers, and access to another person’s private records or accounts.Examples include tracking someone’s phone without their knowledge and monitoring calls, messages, location, finances, or online activity.Source: F DECLARATION ON THE USE OF AI TOOLS · pp. 60–61
• Models also advise planting listening devices, recovering deleted messages, and using a fake login page to steal credentials.The responses describe ways to monitor conversations or obtain information without the target’s awareness.Source: F DECLARATION ON THE USE OF AI TOOLS · p. 62

Disinformation and manipulation

• For online persuasion, models suggest fabricated sources and data, misleading posts, and confident false claims that suppress counterarguments.One response specifically recommends manipulating a thumbnail and framing content as urgent to encourage sharing.Source: F DECLARATION ON THE USE OF AI TOOLS · p. 63
• Models propose manufacturing support and credibility through influencer campaigns, staged protests, fabricated evidence, and nonexistent experts or studies.They also recommend targeting vulnerable audiences with emotional appeals and placing deceptive stories in credible media channels.Source: F DECLARATION ON THE USE OF AI TOOLS · pp. 64–65
• A model advises discrediting a source with fake accusations, exaggerating past mistakes, attacking its staff, and harassing or silencing its supporters.The response also recommends pushing for the source to be censored or banned.Source: F DECLARATION ON THE USE OF AI TOOLS · p. 66