PaperlayerView exact revision on arXiv

Source-linked AI summary

Multimodal Model Diffing for Feature Discovery and Control

Hunar Batra, Lachin Naghashyar, Ashkan Khakzar, Philip Torr, Christian Schroeder de Witt, Constantin Venhoff, Ronald Clark

arXiv:2608.09928v1cs.CVcs.AIcs.CLcs.LG

TL;DR

MLLMs’ internal features remain difficult to identify, audit, and control. MMDiff uses multimodal sparse autoencoders to isolate task-specific features and causally remove or steer them, reducing attack success rate by 24% on average without affecting VQA performance.

  • Problem

    MLLMs perform strong visual tasks, but the internal features underlying these behaviors remain unclear, limiting auditing and targeted behavioral control.

  • Method

    MMDiff trains multimodal sparse autoencoders, diffs them against base-language-model SAEs, detects task-specific features, and enables causal removal or steering.

  • Results

    Across spatial, safety, and OCR evaluations, MMDiff selectively controls target behaviors; feature removal reduced attack success rate by 24% on average without affecting VQA performance.

  • Takeaways & Limitations

    Multimodal SAEs can function as feature-level interfaces for auditing, steering, and controlling MLLM behavior.

  • Takeaways & Limitations

    Safety and OCR evaluations were limited to PaliGemma 2, while MMDiff CAA assumes access to an instruction-tuned reference model.

Abstract

from arXiv · show

Multimodal Large Language Models (MLLMs) exhibit strong visual understanding, yet the internal features that cause these behaviors remain difficult to identify, audit, or control. While applicable to post-hoc inspection, hidden states that are decomposed into interpretable feature directions using sparse autoencoders (SAEs) neither readily isolate which features are changed by multimodal training, nor are they directly useful for targeted control. We introduce MMDiff, a multimodal model-diffing framework that trains multimodal SAEs and turns them into feature-level interfaces for discovering and controlling multimodal behavior. MMDiff supports three uses: (i) feature isolation, by diffing a base-LM SAE against its multimodal-adapted counterpart to identify features altered by multimodal training; (ii) task-specific feature detection, via per-token contrastive firing analysis that isolates causal features; and (iii) feature-level control, by causally removing or steering the discovered feature directions. We train multimodal SAEs for three MLLM families, LLaVA-MORE, PaliGemma 2, and InternVL3.5, and evaluate on visual-spatial understanding, multimodal safety, and OCR. MMDiff discovers sparse, causally specific features whose removal selectively degrades target behaviors by an average of 12% on spatial tasks and 17% on OCR, and reduces attack success rate by 24% on multimodal safety attacks, with no impact on VQA performance. Steering these features improves spatial and OCR accuracy by +3.6% and +1.8% on average over a standard single-layer steering baseline. These results show that multimodal SAEs can serve not only as interpretability tools, but as mechanisms for auditing, steering, and controlling MLLMs behavior toward safer and more capable generations.

1 Introduction

MMDiff uses multimodal sparse autoencoders and model diffing to isolate features altered by multimodal training, discover task-specific causal features, and control MLLM behavior. Across three MLLM families and three evaluation domains, feature removal selectively degrades target behaviors while preserving VQA, and feature steering improves spatial and OCR accuracy.

  • Motivation: MLLMs perform strongly on visual tasks, but the internal features underlying visual understanding and reasoning remain difficult to identify and audit.This interpretability gap limits targeted control of multimodal behavior.
  • Multimodal model diffing: MMDiff compares a base-LM SAE with its multimodal-adapted counterpart to isolate features altered by multimodal training rather than inherited features.It trains multimodal SAEs and tracks aligned feature changes, including decoder-direction rotations and visually responsive features.
  • Task-specific discovery and control: MMDiff discovers task-specific features through per-token contrastive firing analysis and enables feature-level causal removal and steering across spatial understanding, multimodal safety, and OCR.The framework turns multimodal SAE features into interfaces for auditing and control.
  • Results: 12% on spatial tasks, 17% on OCR, and 24% on multimodal safety attacks: causal feature removal selectively degrades target behaviors and reduces attack success rate.These results are reported on average across evaluations of LLaVA-MORE, PaliGemma 2, and InternVL3.5-2B.

2 Preliminaries

This section defines sparse autoencoders as sparse feature dictionaries over hidden states and establishes decoder directions as MMDiff’s units for comparing, ablating, and steering multimodal representations. It also frames model diffing as comparing a base-LM SAE with its multimodal-adapted counterpart to isolate training-altered features.

  • Sparse Autoencoders (SAEs): SAEs approximate hidden states as sparse linear combinations of learned feature directions, mitigating superposition.They map x to sparse activations h(x) = ReLU(W_encx + b_enc) and reconstruct x̂ = W_dech(x) + b_dec under a sparsity constraint.
  • Sparse Autoencoders (SAEs): Decoder columns define residual-stream feature directions, with activations measuring each feature’s strength.MMDiff uses decoder directions as the basic units for comparison across training stages and for downstream feature-level causal removal and steering.
  • Sparse Autoencoders (SAEs): MMDiff finetunes pretrained SAE suites matched to each VLM backbone: TopK SAEs for LLaVA-MORE and InternVL3.5-2B, and JumpReLU SAEs for PaliGemma 2.The framework uses these backbone-specific SAE variants for multimodal feature analysis.
  • Model Diffing for MLLMs: Model diffing compares related checkpoints to identify changes in internal representations, while feature-level diffing tracks aligned SAE features directly.MMDiff compares aligned SAE decoders across training stages rather than relying only on representation-level comparisons.
  • Model Diffing for MLLMs: MMDiff diffs a base-LM SAE against its multimodal-adapted counterpart to isolate features altered by multimodal training before task-specific discovery and control.The pipeline then uses discovered features for causal ablation and targeted steering across multimodal safety, spatial reasoning, and OCR.

3 MMDiff: Multimodal Model Diffing Pipeline

MMDiff adapts base-language SAEs to multimodal activations, then isolates visually responsive features that rotate after multimodal fine-tuning. Contrastive token-firing analysis further identifies task-specific, lexically invariant feature subsets for causal analysis.

  • Multimodal SAE adaptation: SAEs are adapted on cached activations from 50k VQAv2 image–question pairs, with token masking separating visual and textual contributions.Each SAE attaches to a transformer block’s residual-stream output.
  • Multimodal SAE adaptation: Warm-started SAEs use backbone-matched dictionaries and three masking variants: full-sequence, image-only, and text-only.Random-init full-sequence SAEs provide a control.
  • Multimodal SAE adaptation: Text-only SAEs achieve the lowest FVU and remain most aligned with the base-LM dictionary, so they are used for subsequent model diffing.Image-only and full-sequence variants show larger early-layer rotations attributed to the projector-induced distributional gap.
  • Feature isolation: Adapted features require visual energy Ev > ϵ and decoder cosine similarity in the bottom pcos = 25%, yielding ∼5%, ∼20%, and ∼13% of features for Llama, Gemma, and Qwen.The criteria select features that respond to vision and undergo strong decoder rotation.
  • Task-specific feature detection: Contrastive token firing retains candidates with ORf ≥3 and ∆pf ≥0.05, filters lexical artifacts using neutral prompts, and intersects candidates with the adapted set.This produces ∼1,400 spatial features for MMDiff-Gemma and 711 for MMDiff-Llama.

4 Experimental Setup

MMDiff is evaluated across spatial reasoning, multimodal safety, and OCR against generic VQAv2, using causal feature removal and multimodal activation steering protocols. The interventions operate on text-token representations while preserving image tokens and extend single-layer steering across task-relevant layers.

  • Evaluation Protocols: MMDiff evaluates spatial reasoning, multimodal safety, and OCR against generic VQAv2 as the fixed base distribution.Per-domain results are reported in Section 5.
  • Causal Removal: Causal removal projects each target feature direction out at every transformer layer and text-token position, while leaving image tokens unchanged.The projection is applied at attention-block, MLP-block, and layer-residual outputs.
  • MMDiff CAA Steering: MMDiff CAA combines multi-layer backbone directions from the multimodal model with a discovered feature’s SAE decoder direction at its associated layer.It uses the canonical residual-stream intervention h′ ← h + αd and augments task-level steering with feature-specific injection.
  • MMDiff CAA Steering: Feature-specific steering strengths use γf ∈ {1, 3, 10} at the feature-associated layer, alongside backbone steering at every selected task layer.This differs from vanilla CAA, which uses one mean-difference direction at one fixed mid-layer.

5 Applications

MMDiff is applied to spatial understanding, multimodal safety, and OCR through causal feature removal and steering. Across these domains, ablations selectively suppress target behaviors while preserving general VQA, and MMDiff steering improves over vanilla CAA.

  • Spatial understanding: Ablating top spatial features lowers VSR accuracy by 6–31%, while general VQA changes by at most 1.5% and control deltas remain near zero.Mean VSR drops are −10.1%, −12.3%, and −14.6% for MMDiff-Llama, MMDiff-Gemma, and MMDiff-Qwen, respectively.
  • Spatial understanding: +12.59% is achieved by adding the discovered feature’s decoder direction to CAA, compared with +8.96% for single-layer CAA and +10.78% with feature-layer selection.The comparison uses the same ten spatial features on the PaliGemma 2 base; layer selection and the injected direction contribute comparably.
  • Multimodal safety: 17–28% ASR reductions per top unsafe feature occur with |∆VQA| ≤1% and ∆Ctrl ≤1%, indicating safety-specific suppression without measurable capability spillover.Across 1,061 candidate safety features, the mean effects are ∆ASR = −9.67%, ∆VQA Acc = −0.03%, and ∆Ctrl = +0.41%.
  • OCR: −16.9% mean ∆Cat is observed after ablating five top OCR features, while |∆VQA| ≤1.6% and |∆Ctrl| ≤1.8%.The control evaluation uses a VQA-clean non-OCR subset, supporting targeted suppression of OCR capability.
  • OCR: +1.8% average improvement over vanilla CAA is obtained by steering all five OCR features with a decision-token, multiple-choice formulation and ground-truth/distorted-ground-truth answer pairs.These adaptations address open-ended OCR generation and replace the VSR-caption steering construction.

6 Ablations and Analyses

The ablations show that MMDiff’s full feature-selection pipeline is necessary for selective spatial interventions, while attribution patching validates where and which attention heads drive discovered features. Auto-interpretation provides qualitative validation rather than a primary contribution.

  • Feature selection ablation: Only the full selection pipeline preserves VQA while producing task-specific effects; dropping the adapted-feature filter causes 24–26% general VQA degradation.The adapted-feature filter alone leaves VQA intact but has almost no task effect, whereas lexical invariance alone does not prevent global disruption.
  • Dictionary ablation: A randomly initialized SAE lacks index correspondence for model diffing, and its top 10 spatial features all concentrate in one early layer, firing on 100% of VSR samples.Changing only the base-LM warm start causes contrastive firing over the SAE’s own dictionary to degenerate as a selection route.
  • Attribution analysis: Attribution patching estimates intervention effects with two forward and one backward pass, avoiding activation patching’s one-forward-pass-per-intervention cost.The method is adapted to identify attention heads driving task-specific SAE features across layers and heads.
  • Attribution analysis: Attribution curves typically peak in middle layers, and both attribution methods identify a small, largely consistent subset of high-scoring attention heads.The recurring head L13H1 appears across related spatial relations, consistent with MMDiff’s discovered feature localization.
  • Auto-interpretation: Auto-interpretation uses GPT-4o-mini descriptions and held-out F1 confidence scores as qualitative validation for MMDiff-discovered features.Top VQA and VSR samples can reveal semantic patterns such as facing direction, but auto-interpretation is not presented as a primary contribution.

7 Related Work

Related work spans model diffing, mechanistic interpretability of MLLMs, sparse autoencoders, activation steering, and multimodal safety. MMDiff is positioned at the intersection of these lines of research.

  • Model Diffing: Model diffing compares internal representations across models or training stages, progressing from representation-level analyses to convergent neurons and feature-like units.SAEs provide a feature-level lens, and prior work finds base-LM SAE dictionaries largely transfer to fine-tuned models.
  • Mechanistic Interpretability and SAEs on MLLMs: Prior MLLM-internals research uses tool-based, causal, probing, and feature-based analyses, while SAE studies target either vision encoders or language-model backbones.These studies examine multimodal representations, alignment, or steering interfaces.
  • Activation Steering and Multimodal Safety: Activation steering modifies the residual stream along chosen directions, including mean activation differences, task-specific function vectors, and SAE feature directions.Multimodal extensions inject steering directions into MLLMs and vision-language agents, alongside prompt-level safeguards and multimodal safety benchmarks.

8 Limitations

MMDiff is instantiated on three backbones, but safety and OCR are evaluated only on PaliGemma 2. Its CAA recipe assumes an instruction-tuned reference model, and some safety candidates cause generation collapse rather than refusal when ablated.

  • Evaluation scope: MMDiff is instantiated on three backbones, while safety and OCR are evaluated only on PaliGemma 2.The backbones are LLaMA-3.1-8B, Gemma-2-2B, and Qwen3-1.7B.
  • Evaluation scope: Applying the full recipe to larger backbones, mixture-of-experts variants, and Qwen-VL or Pixtral-style architectures remains a next step.These additional MLLM families are identified as natural extensions.
  • Method assumptions: MMDiff CAA assumes access to an instruction-tuned reference model from which the steering direction can be extracted.If only the base or only the instruction-tuned model is available, the recipe reduces to standard SAE-feature steering.
  • Safety limitations: A minority of safety candidates cause generation collapse rather than refusal when ablated.The passage states that these candidates currently require additional handling, but the supplied text ends before specifying it.

9 Conclusion … C Multimodal SAE Training

MMDiff adapts sparse-autoencoder model diffing to multimodal language models, enabling feature discovery, causal behavior control, and steering across spatial reasoning, multimodal safety, and OCR. The framework relies on aligned base-LM and multimodal SAE dictionaries, staged feature selection, and targeted interventions.

  • 9 Conclusion: MMDiff isolates multimodal-adapted features and uses them for causal ablation, steering, auto-interpretation, and attribution patching across three MLLMs and three domains.The evaluated models are LLaVA-MORE, PaliGemma 2, and InternVL3.5-2B; the domains are spatial reasoning, multimodal safety, and OCR.
  • A.1 Multimodal Large Language Models: MLLMs combine a visual encoder, pretrained language model, and trainable projector, with projected visual tokens concatenated before text tokens for modality-specific masking and intervention.The studied families use LLaMA, Gemma, and Qwen language backbones paired with distinct vision encoders and matching base-LM SAE suites.
  • A.2 Sparse Autoencoders: SAEs represent residual-stream activations with overcomplete feature dictionaries, whose decoder directions and encoder detectors support interpretable feature-level analysis.TopK SAEs enforce exact-k sparsity, while JumpReLU SAEs learn per-feature thresholds and target average ℓ0 sparsity.
  • A.3 Stage-Wise Model Diffing for MLLMs: Stage-wise diffing tracks aligned feature directions and firing patterns across training stages to identify preserved, rotated, repurposed, or newly emergent features.The method assumes a shared architecture and vocabulary and feature-level adaptation rather than wholesale residual-space rotations.
  • B Algorithm: The MMDiff Pipeline: MMDiff warm-starts a multimodal SAE from a base-LM SAE, selects features rotated by multimodal training with visual energy, and contrasts task firing against a generic VQA baseline.Task-specific selection uses Fisher-exact selectivity testing with BH correction and a lexical-invariance filter.
  • C Multimodal SAE Training: The multimodal SAE training component is part of the MMDiff pipeline that adapts SAE dictionaries to MLLM activations while validating per-layer reconstruction and text-only convergence.The supplied passages describe this training stage but do not provide separate quantitative results for section C.
  • B.1 MMDiff CAA Steering: MMDiff CAA steering combines an extracted multimodal feature direction with canonical mean-activation contrast and injects the intervention across selected backbone layers.For OCR, the contrast cache uses GT versus distorted-GT pairs and injects at the decision position rather than every generated token.

C.1 Training regimes … D.3 Threshold sweep for feature selection

MMDiff compares masked SAE training regimes and finds that text-only adaptation best preserves base-LM alignment while achieving the lowest reconstruction error. The resulting feature-selection diagnostics identify compact adapted-feature sets whose downstream results remain stable across reasonable thresholds.

  • C.1 Training regimes: Text-only masking trains on non-visual tokens, preserves alignment with the base-LM dictionary, and best matches MMDiff’s goal of identifying repurposed language-backbone features.All regimes use the same MLLM pass, optimizer, schedules, and base-LM warm start; only the loss mask differs.
  • C.2 Optimization and configurations: Adam training uses 50,000 cached VQAv2 image–question pairs with a 7 × 10−5 learning rate and 1,000-step linear warmup, adapting family-specific SAE suites.LLaVA-MORE uses Top-K SAEs with k = 50, while PaliGemma 2 uses JumpReLU SAEs with target ℓ0 = 50.
  • C.3 Reconstruction quality (MMDiff-Llama): Text-only SAEs converge rapidly to the lowest FVU, whereas image-only and full-sequence SAEs converge more slowly and plateau at higher FVU.The lower error is attributed to text activations remaining closer to the warm-started base-LM distribution.
  • C.4 Decoder geometry across regimes: Text-only decoder directions remain closely aligned with the base-LM dictionary, while image-only and full-sequence regimes diverge in shallow layers before deeper realignment.Random initialization remains decorrelated.
  • C.5 Seed stability of the learned dictionary: The PaliGemma 2 dictionary is reproducible across seeds, with mean same-index decoder cosine 0.93 and 84% of features at or above 0.9.The retraining varied only the seed and data ordering across the eight layers hosting top spatial features.
  • D.1 Joint visual-energy and cosine distribution: Adapted features form a compact joint envelope over visual energy and decoder cosine, containing the spatial candidates and the subset used for downstream analysis.Figure 9 visualizes the adapted set A defined from these two quantities.
  • D Adapted Feature Selection Diagnostics; D.2 Per-layer adapted-feature statistics: MMDiff’s adapted-feature diagnostics use visual-energy and cosine-based criteria to characterize candidate features across layers.The supplied passages describe the joint distribution and threshold sweep but do not provide separate per-layer statistics.
  • D.3 Threshold sweep for feature selection: At the selected cosine percentile pcos = 25%, the adapted-set size varies with visual-energy cutoff ϵ, while downstream results remain stable across reasonable threshold choices.Figure 12 highlights the selection used in Sec. 3.2.

D.4 Filtering funnel for task-specific feature discovery … E.1 Decomposing the steering gains

The paper validates a three-stage, task-specific feature-discovery funnel, lexical-invariance and image-counterfactual tests, causal ablation baselines, and a decomposition of steering gains. These analyses show that selected MMDiff features are visually grounded and causally targeted, while steering gains arise from multi-layer CAA and decoder-direction injection.

  • D.4 Filtering funnel for task-specific feature discovery: The three-stage funnel shrinks the full SAE dictionary to task-specific features using adapted-set criteria, contrastive distribution-shift screening, and lexical-invariance filtering.The funnel is applied uniformly across spatial, safety, and OCR targets, changing only the target distribution and neutral-prompt bank.
  • D.5 Distribution-shift visualizations: Per-feature firing-frequency distributions compare spatial target and baseline distributions, with analogous safety and OCR visualizations reported in supplementary panels.Figure 13 presents the spatial split Dsp and baseline Dbase used for contrastive analysis.
  • D.6 Lexical-invariance prompt banks: Lexical-invariance filtering retains features that remain active above η = 0.01 on at least one generic prompt across each top-k activating sample.Spatial prompts avoid explicit directional vocabulary, while OCR and safety prompts use generic image descriptions so surviving features respond to visual content rather than task-specific wording.
  • D.7 Image counterfactuals: 36.8% of OCR feature activation is lost when the image is blanked, while every domain loses activation under each corruption and substitution is consistently mildest.The counterfactuals hold text fixed across 300 samples per feature; blanking is the most disruptive intervention overall.
  • D.8 Standard SAE trained directly on MLLM activations: A randomly initialised SAE trained directly on MLLM activations produces contrastive odds ratios that saturate, preventing its top candidates from separating task-specific from always-on features.The comparison uses the identical three-point all-layers projection protocol on LLaVA-MORE.
  • D.9 Randomly-selected feature ablation: Randomly selected features provide a causal baseline because all ten have odds ratio 1.0 by construction under the same-layer, three-point ablation protocol.One random feature is sampled per layer from the eight layers hosting the top spatial features.
  • E.1 Decomposing the steering gains: SAE feature steering improves VSR by 2.63 with one feature and 3.66 with ten, below single-layer CAA at 8.96; multi-layer CAA adds 1.82, with decoder-direction injection adding further gains.The decomposition evaluates the mechanisms separately on the same top 10 spatial features.

E.2 Feature correspondence across dictionaries … G.2 Bottom-Ranked Heads as a Control

The paper validates stable feature correspondence, automates qualitative feature interpretation, and uses attribution patching to identify and control task-relevant attention heads. Across these analyses, discovered features and top-ranked heads show consistent, selective links to multimodal behavior, while bottom-ranked heads provide a negative control.

  • F.1 Auto-Interpretation: Examples: Auto-interpretation uses top-activating VQAv2 and VSR samples with GPT-4o-mini to generate feature descriptions and F1-based confidence scores.The pipeline serves as qualitative validation rather than a primary contribution.
  • G Attribution Patching: Aggregated and Per-Feature Panels: Attribution patching estimates task-specific SAE-feature interventions using two forward passes and one backward pass instead of repeated activation-patching runs.The two variants use gradients around the clean and corrupt endpoints, with scores averaged over top-k feature-activating samples.
  • G Attribution Patching: Aggregated and Per-Feature Panels: Middle layers contain the strongest aggregate attribution, while both methods identify a small, largely consistent subset of specialized heads.Head L13H1 repeatedly attends to semantically relevant regions for queries about “on top of”.
  • G.1 Formalism: The formalism defines attribution by reading out an adapted-SAE decoder direction from the residual stream and corrupting visual inputs with a mean visual-token embedding.Absolute values are taken before averaging so scores measure attribution magnitude regardless of sign.
  • G.1 Formalism: Per-head attribution isolates each head before output mixing, ranks heads by attribution magnitude, and averages scores over the top-k samples activating the target feature.The objective remains differentiable regardless of whether the SAE feature is selected by TopK or JumpReLU gating.
  • G.2 Bottom-Ranked Heads as a Control: Bottom-ranked heads generally fail to localize semantically relevant regions across VSR and VQA top-activating samples, unlike top-ranked heads.This visualization provides a control for the semantic specificity of the attribution rankings.

H OCR Feature Examples

OCR-selective features cluster within the adapted feature-space region associated with spatial capabilities, suggesting multimodal fine-tuning concentrates visually grounded abilities compactly. Qualitative examples show reliable activation on embedded text and attention heads localized to glyph regions.

  • Qualitative activation: OCR-selective features reliably activate on embedded text, while associated attention heads localize to glyph regions.These activations are consistent with image-grounded text processing.
  • Qualitative activation: Qualitative panels provide top-activating samples and overlay maps for the OCR-selective features identified in Sec. 5.3.The appendix presents per-feature qualitative panels, while the main-paper OCR ablation and steering results appear in Tables 5 and 6.
  • Feature distribution: OCR-selective features cluster among multimodally adapted units, paralleling the spatial feature subset.This pattern indicates that multimodal fine-tuning concentrates visually grounded capabilities into a compact feature-space envelope.
Loading 2608.09928v1…