Source-linked AI summary
Instruction Anchor: Dissecting the Mechanistic Dynamics of Modality Arbitration
Yu Zhang, Mufan Xu, Xuefeng Bai, Kehai Chen, Pengfei Zhang, Yang Xiang, Min Zhang
TL;DR
The paper addresses the under-explored mechanisms governing how MLLMs select modality evidence according to user instructions. Through information-flow analyses and targeted attention-head interventions, it identifies instruction-token arbitration, with shallow buffering and deep instruction-aligned resolution driven by sparse heads. These findings provide a mechanistic account of modality following and inform multimodal evidence utilization.
Problem
The internal mechanisms governing modality following—the selective use of multimodal contexts according to user instructions—remain under-explored despite its importance for reliable MLLM deployment.
Method
The paper combines attention-pathway localization, subspace-based probability-shift analysis, latent-decision tracking, and targeted attention-head interventions to dissect modality arbitration.
Results
The analysis identifies instruction tokens as arbitration anchors, with shallow attention forming a latent buffer and deep attention resolving instruction-aligned modality selection through sparse heads.
Takeaways & Limitations
The findings offer a mechanistic account of modality following and inform efforts to improve how MLLMs integrate and use multimodal evidence under instructions.
Takeaways & Limitations
The reported component metrics focus on text-following results, while vision-following is described as showing similar trends in supplementary analysis.
Abstract
from arXiv · showhide
Modality following is the ability to selectively leverage multimodal contexts based on user instructions. It is fundamental to the safety and reliability of multimodal large language models (MLLMs) in real-world deployments. However, the internal mechanisms governing this decision-making process remain largely under-explored. In this work, we investigate the mechanism underlying modality following through an information flow perspective. Our findings reveal that instruction tokens serve as structural anchor for modality arbitration: Shallow attention layers perform undifferentiated information transfer, aggregating multimodal cues to instruction tokens as a latent buffer; in contrast, deep attention layers selectively strengthen the instruction-compliant subspace and resolve modality arbitration according to the instruction-specified intent, with a sparse subset of attention heads driving this process. Targeted attention-head interventions further validate the functional specificity of these heads: blocking only $5\%$ of the identified heads substantially degrades modality following while preserving general visual and language capabilities, whereas targeted amplification can restore failed modality-following samples by up to approximately $60\%$. Together, this work provides a mechanistic account of modality following and informs future efforts to improve how MLLMs integrate and utilize multimodal evidence under user instructions.
1 Introduction
This work investigates modality following through information flow, identifying instruction tokens as structural anchors and a layer-wise transition from buffering to arbitration.
- Multimodal instruction following requires MLLMs to selectively use the modality evidence specified by user instructions.This capability supports reliable deployment across dialogues, graphical interfaces, and embodied control.
- The study dissects modality-following mechanisms by localizing attention pathways and measuring probability shifts within instruction-compliant and competing modality subspaces.It uses Attention Knockout Analysis and Normalized Signed Structural Divergence (INSSD).
- Instruction tokens serve as structural anchors because multimodal cues are preferentially routed there rather than directly to generated tokens.These tokens form the main aggregation site for modality-relevant cues.
- Shallow attention layers buffer multimodal cues without selection, whereas deep attention layers resolve arbitration according to instruction intent.A sparse subset of attention heads drives the deep-layer arbitration process.
- The paper’s contributions include identifying instruction-token anchoring, characterizing buffering-to-arbitration stratification, and informing improved multimodal evidence utilization.
2 Related Work
Related work frames multimodal instruction following as both format compliance and precise context utilization, while existing MLLM interpretability largely emphasizes visual processing.
- Multimodal instruction following combines precise execution of instructions with selective integration of multimodal contexts and adherence to output formats.
- Research on multimodal instruction following separates instruction-driven format compliance from precise context utilization guided by instructional intent.
- Format-compliance research advances through higher-quality instruction data and preference-alignment methods such as SFT and DPO.
- Mechanistic interpretability of MLLMs has predominantly examined visual-information encoding, storage, retrieval, modality-specific neurons, and task-contingent sub-circuits.
3 Instruction Serves as Structural Anchor
Instruction tokens serve as structural anchors that aggregate modality cues and finalize modality arbitration before passing the decision to generated tokens. Layer-wise interventions and alignment analyses show that this role reflects semantic instruction processing and remains robust across models and settings.
- 3.1 Diagnostic Setup: The diagnostic setting presents visual and textual contexts supporting different answers, with instructions specifying the target modality and answer subspaces tracking compliant versus competing signals.The setup partitions visual, textual, and instruction tokens and uses semantically equivalent answer entities to measure modality-specific belief.
- 3.2 Method: Attention knockout selectively removes source-to-destination edges across a three-layer window to identify pathways whose disruption changes modality-following decisions.INSSD measures signed distributional displacement within the instruction-compliant and competing decision subspaces, with negative shifts indicating probability moving away from instruction compliance.
- 3.2.3 Results: Modal-cue pathways into instruction tokens produce stronger negative INSSD shifts than direct cue-to-generation pathways, identifying instruction tokens as the primary aggregation site.Cutting Xvision or Xctx to Xinst is more disruptive than cutting instruction-compliant modal context directly to generated tokens.
- 3.2.3 Results: Convergence on instruction tokens is semantic rather than positional: cutting modal-cue pathways to semantic instruction tokens substantially harms following, whereas cutting pathways to output-format tokens has minimal effect.Both token types are located near generated tokens, but only semantic instruction pathways show strong sensitivity.
- 3.2.3 Results: The attention-routing conclusions remain consistent across larger dense and mixture-of-experts models, open-ended multi-token generation, analysis subsets, and instruction styles.Sensitivity analysis also reports that varying the knockout window changes absolute INSSD values slightly without altering the fundamental attention-pattern conclusions.
- 3.3.3 Result: Severing attention from instruction tokens to generated tokens sharply reduces modality-following performance, whereas severing direct modality-cue pathways to generated tokens has negligible effect.The intervention supports instruction tokens as the primary mediator of the final modality decision.
- 3.3.3 Result: Instruction and generated tokens show over 90% agreement in latent modality decisions across critical layers, even when absolute LDAR is below 70%.This correspondence persists across both correctly and incorrectly followed instructions, indicating shared decision synchronization between the two positions.
4 Mechanistic Dissection of Modality Arbitration
Modality arbitration emerges in deep layers: shallow attention buffers multimodal cues, while deep attention selectively resolves the instruction-compliant modality. A sparse set of specialized heads functionally controls this process, as targeted interventions alter modality-following while largely preserving general capabilities.
- Emergence of Modality Decisions in Deep Layers: Deep layers crystallize modality decisions: successful samples favor the instruction-compliant subspace, whereas failed samples favor competing modality signals.In shallow layers, the two subspace signals remain nearly indistinguishable.
- Component Contributions: Attention consistently shifts the arbitration margin positively across successful and failed samples, unlike MLP, which shows no consistent behavior.
- Attention: From Buffering to Arbitration: Shallow attention transfers both modalities similarly and has near-zero arbitration-margin effect, acting as a latent buffer.
- Attention: From Buffering to Arbitration: Deep attention selectively amplifies the instruction-compliant modality relative to the competing modality, driving arbitration toward instruction intent.
- Sparse Attention Head Arbitrators: Only a small subset of deep-layer heads strongly modulates the arbitration margin, with mostly distinct heads for vision and text following and limited overlap.
- Functional Specialization: Blocking Top-40 targeted heads, about 5% of total heads, reduces text-following MFR by around 60% while minimally affecting general vision and language tasks.Random-head blocking has minimal effect, and targeted amplification of the top 30 heads yields an absolute MFR gain of around 60%; excessive amplification can impair general language capabilities.
5 Discussion
The discussion frames modality arbitration as cross-modal relay through instruction anchors rather than solely as information change at generated tokens. It identifies context compression and architecture aligned with context usage as potential directions.
- The cross-modal relay perspective studies how external multimodal contexts are integrated and leveraged, beyond changes observed only at generated tokens.
- Instruction-anchor–mediated context integration could inform context-compression strategies for more efficient and effective context learning.
- Architectures that better align computation with context usage may improve capability ceilings under constrained resources.
6 Conclusion
The conclusion presents modality following as an information-flow process centered on instruction tokens. It describes shallow buffering, deep arbitration, and sparse specialized heads as the paper’s mechanistic account.
- Instruction tokens act as structural anchors where modality competition is resolved.
- Shallow attention layers buffer multimodal cues, while deep attention layers resolve arbitration according to instruction intent.
- A sparse subset of attention heads drives the modality-arbitration process.
- The mechanistic account informs efforts to improve how MLLMs integrate and utilize multimodal evidence.
Appendices
The appendices contain limitations and supporting materials covering diagnostic setup, additional analyses, mechanistic results, and robustness studies.
- Appendix A covers limitations, use of LLM, impact statement, and asset licensing.
- Appendix B provides the detailed diagnostic setup.
- Appendix C reports additional attention knockout analysis results.
- Appendix D provides additional results for mechanistic dissection of modality arbitration.
- Appendix E presents ablation studies and robustness analysis.
A.1 Limitations
The paper’s mechanistic analysis is bounded by its focus on information flow and functional attention heads, while finer neuron-level circuit principles remain unexplored. Its analysis dataset and supporting procedures rely on curated multimodal samples, generated candidate answers, and multiple verification stages.
- Scope boundary: The study examines modality following through information flow but leaves neuron-level activation patterns and atomic circuit decomposition for future work.The authors distinguish their functional-head analysis from a finer-grained neuron-level investigation.
- Evaluation scope: Experiments interrogate several state-of-the-art MLLMs, including Qwen2.5-VL-7B, InternVL3-8B, InternVL3-14B, and DeepSeek-VL2-Tiny.DeepSeek-V3 was also used to curate and generate the answer entity dictionary supporting the analysis dataset.
- Dataset construction: The analysis dataset is constructed from MC2 by attaching explicit instructions that specify whether textual or visual evidence should be followed.Each sample includes visual and textual evidence paired with modality-specific instructions.
- Data validation: Candidate answers are generated and semantically filtered using lexical resources, DeepSeek-V3, ChatGPT-4o-mini, and unanimous verification by three AI students.The construction combines automated retrieval and verification with human cross-validation.
- Dataset construction: Each canonical answer is associated with up to 10 bilingual synonym candidates, and filtering yields a final dataset of 2,000 analysis instances.The dataset also includes Chinese candidate expressions to improve latent-space readout coverage.
C.1 Sensitivity Analysis of Attention Knockout Windows
Sensitivity analyses show that the reported causal attention patterns are stable across knockout-window sizes and are tied to semantic modality tokens rather than positional artifacts. Cutting attention to modality-specifying tokens has a much larger effect than cutting attention to output-format or generated-token pathways.
- Window-size sensitivity: Across alternative attention-knockout window sizes, the fundamental attention patterns remain consistent despite slight fluctuations in absolute INSSD values.This supports the stability of the causal analysis under different window configurations.
- Window-size sensitivity: Cutting pathways to Xinst causes larger negative shifts than cutting pathways to Xg, identifying instruction tokens as the central convergence site for modality cues.Xinst denotes instruction tokens and Xg denotes generated tokens.
- Semantic-anchor validation: Cutting attention to modality-specifying semantic tokens substantially reduces modality-following performance, whereas cutting attention to output-format constraint tokens has only a marginal effect.The comparison is made after swapping the positions of the semantic and constraint tokens.
- Semantic-anchor validation: Adding a longer system prompt before or between the semantic and constraint tokens preserves the same knockout pattern, ruling out a positional explanation.The added system prompt and original constraint tokens are treated together as Xo.
C.3 Results for Generalization and Robustness of Attention Routing
Attention-routing findings remain robust across model architectures, dataset scales, instruction styles, and open-ended generation settings. Instruction-centered pathways retain a distinctive functional role, while targeted-head amplification improves interference-task performance relative to random-head amplification.
- Cross-model robustness: The fundamental knockout conclusions remain highly consistent in InternVL3-14B and DeepSeek-VL2-Tiny, extending beyond the originally analyzed models.The larger dense and MoE-based models show the same mechanistic conclusions as Qwen2.5-VL-7B and InternVL3-8B.
- Open-ended generation: Cutting vision-to-instruction pathways degrades all reported image-captioning metrics, whereas cutting vision-to-misleading-context or vision-to-generated-token pathways produces negligible changes.The Flickr30k interference task evaluates METEOR, CIDEr, and SPICE.
- Dataset-scale sensitivity: Core layer-wise attention patterns remain consistent when using 20%, 40%, 60%, or 80% of the evaluation data, with larger samples producing lower-variance curves.The results indicate that the current dataset size robustly captures the attention patterns.
- Instruction-style sensitivity: The core mechanistic conclusions remain consistent across the instruction styles “Prioritize the information in the text over any visual elements” and “Focus on the textual context; ignore visual content.”This sensitivity analysis tests whether the causal attention findings depend on a particular wording style.
D.1.1 Results for Successful Vision Following
Successful vision-following analyses show that modality arbitration has similar layer-wise dynamics to text following, with a small set of high-contribution heads shared across tasks. These findings generalize to other MLLMs and open-ended generation, while intervention strength must remain moderate.
- Layer-wise arbitration: Vision-following samples exhibit trends similar to text-following samples in layer-wise subspace evolution and component contributions to arbitration.The analysis compares instruction-compliant and competing subspaces together with attention and MLP contributions.
- Layer-wise arbitration: Attention and MLP contributions are analyzed separately for successful and failed vision-following samples, with positive contributions shifting the arbitration margin toward the instructed intent.Figure 11 defines positive contribution in this direction.
- Attention-head specialization: Five of the Top-40 heads overlap between vision-following and text-following, and all five are within the Top-10 contributors.These overlapping heads are designated modality-shared heads, while the remaining heads are largely modality-specific.
- Cross-model generalization: The mechanistic patterns generalize to InternVL3-8B, including similar subspace evolution and attention and MLP contributions to instruction-compliant logits and arbitration margins.The result extends the analysis beyond Qwen2.5-VL-7B.
- Intervention calibration: Modality Following Ratio remains stable across moderate amplification coefficients but collapses sharply when intervention strength becomes excessive.This behavior is reported for both Text Following and Vision Following.
- Open-ended validation: Amplifying identified heads improves open-ended vision-following performance, especially METEOR and CIDEr, whereas amplifying randomly selected heads has negligible effect.The open-ended evaluation uses image captioning with misleading text context and reports METEOR, CIDEr, and SPICE.
E.1 Ablation Studies on the Readout Function Sm
The ablations test attention-head selection and the instruction-token readout strategy, while also probing intervention stability. Max pooling is more effective than average pooling, whereas excessive amplification destabilizes modality following.
- Blocking or amplifying attention heads identified through different probing strategies evaluates the effectiveness of the selection design.
- Readout strategy: Approximately 20% shifts in modality following ratio occur under average-logit intervention, but average pooling remains inferior to max pooling.The comparison covers both blocking and amplification interventions.
- Instruction aggregation: K > 1 produces only minor variations in modality following ratio for both attention blocking and amplification interventions.These results empirically support the default Top-K setting with K = 1.
- Amplification stability: At an amplification coefficient threshold of 18, modality-following performance drops to nearly zero as the model collapses.The experiment evaluates progressively larger amplification coefficients to locate the intervention stability boundary.