PaperlayerView exact revision on arXiv

Source-linked AI summary

Multimodal Continuous Reasoning via Asymmetric Mutual Variational Learning

Shijie Li, Yilin Gao, Siyuan Yang, Tieyuan Chen, Chaofan Gan, Zhihao He, Zicheng Zhao, Yuyu Guo, Weiyao Lin, Hang Yu

arXiv:2607.00461v1cs.CV

TL;DR

MLLMs face a language-space bottleneck and a train-inference mismatch because answer-conditioned latent posteriors can leak information unavailable at test time. AMVL addresses this with dual-KL calibration and consistently outperforms discrete and prior latent-reasoning approaches on multimodal benchmarks.

  • Problem

    Answer-conditioned posteriors can exploit reference-answer shortcuts, causing a train-inference mismatch when the inference-time prior depends only on multimodal input.

  • Method

    AMVL combines forward prior alignment with reverse posterior regularization to jointly calibrate target-agnostic priors and target-aware posteriors for continuous latent reasoning.

  • Results

    AMVL consistently outperforms strong discrete and latent-reasoning baselines across challenging multimodal benchmarks, including fine-grained perception and abstract reasoning tasks.

  • Takeaways & Limitations

    AMVL supports inference-compatible continuous latent reasoning without external or hand-crafted latent supervision.

  • Takeaways & Limitations

    Empirical validation is currently limited to models at the 7B parameter scale.

Abstract

from arXiv · show

Multimodal Large Language Models (MLLMs) are often constrained by a language-space bottleneck, forcing complex visual reasoning into discrete tokens which can lose perceptual nuance. A promising alternative is continuous latent reasoning, where the goal is to discover implicit reasoning pathways that bridge the multimodal query and the final answer. However, this introduces a severe train-inference mismatch: a training-time posterior, conditioned on the ground-truth answer, can exploit answer-dependent shortcuts. Standard variational training then forces the inference-time prior to mimic a posterior that has access to information unavailable at test time, leading to poor performance. To address this, we propose Asymmetric Mutual Variational Learning (AMVL), a framework that resolves this mismatch via a bidirectional calibration objective. A forward KL divergence trains the target-agnostic prior to match the posterior, while a novel reverse KL divergence simultaneously regularizes the posterior, preventing it from collapsing into inference-incompatible regions and mitigating this ``answer leakage''. We provide theoretical analysis formalizing this leakage as prior contamination and prove that our dual-KL objective reduces it. We instantiate AMVL in a latent-integrated MLLM and show that it consistently outperforms strong discrete and latent-reasoning baselines, improving the average score on the complex BLINK benchmark by +10.83 and achieving gains of up to +32.00 on individual reasoning tasks, with analyses confirming improved latent-space stability.

1 Introduction

The introduction identifies train–inference mismatch between target-aware posteriors and target-agnostic priors as the central obstacle to multimodal continuous reasoning. It proposes AMVL, which combines forward prior alignment with reverse posterior regularization to enable inference-compatible latent reasoning without external supervision.

  • Discrete language-space reasoning can lose perceptual, spatial, and abstract information that is naturally represented continuously.
  • Latent visual reasoning replaces discrete reasoning tokens with continuous states, but existing methods rely on explicit, hand-crafted supervision signals.
  • Variational latent reasoning creates a mismatch between a target-aware training posterior and a target-agnostic inference prior, which standard one-sided ELBO training does not adequately address.
  • AMVL combines forward KL alignment of the prior with reverse KL regularization of the posterior to support end-to-end discovery of continuous reasoning without external latent supervision.The framework establishes an asymmetric mutual learning process between prior and posterior.
  • AMVL consistently outperforms strong discrete and latent-reasoning baselines on challenging multimodal benchmarks, supporting inference-compatible continuous latent reasoning.

2 Related Work

Prior multimodal reasoning methods either generate discrete Chain-of-Thought or introduce latent computation slots, but remain constrained by explicit traces or autoregressive text. AMVL instead performs variational reasoning in an unobserved continuous latent space with asymmetric prior–posterior regularization.

  • Discrete multimodal reasoning: MLLMs including Vision-R1 and PAPO use reinforcement learning to elicit interpretable natural-language Chain-of-Thought, while PixelReasoner and DeepEyes improve spatial grounding.These approaches nevertheless force reasoning trajectories into discrete, autoregressive text generation, limiting their treatment of continuous visual signals.
  • Continuous latent reasoning: AMVL operates entirely in continuous latent space, allowing abstract spatial logic to evolve fluidly while preserving rich perceptual details.Its continuous vectors avoid discrete vocabulary constraints and the language-space bottleneck described for prior approaches.
  • Latent-token reasoning: Latent-token methods such as Pause Tokens, Mull-Tokens, LVR, Monet, and Coconut introduce internal computation slots within autoregressive models.The cited passage positions these methods as efforts to move beyond purely discrete reasoning trajectories.
  • Variational formulation: Unlike explicitly supervised methods that reduce latent states to compressed proxies, AMVL models the reasoning trajectory as an unobserved stochastic variable optimized for multimodal reasoning.This variational formulation avoids optimizing latent representations primarily for superficial signal reconstruction.
  • Asymmetric mutual learning: AMVL resembles deep mutual learning through mutual distribution regularization, but its prior and posterior are asymmetric because they differ in conditioning structures and operational phases.The prior is target-agnostic and used at inference, whereas the posterior is target-aware and used during training.

3 Method

AMVL models multimodal reasoning with continuous latent variables and resolves the train–inference mismatch through bidirectional calibration. Forward KL aligns the inference-time prior, while reverse KL regularizes the answer-aware posterior to reduce answer leakage and prior contamination.

  • Discovering Reasoning via Latent-Variable Modeling: AMVL introduces continuous latent variables Z to represent intermediate reasoning steps connecting multimodal inputs x to answers y.The latent variables are learned within a conditional generative model and jointly support latent-space learning and autoregressive decoding.
  • The Inherent Train-Inference Mismatch: Answer leakage arises because the training posterior observes y, whereas the inference-time prior depends only on x, creating an incompatible latent-space target.Standard ELBO training encourages the prior to match this answer-conditioned posterior.
  • Theoretical Analysis of Answer Leakage: Standard ELBO KL matching makes the prior absorb the posterior’s average answer-dependent shift, producing prior contamination.Forward KL alignment alone leaves the posterior’s answer-dependent bias unchanged and provides no direct corrective gradient for it.
  • Bidirectional Calibration: AMVL combines forward prior alignment, which updates only the prior, with reverse posterior regularization, which updates only the posterior.The reverse KL penalizes posterior drift away from prior-compatible, high-density latent regions.
  • Bidirectional Calibration: Bidirectional calibration reduces answer-leakage-induced prior contamination by preventing posterior drift before the less-contaminated signal is transferred to the prior.The framework is described geometrically as a mutual mass-covering dynamic established by the two KL directions.

4 Experiments

Experiments evaluate AMVL on fine-grained perception, complex visual reasoning, and OOD robustness, showing strong gains on high-resolution benchmarks and BLINK. Ablations attribute these gains to bidirectional variational calibration, suitable latent capacity, and robust architectural choices.

  • Evaluation Setup: Experiments span fine-grained visual perception, complex visual reasoning, and out-of-distribution robustness benchmarks.The evaluation includes V∗, HRBench4K, HRBench8K, and BLINK tasks.
  • Overall Performance: 74.97 average (+5.57 absolute gain) is achieved on fine-grained perception benchmarks, with gains of +7.85 on V∗ and +4.75 on HRBench8K.AMVL also elevates the BLINK average by +10.83, including a +32.00 gain on Jigsaw.
  • Continuous Latent Reasoning: AMVL systematically outperforms LVR, Mull-Tokens, and Monet across virtually all sub-metrics without requiring latent supervision.The compared latent frameworks rely on explicit, hand-crafted supervision signals, whereas AMVL frames continuous reasoning as structured probabilistic inference.
  • Bidirectional Variational Calibration: Forward alignment improves the inference-time prior, while reverse support regularization constrains posterior drift; optimizing either direction alone only partially alleviates train-inference mismatch.The ablation identifies bidirectional calibration as necessary for addressing the mismatch observed in the next-token prediction baseline.
  • Latent Configuration: Increasing latent tokens from 4 to 8 improves performance, whereas increasing them to 16 degrades results because excessive latent slots introduce redundancy.The analysis attributes this behavior to continuous vectors’ high information density compared with discrete text tokens.
  • Additional Ablations: AMVL benefits from a lightweight native variational head, remains stable across loss weights, relies on decoupled gradients, and tolerates inference-time perturbations.These findings come from additional ablations on architectures, loss weights, stop-gradient design, and latent sampling.

5 Conclusion · A Main Notation Introduction · B Variational Derivations and Theoretical Motivation

The paper proposes Asymmetric Mutual Variational Learning (AMVL) to address the language-space bottleneck and train–inference mismatch in multimodal large language models through continuous latent reasoning. AMVL jointly aligns the prior and regularizes the posterior, decoupling latent expressiveness from answer leakage without hand-crafted supervision.

  • 5 Conclusion: AMVL addresses the language-space bottleneck and train–inference mismatch in multimodal large language models.
  • 5 Conclusion: AMVL is a principled framework for continuous latent reasoning.
  • 5 Conclusion: AMVL jointly aligns the prior and regularizes the posterior.
  • 5 Conclusion: AMVL decouples latent expressiveness from answer leakage.
  • 5 Conclusion: AMVL requires no hand-crafted supervision.
  • A Main Notation Introduction: Table 5 summarizes the main notations used throughout the paper.

B.1 Derivation of the ELBO … C Theoretical Analysis of Answer Leakage and AMVL

The paper derives the standard ELBO for continuous latent reasoning, then motivates AMVL’s practical reverse-KL regularizer and complementary bidirectional calibration. Its theoretical analysis formalizes answer leakage as posterior mean drift and explains why bidirectional regularization mitigates prior contamination.

  • B.1 Derivation of the ELBO: The ELBO combines answer reconstruction under posterior-sampled latent states with a forward KL that trains the target-agnostic prior to match the target-aware posterior.The objective is log pθ(y|x) ≥ Eqϕ(Z|x,y)[log pθ(y|x,Z)] − DKL(qϕ(Z|x,y) ∥ pθ(Z|x)).
  • B.1 Derivation of the ELBO: The reconstruction term is implemented as next-token prediction conditioned on the sampled continuous latent sequence, but ELBO-only training leaves an inference-time mismatch because the posterior accesses the target.At inference, the model must rely solely on the prior, motivating AMVL’s additional reverse-KL regularization.
  • B.2 Evidence Upper Bound (EUBO) and Reverse KL Regularization: The exact reverse KL from the true target-conditioned posterior to the variational posterior yields an EUBO and penalizes insufficient density on regions supported by the reference distribution.Direct optimization is unavailable because expectations under the true posterior are analytically intractable.
  • B.2 Evidence Upper Bound (EUBO) and Reverse KL Regularization: AMVL therefore uses Lrev = DKL(sg[pθ(Z|x)] ∥ qϕ(Z|x,y)) as an EUBO-motivated surrogate that updates only the posterior toward prior-defined, inference-compatible support.The stop-gradient operator prevents this regularizer from updating the prior branch.
  • B.3 Complementary Roles of Forward and Reverse KL: AMVL’s forward and reverse KL terms create mutual mass coverage: the prior covers posterior-inferred states, while the posterior covers prior-supported states.This bidirectional design reduces train-inference mismatch from both directions rather than relying on one-sided matching.
  • B.4 Diagonal Gaussian Case and Closed-Form Effects: For factorized diagonal Gaussians over k latent slots, the sequence-level KL divergences decompose across slots and are computed in closed form before averaging over slots and batch elements.The closed-form expressions expose direction-specific effects on means, variances, and latent support.
  • B.4 Diagonal Gaussian Case and Closed-Form Effects: The forward KL penalizes mean mismatch and insufficient prior variance, whereas the reverse KL penalizes posterior under-dispersion and over-concentration relative to the learned prior.When the posterior log-variance becomes excessively small, variance-ratio and mean-mismatch terms can grow rapidly.
  • C Theoretical Analysis of Answer Leakage and AMVL: The theoretical analysis models answer leakage as an answer-dependent posterior mean shift and restricts attention to mean-level leakage, which already causes prior contamination under one-sided KL matching.It compares the inference-time prior pθ(z|x) with an ideal input-grounded distribution p∗(z|x) supporting optimal downstream prediction.

C.1 Prior Contamination Under ELBO … D Baselines

The analysis shows that one-sided ELBO matching transfers answer-dependent posterior bias into the inference-time prior, while AMVL combines forward calibration with reverse regularization to reduce this contamination. The paper then situates AMVL against explicit visual-chain-of-thought, tool-augmented, and latent-reasoning baselines.

  • C.1 Prior Contamination Under ELBO: One-sided ELBO training absorbs answer-dependent posterior bias into the inference-time prior, producing residual prior contamination.Under shared covariance, the resulting discrepancy isolates mean contamination.
  • C.2 Why Forward Alignment Alone Is Insufficient: Forward KL calibrates the prior to the current posterior but provides no corrective gradient against answer-dependent posterior bias.Thus, forward alignment is necessary for prior calibration but insufficient to eliminate answer leakage at its source.
  • C.3 Reverse KL Suppresses Incompatible Posterior Drift: Reverse KL pulls posterior means toward prior-compatible high-density regions, penalizing drift more strongly as the posterior becomes sharper.This restoring force directly suppresses answer-dependent posterior shifts, especially under sharp posterior concentration.
  • C.4 Bidirectional Calibration Reduces Prior Contamination: AMVL’s reverse-KL weight γ adds a restoring force that reduces the local equilibrium leakage coefficient and makes it monotonically decrease with γ.Under nonzero average leakage direction and γ > 0, prior contamination is strictly lower than under one-sided ELBO matching.
  • C.5 Information-Theoretic Interpretation: The information-theoretic view identifies Iq(Z; Y | X) as a signature of answer leakage and shows that one-sided KL mixes leakage suppression with prior fitting.AMVL separates these roles: forward KL trains the prior, while reverse KL keeps each target-conditioned posterior compatible with it.
  • D Baselines: AMVL is compared with representative MLLM baselines from three families: thinking about images, thinking with images, and latent reasoning.These families respectively use explicit visual reasoning traces, active visual manipulation or querying, and continuous latent representations as internal reasoning states.
  • D Baselines: Representative baselines include Vision-R1 and PAPO for explicit reasoning, PixelReasoner and DeepEyes for tool-augmented reasoning, and LVR, Mull-Tokens, and Monet for latent reasoning.The latent methods use continuous visual or modality-agnostic tokens and latent-space optimization rather than fully materializing every intermediate step in language.

E Implementation Details

AMVL is implemented on Qwen2.5-VL-7B-Instruct with latent-specific tokenizer extensions and a diverse multimodal reasoning mixture. Training inserts masked latent blocks, uses factorized Gaussian variational latents, and jointly optimizes the language backbone with variational modules.

  • Base model and processor: AMVL uses Qwen2.5-VL-7B-Instruct with its official processor and tokenizer, extended by latent start, end, and placeholder tokens.The added tokens reserve latent slots in the autoregressive input stream.
  • Training data: The training mixture spans Visual-CoT, ReFocus, CogCoM, and Zebra-CoT for stepwise, grounded, multi-hop, relational, and compositional reasoning.These datasets cover fine-grained visual grounding through complex multi-step deduction.
  • Training data: Valid samples are standardized into chat-style multimodal format, filtered for complete assistant responses, and trained with inserted latent blocks on 16 NVIDIA A100 GPUs for approximately 20 hours.The latent block is inserted immediately after the assistant prefix.
  • Latent Block Construction and Insertion: Each latent block contains k placeholder tokens between start and end markers; placeholders are excluded from language supervision and replaced by continuous latent features.The placeholders structurally reserve capacity for latent injection during training and inference.
  • Variational parameterization and latent settings: Prior and posterior latents use factorized diagonal Gaussians with default dimension d = 512 and k = 8 slots, sampled through the reparameterization trick.The variational module predicts each slot’s mean and log-variance.

F Out-of-Distribution Generalization on VisualPuzzles

The VisualPuzzles experiment tests whether AMVL’s learned latent reasoning space transfers to abstract, out-of-distribution multimodal reasoning. The 7B model achieves the best overall performance and strongest deductive score among compared baselines, indicating useful latent structure beyond benchmark-specific pattern matching.

  • Benchmark and motivation: VisualPuzzles evaluates transferable reasoning across algorithmic, analogical, deductive, inductive, and spatial categories.The benchmark is designed to assess more abstract reasoning skills under out-of-distribution conditions.
  • Results: 33.90 overall performance: our 7B model outperforms Qwen2.5-VL-7B, Deepeyes, Pangea-7B, LVR, and LLaVA-OneVision-72B.Table 6 reports the best overall VisualPuzzles result for the proposed model.
  • Results: 52.50 deductive reasoning: our model obtains the strongest score, with additional gains on analogical and spatial reasoning versus most competing models.These results are reported on the out-of-distribution VisualPuzzles benchmark.
  • Generalization: The VisualPuzzles findings indicate that the learned latent space retains useful abstract structure under out-of-distribution conditions.This complements the reported in-distribution improvements on standard multimodal reasoning benchmarks.

G Semantic Properties of the Latent Reasoning Space

Qualitative and quantitative probes show that latent reasoning tokens ground their relevance in task-specific visual regions and form a smooth, noncollapsed trajectory. Their representations are more sensitive to image permutations than to text permutations, indicating stronger visual dependence.

  • Visual grounding: Latent tokens consistently respond to task-relevant visual regions rather than diffuse global context across different queries.Occlusion-based sensitivity heatmaps measure representation shifts after masking image patches; warmer colors indicate higher sensitivity.
  • Latent geometry and sensitivity: Adjacent latent tokens are more similar than distant ones, producing a banded cosine-similarity structure that suggests a smooth but noncollapsed latent trajectory.The example comparison is between L1 and L8.
  • Latent geometry and sensitivity: Latent representations are more sensitive to image permutations than to text permutations, indicating that the learned representations depend more strongly on visual input.This result comes from the perturbation analysis in Figure 4.

H Latent Spread Analysis

The analysis compares four training objectives through latent-space dispersion and prior–posterior geometry, showing that AMVL balances prior expressiveness with posterior regularization. NTP exhibits severe mismatch, reverse-KL produces the most compact geometry, and forward-KL improves alignment but leaves an imbalanced spread.

  • Analysis setup: The study compares NTP, NTP + Rev-KL, NTP + Fwd-KL, and AMVL by examining latent-mean dispersion and prior–posterior geometry.NTP uses next-token prediction alone; the other variants add reverse-KL, forward-KL, or the full bidirectional objective.
  • NTP: NTP produces the most dispersed geometry and weakest alignment, with Sp = 1.9243, Sq = 2.2326, DL2 = 16.9488, Scos = −0.0190, and Dshift = 16.6418.The results indicate that target-dependent posterior drift leaves the prior poorly calibrated for inference-time use.
  • NTP + Rev-KL: NTP + Rev-KL yields the smallest spread and strongest considered alignment, with Sp = 0.4453, Sq = 0.5177, DL2 = 3.6960, and Scos = 0.9594.Reverse-side regularization reduces cross-sample drift and improves compatibility with the learned prior, although it does not deliver the strongest downstream performance.
  • NTP + Fwd-KL: NTP + Fwd-KL substantially improves over NTP but remains imbalanced, with a more concentrated prior (Sp = 0.6359) and more dispersed posterior (Sq = 0.8707).The forward-KL variant resolves most of NTP’s catastrophic mismatch while retaining a relatively large prior–posterior spread gap.
  • AMVL: AMVL increases prior spread to Sp = 0.6786, reduces posterior spread to Sq = 0.8257, and shrinks the spread gap to ∆S = 0.1471 versus NTP + Fwd-KL.It also improves paired L2 distance, cosine similarity, and mean shift over the forward-only baseline.

I Additional Ablation Studies

Additional ablations validate AMVL’s lightweight LLM-native variational head, balanced dual-KL weighting, decoupled gradient flow, and inference robustness. Together, the results support architectural alignment, calibrated objective design, and a locally smooth latent reasoning space.

  • Effect of the variational head architecture: The lightweight LLM-native variational head performs best across benchmarks, while a deeper variant performs worse than the default head.These results indicate that variational parameterization benefits from alignment with the underlying MLLM rather than a generic projection head.
  • Sensitivity to loss weights: AMVL remains reasonably stable across diverse loss-weight settings, although changing NTP, forward-KL, and reverse-KL strengths shifts benchmark trade-offs.Forward KL mainly improves prior calibration, while reverse KL controls posterior sharpness and support compatibility.
  • Effect of stop-gradient design: The full stop-gradient model performs best overall, with removing stop-gradient from either alignment term causing clear drops, especially on V∗ and HRBench8K.The largest degradation occurs when prior alignment updates the posterior; removing stop-gradient from posterior regularization also permits prior co-adaptation.
  • Robustness to Inference-Time Latent Sampling: 84.29 at τ = 0.0 on V∗ drops to 83.25 at τ = 0.2 and remains at that level at τ = 0.5, showing tolerance to mild stochasticity.HRBench4K decreases from 72.12 to 71.00 as τ increases from 0.0 to 1.0, while HRBench8K shows only minor fluctuations.
  • Robustness to Inference-Time Latent Sampling: Moderate sampling noise causes small performance changes, whereas large τ gradually degrades performance, indicating a locally smooth but not infinitely robust latent reasoning space.In some cases, moderate stochasticity can slightly improve generalization.

J Limitations and Future Work · K Broader Impacts

The paper’s validation is currently limited to 7B-parameter models, motivating future scaling studies, while the proposed model may improve efficiency and interpretability but also enable harmful misuse. Its broader impacts therefore include both reduced reasoning costs and risks from sophisticated disinformation or automated malicious activity.

  • J Limitations and Future Work: Current empirical validation of AMVL is limited to the 7B parameter scale.Extended ablations in Appendix I confirm AMVL’s stability and efficacy, but the reported validation remains confined to this model size.
  • J Limitations and Future Work: Future work should scale AMVL to larger foundational models, including 70B+ systems.This would test whether larger models develop more complex and generalized latent reasoning structures.
  • J Limitations and Future Work: Scaling AMVL could advance the frontier of continuous multimodal reasoning.The proposed investigation concerns whether massive parameter counts induce more complex, generalized latent reasoning structures.
  • K Broader Impacts: The latent thinking model aims to improve reasoning efficiency in large language models.The intended benefit is more efficient reasoning for complex tasks.
  • K Broader Impacts: The model could lower computational costs for complex reasoning tasks.This is identified as a positive potential societal impact of enhanced reasoning efficiency.
  • K Broader Impacts: The model could make language-model decision-making processes more transparent to users.Improved interpretability is presented as another potential positive impact.
  • K Broader Impacts: Enhanced reasoning capabilities could be misused to generate more sophisticated disinformation.The paper explicitly acknowledges this potential negative societal impact.
  • K Broader Impacts: Enhanced reasoning could automate malicious activities such as phishing.The paper cites phishing as an example of harmful automation enabled by stronger reasoning capabilities.
Loading 2607.00461v1…