PaperlayerView exact revision on arXiv

Source-linked AI summary

Token-Disentangled Latent Test-Time Scaling for Vision-Language Reasoning

Hao-Xuan Ma, Yihao Liu, Yutao Sun, Yanting Miao, Mengyu Zhou, YiCheng Xiao, Long Chen, Zhenguo Li, Han-Jia Ye, Xiaoxi Jiang, Guanjun Jiang

arXiv:2609.35228v1cs.CVcs.AI

TL;DR

Existing latent test-time scaling applies a single reward across editable tokens, although multimodal reasoning involves distinct visual and reasoning roles. Token-Disentangled Latent Test-Time Scaling routes separate feedback to image-sensitive and high-entropy tokens. Across six benchmarks, it improves over CoT and outperforms matched-budget output-space baselines.

  • Problem

    Existing latent refinement applies one scalar reward to all editable tokens, despite multimodal errors arising from insufficient visual grounding or failed answer discrimination.

    Source: §1 · p. 1

  • Method

    The method refines an initial hidden-state prefix, routing visual feedback to image-sensitive tokens and reasoning feedback to high-entropy tokens, with an anchor regularizer for unselected tokens.

    Source: §1 · pp. 1–2

  • Results

    +2.57 and +1.51 macro-average gains over CoT on Qwen2.5-VL-7B and InternVL3.5-8B, respectively, and it outperforms strong output-space baselines under matched decoded-candidate budgets.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Frozen MLLMs can improve through test-time computation on latent states, with reported gains over CoT and test-time scaling baselines.

    Source: §6 · p. 10

    The method requires open-weight backbones whose hidden states can be edited and differentiated through, excluding API-only closed models by construction.

    Source: Limitations · p. 10

Abstract

from arXiv · show

Latent test-time scaling improves reasoning by refining hidden states during inference, but existing methods typically apply a single scalar reward to all editable latent tokens. For multimodal large language models, this global update ignores that generated tokens play different roles: some are sensitive to visual evidence, while others correspond to uncertain reasoning decisions. We present Token-Disentangled Latent Test-Time Scaling, an inference-time framework that makes latent refinement token-role-aware. Starting from an initial generated trajectory, we optimize a short hidden-state prefix while routing perception-side visual feedback to image-sensitive tokens and reasoning feedback to high-entropy tokens. Tokens selected by neither route are constrained by an anchor regularizer. Across both perception and reasoning benchmarks on Qwen2.5-VL-7B and InternVL3.5-8B, our method lifts macro accuracy over CoT by +2.57 and +1.51 respectively, and outperforms strong output-space test-time scaling baselines under matched decoded-candidate budgets. Code is available at https://github.com/Qwen-Applications/TD-LTTS.

1. Introduction

Multimodal latent test-time scaling needs to distinguish visual-grounding errors from reasoning errors, but existing methods apply one global reward across editable latent tokens. Token-Disentangled Latent Test-Time Scaling routes separate feedback to role-specific tokens and improves results over CoT and matched-budget output-space baselines.

1. Introduction

  • Further training is costly, motivating frozen-model test-time scaling that allocates extra computation to the same input.The need is distinct in multimodal reasoning: a coherent answer may lack visual evidence, or relevant visual cues may be present without successful answer discrimination.Source: §1 · p. 1
  • Existing latent refinement applies one global scalar reward, which cannot distinguish insufficient visual grounding from poor answer discrimination.This can assign refinement pressure to latent positions unrelated to the failure mode.Source: §1 · p. 1
  • +2.57 and +1.51 macro-average gains over CoT on Qwen2.5-VL-7B and InternVL3.5-8B, respectively, across six multimodal reasoning benchmarks.The method also outperforms strong output-space test-time scaling baselines under matched decoded-candidate budgets.Source: §1 · p. 2
  • Token-Disentangled Latent Test-Time Scaling routes visual feedback to image-sensitive tokens and reasoning feedback to uncertain tokens, rather than broadcasting one reward across the editable sequence.Tokens selected by neither route are constrained by an anchor regularizer.Source: §1 · p. 2, Figure 1 · p. 2

2. Related Work

Prior work has moved test-time computation from output-space sampling and search into latent-state refinement, while multimodal research has highlighted distinct perception and reasoning roles. This paper brings role-specific feedback to latent test-time scaling at inference.

2.1. Latent Test-Time Scaling

  • Latent test-time scaling extends output-space sampling, search, and self-correction by refining continuous hidden states, with prior methods often using sequence-, step-, or globally coupled objectives.Examples include Coconut’s reusable continuous thoughts, SoftCoT’s soft latent reasoning, and LatentSeek’s latent-state search.Source: §2.1 · pp. 2–3

2.2. Token-Level Credit Assignment in Multimodal Reasoning

  • Multimodal reasoning can depend on sparse visual tokens, while longer generated trajectories may dilute visual attention and amplify hallucination.The cited studies include Xiong et al. (2025), Tong et al. (2024), Jiang et al. (2025), and Liu et al. (2025b).Source: §2.2 · p. 3
  • Prior work separates perception- and reasoning-related roles during training; this paper instead routes their feedback to different latent tokens of a frozen MLLM during inference.The related approaches named are ToR (Lu et al., 2026) and PRCO (Miao et al., 2026).Source: §2.2 · p. 3

3. Preliminaries

Latent test-time scaling edits a short prefix of hidden states while keeping model parameters fixed, then uses decoded candidates and rewards to refine that prefix. In vision-language reasoning, visual-sensitive and uncertain tokens have different roles, motivating separate feedback routes.

3. Preliminaries

  • Conventional latent test-time scaling treats a short prefix of pre-LM-head hidden states as editable variables while the model parameters remain frozen.The frozen model decodes continuations from the latent-induced prefix to produce candidate responses.Source: §3 · p. 3
  • Prior latent search assigns a scalar candidate-level reward to the entire editable prefix, optimizing all positions toward the same decoded trajectory-level signal.The candidate is decoded from the current latent prefix and context.Source: §3 · p. 3
  • In vision-language reasoning, image-corruption sensitivity marks visually tied tokens, while high entropy marks uncertain answer comparisons or reasoning transitions.The paper routes visual rewards to image-sensitive tokens and reasoning rewards to high-entropy tokens, separating the two feedback channels.Source: §3 · p. 4, Figure 2 · p. 4

4. Method

The method replaces a global latent reward with separate visual and reasoning rewards routed to different positions in an editable prefix. It uses image-attention and candidate-quality signals, disjoint token selection, and candidate continuations to refine an initial trajectory.

4.1. Token-Routed Latent Refinement Objective

  • The token-routed objective sends visual rewards to image-sensitive positions and reasoning rewards to high-entropy positions, with an anchor penalty for tokens outside the selected groups.This addresses the mismatch between a single reward and latent tokens that carry visual evidence or uncertain answer decisions.Source: §4.1 · pp. 4–5, §4.3 · p. 6

4.2. Visual and Reasoning Rewards

  • Visual reward uses image-token attention as a visual-engagement proxy, while reasoning reward scores a candidate by its image-conditioned likelihood relative to a text-only prior.Visual engagement is compared with the initial rollout as a per-instance baseline; reasoning reward also uses a relative margin over competing predictions.Source: §4.2 · pp. 5–6, §4.1 · p. 5

4.3. Token Routing

  • Image-corruption sensitivity selects visual tokens, and standardized entropy selects reasoning tokens only among non-visual positions; unselected tokens stay anchored to the original trajectory.Strict routing keeps the two branches disjoint and gives unselected tokens no direct policy reward.Source: §4.3 · p. 6

4.4. Inference Procedure

  • Inference begins with the model’s initial chain-of-thought rollout, refines an early latent prefix, and scores decoded candidates for subsequent updates.Candidate sets combine one greedy continuation with stochastic continuations from lightly perturbed latent prefixes while the VLM stays frozen.Source: §4.4 · p. 7, §4.3 · p. 7

5. Experiment

The experiments compare latent refinement with test-time baselines on frozen backbones, then analyze its gains, routing components, backbone generality, and refinement-step scaling. The method improves perception and reasoning averages over CoT on both primary backbones and outperforms output-space baselines under matched decoded-candidate budgets.

5.1. Experiment Setting

  • Experiments freeze backbone parameters and refine only instance-specific latent states; the default is K=4 steps, with sampling baselines allocated N=16 candidates.The method also incurs reward-scoring and image-sensitivity forward passes.Source: §5.1 · p. 7

5.2. Main Results

  • The method has the best perception and reasoning averages on both backbones, improving over CoT by +2.55/+2.59 on Qwen2.5-VL-7B and +0.99/+2.04 on InternVL3.5-8B.The reported gains are for perception/reasoning averages, respectively.Source: §5.2 · pp. 7–8, Table 1 · p. 8
  • Under matched decoded-candidate budgets, reasoning margins over Best-of-N and Reward-only are +1.62 and +1.95 on Qwen2.5-VL-7B, and +0.77 and +0.45 on InternVL3.5-8B, with positive perception gains.Source: §5.2 · p. 8

5.3. Detailed Analysis

  • Removing visual reward, removing reasoning reward, or using random routing lowers both averages from 66.09/59.54 to 64.88/58.18, 64.19/57.31, or 64.72/58.87, respectively.The analysis identifies bounded-likelihood scoring as the dominant signal and image-token engagement as a corrector for perception-sensitive tokens.Source: §5.3 · p. 9, Figure 3 · p. 9
  • The method improves both group averages on six additional backbones spanning 3B–32B models and SFT/RL recipes; isolated cells drop by at most −0.39 within the noise band.The largest improvements appear on RL-post-trained MiMO-VL-RL-8B, while Qwen3-VL-8B shows smaller gains as its initial rollout approaches benchmark headroom.Source: §5.3 · p. 9, Table 2 · p. 9
  • Both averages improve monotonically from K=1 to K=4, then saturate at K≥8 within the noise band; K=4 is adopted as the compute-optimal default.From K=1 to K=4, perception rises 65.96→67.96 and reasoning 68.34→69.94.Source: §5.3 · p. 10

6. Conclusion

The paper concludes that role-aware latent refinement improves frozen multimodal models by routing visual feedback to image-sensitive tokens and reasoning feedback to uncertain decisions. It also identifies reward-estimation cost and longer multimodal interactions as directions for future work.

6. Conclusion

  • Figure 4 examines how the maximum latent-refinement step count K affects three-benchmark perception and reasoning averages.Source: Figure 4 · p. 10
  • Token-Disentangled Latent Test-Time Scaling refines frozen MLLMs by routing visual feedback to image-sensitive tokens and reasoning feedback to uncertain decision tokens.The framework avoids applying a single global reward to all editable latent states.Source: §6 · p. 10
  • Future work can reduce reward-estimation cost and extend the framework to longer multimodal interactions.Source: §6 · p. 10

Limitations

The method adds inference cost, relies on proxy rewards that can be noisy, and requires editable open-weight backbones. The appendix also outlines implementation, ablation, and qualitative analyses, including the visual-reward baseline.

Limitations

  • Each example incurs extra inference cost from latent refinement and multiple candidate continuations; proxy rewards can be noisy on ambiguous images, underspecified questions, or answers requiring external knowledge.The bounded tanh in the visual reward limits but does not eliminate this noise.Source: Limitations · p. 10
  • Hard top-K routing is used for branch disjointness, soft or learned routing is deferred, and the method requires editable, differentiable open-weight backbones, excluding API-only closed models.Longer multi-turn settings and broader real-world domains remain future work.Source: Limitations · p. 10

Appendix Overview

  • The appendix covers implementation details, supplementary ablations, and qualitative routed-token case studies, including a representative failure example.Source: Appendix Overview · p. 13

A. Initial-Rollout Visual Baseline

  • The initial CoT rollout is the per-instance visual baseline with zero visual reward; later candidates receive relative engagement feedback, not an answer-correctness verification.The visual signal is used only for image-sensitive latent tokens.Source: A. Initial-Rollout Visual Baseline · p. 13

B. Implementation Details

The implementation refines a short prefix of generated hidden states through role-specific reward feedback while keeping model weights frozen. It specifies reward components, default routing and optimization settings, candidate decoding, answer aggregation, and model-specific hidden-state handling.

B.1. Reasoning Reward Details

  • The reasoning reward uses a text-prior-corrected score mapped to a bounded, per-step baseline-centered scalar, alongside a margin against competing predictions.The competing set includes the initial rollout when available.Source: §B.1 · p. 13

B.2. Default Configuration

  • The default configuration uses four refinement steps and an editable prefix of min(⌊0.5T⌋, 300) starting at position zero; visual and reasoning masks are strictly disjoint, with budgets of 0.4 of editable and non-visual tokens, respectively.It uses Adam at learning rate 0.05, anchor weight 0.05, and logit-entropy weight 0.01.Source: §B.2 · pp. 13–14
  • Visual engagement is scored with temperature 0.2 at the final attention layer over at most 128 generated tokens, while image sensitivity uses patch blackening with patch size 14 and drop probability 0.5.The reasoning reward weights the bounded score, margin, and text prior by 1.0, 0.5, and 1.0, respectively.Source: §B.2 · p. 14

B.3. Latent Editing and Candidate Decoding

  • Editable states are the generated response’s last-decoder-layer hidden states before the frozen LM head; visual-encoder states, model weights, and intermediate decoder layers are not edited.The states are collected through the model’s native generation API.Source: §B.3 · p. 14
  • Each refinement step decodes one greedy prefix and three noisy-prefix variants from the edited states, then continues generation with the original multimodal prompt and unedited response prefix.Each candidate uses a fresh generation call rather than reusing a KV cache.Source: §B.3 · p. 14
  • The default selects the most frequent normalized short answer, breaking ties by the highest reward; InternVL uses an adapter to match Qwen-style generation outputs for latent-prefix handling.A reward-weighted log-sum-exp aggregation is implemented but is not the default.Source: §B.3 · p. 14

C. Additional Ablation Studies

The ablations use Qwen2.5-VL-7B and treat LogicVista and RealWorldQA as representative reasoning-heavy and perception-heavy benchmarks.

C. Additional Ablation Studies

  • The ablation baseline rollouts are 44.52 on LogicVista and 63.66 on RealWorldQA, matching the CoT entries in Table 1.All ablations use Qwen2.5-VL-7B as the frozen backbone unless otherwise specified.Source: C. Additional Ablation Studies · p. 14
  • The reported accuracy values use the two-benchmark average as Macro, with parenthetical improvements computed in percentage points over the baseline from unrounded scores.LogicVista and RealWorldQA represent reasoning-heavy and perception-heavy settings, respectively.Source: C. Additional Ablation Studies · p. 14
  • The ablation section uses LogicVista and RealWorldQA as representative reasoning-heavy and perception-heavy benchmarks, and also includes MathVista in the matched-budget comparison.The section’s baseline rollout values match the CoT entries in Table 1.Source: C. Additional Ablation Studies · p. 14

C.1. Token Routing Budget

The routing ablations test budget size, mask overlap, asymmetric budgets, and swapped reward assignments. They favor moderate, disjoint routing with the intended visual-versus-reasoning token assignment.

C.1. Token Routing Budget

  • Both benchmarks peak at routing budget 0.4, while every tested budget from 0.1 to 0.8 remains above baseline.The diagonal sweep varies visual and reasoning budgets together.Source: §C.1 · p. 15, Table 3 · p. 15
  • At budget 0.4, strict disjoint routing is stronger than overlapping masks on both benchmarks.This supports keeping the visual and reasoning masks separate.Source: §C.1 · p. 15, Table 4 · p. 15
  • The symmetric setting ρ_v = ρ_r = 0.4 is best on both benchmarks, while nearby asymmetric settings such as ρ_v = 0.4, ρ_r = 0.6 remain competitive.The reported optimum is reasonably flat around the default.Source: §C.1 · p. 15, Table 5 · p. 15
  • Swapping the visual and reasoning reward assignments is weaker than the correctly routed strict setting on both benchmarks, indicating that reward-token assignment direction matters.The comparison argues against gains arising simply from updating an arbitrary token subset.Source: §C.1 · pp. 15–16, Table 6 · p. 15

C.2. Routing Signal

The additional studies examine routing signals, reward components, optimization settings, matched-budget performance, and qualitative examples. They report that role-aware routing and the full refinement setup outperform relevant alternatives, with additional scoring work beyond candidate decoding.

C.2. Routing Signal

  • Image-sensitivity top-k outperforms prototype-similarity and visual-focus alternatives as a visual routing signal.The visual-routing signal comparison is reported as an ablation.Source: §C.2 · p. 16, Table 7 · p. 16
  • Likelihood-based reasoning routing signals outperform random non-visual routing, and negative log-probability is slightly stronger than token loss.Negative log-probability is used as the default reasoning-side routing signal.Source: §C.2 · p. 16, Table 8 · p. 16
  • Across patch-blackening settings, Macro remains above 56.20 versus a base Macro of 54.09, and the default patch size 14 with drop probability 0.50 reaches Macro 57.22.Large patches combined with high drop probability are the weakest setting, suggesting stronger corruption makes the sensitivity signal noisy.Source: §C.2 · p. 16, Table 9 · p. 16

C.3. Reward Components

  • The default visual engagement reward outperforms both alternatives, while removing the bounded score, margin, or text-only prior hurts reasoning performance on both benchmarks.The visual reward is stable across τ_vis ∈ [0.05, 1.0], with τ_vis = 0.2 optimal; final-layer engagement scoring outperforms a middle-layer choice.Source: §C.3 · p. 16, Table 11 · p. 17, Table 12 · p. 17

C.4. Latent Optimization Hyperparameters

  • The default latent-optimization setting is best on both benchmarks; four steps outperform two or eight, and updating the full latent span is harmful.Anchor-only and logit-only variants remain effective, but neither exceeds the default, indicating that the two regularizers complement each other.Source: §C.4 · p. 18, Table 15 · p. 18

C.5. Matched-Budget Comparison

  • Under matched decoded-candidate budgets, the method scores strongest on MathVista and LogicVista, with generation calls and GPU seconds roughly on par with Best-of-N and Reward-only.It uses noticeably more scoring calls for visual-engagement and image-sensitivity passes; the comparison uses 16 decoded candidates for output-space baselines.Source: §C.5 · p. 18, Table 16 · p. 18

D. Qualitative Case Studies

  • In the MMStar dog-counting example, visual-routed tokens concentrate on image-dependent phrases while reasoning-routed tokens concentrate on the count decision.Across examples, reasoning-heavy cases use high-entropy positions; in the LogicVista example the reasoning reward changes the answer from B to C, while visual reward keeps refinement tied to image evidence.Source: Table 10 · p. 17, D. Qualitative Case Studies · p. 20

Related papers

Loading 2609.35228v1…