PaperlayerView exact revision on arXiv

Source-linked AI summary

Latent Reasoning in Continuous Space for Unified Multimodal Models

Byungwoo Jeon, Yoonwoo Jeong, Hyunseok Lee, Minsu Cho, Jinwoo Shin

arXiv:2610.42149v1cs.CVcs.CLcs.LG

TL;DR

Prior reasoning-augmented UMMs struggle to transfer language-based reasoning and test-time scaling to image generation, which operates in a continuous space. LARC uses continuous latent reasoning trained through curriculum SFT and information-gain RL. It improves text-to-image benchmark performance, achieves a 7%p gain over BAGEL, and exhibits test-time scaling.

  • Problem

    Reasoning-augmented UMMs struggle to inherit language models’ test-time scaling for image generation because discrete token reasoning mismatches continuous image-generation representations.

    Source: §1 · p. 1

  • Method

    LARC progressively replaces token reasoning with continuous hidden states through curriculum SFT, then uses information-gain RL to refine latent reasoning without ground-truth chains or an external evaluator.

    Source: §1 · p. 2

  • Results

    LARC improves text-to-image benchmarks over base UMMs and reasoning-augmented variants, achieves a 7%p gain over BAGEL, and shows clear test-time scaling.

    Source: §1 · p. 2, §5 · p. 9

  • Takeaways & Limitations

    LARC demonstrates that continuous-space reasoning in a UMM can exhibit test-time scaling for image generation.

    Source: §5 · p. 9

    Training requires reinforcement learning over a multibillion-parameter UMM backbone, incurring non-trivial compute and carbon footprint.

    Source: Broader Impact Statement · p. 10

Abstract

from arXiv · show

Unified Multimodal Models (UMMs) achieve remarkable success on diverse multimodal understanding and generation tasks, yet they still struggle to effectively leverage language-based chain-of-thought reasoning for image generation. As a result, existing reasoning-augmented UMMs often fail to inherit the test-time scaling and fine-grained controllability observed in large language models. We attribute this failure to a fundamental modality mismatch: reasoning is performed in a discrete token space, whereas image generation operates in a continuous space. To bridge this gap, we introduce a Latent Reasoning framework in Continuous space (LARC), which alleviates the token-level bottleneck by reasoning over continuous hidden states. Specifically, the training framework consists of two stages: (i) curriculum supervised fine-tuning (SFT), and (ii) information-gain reinforcement learning (RL). First, the curriculum SFT stage gradually converts token-level reasoning into latent reasoning. Subsequently, the information-gain RL stage self-evolves latent reasoning without ground truth supervision. Experiments across diverse text-to-image generation benchmarks show that LARC consistently improves generation quality, prompt following, and compositional fidelity. Notably, LARC achieves state-of-the-art performance on GenEval with a 7%p gain over BAGEL and demonstrates clear test-time scaling behavior, outperforming token-based generation under both parallel and sequential test-time scaling strategies.

1 Introduction

Unified multimodal models struggle to scale reasoning for image generation because discrete token reasoning mismatches generation’s continuous space. LARC addresses this with a two-stage latent-reasoning recipe and improves benchmark performance while exhibiting test-time scaling.

1 Introduction

  • UMMs integrate image-generation modules into language models with rich textual semantics, offering an alternative to generation-only models that struggle with complex semantics such as attribute binding and world modeling.Recent generative models already support text-to-image synthesis, image editing, and instruction-following generation.Source: §1 · p. 1
  • Prior reasoning-augmented UMMs do not gain smoothly from larger test-time reasoning budgets, which the authors attribute to a mismatch between discrete reasoning and continuous image generation.Discrete chains struggle to encode dense, spatially grounded information, limiting gains to surface-level prompt manipulation.Source: §1 · p. 1
  • LARC combines curriculum supervised fine-tuning that progressively replaces token reasoning with continuous hidden states and information-gain reinforcement learning with collapse prevention.The recipe introduces controlled stochasticity and needs neither ground-truth latent chains nor an external evaluator.Source: §1 · p. 2
  • LARC surpasses base UMMs and recent reasoning-augmented variants on three text-to-image benchmarks, and doubling latent reasoning compute yields monotonic gains across all three.It improves OneIG-Bench by +7.6 absolute points over Uni-CoT.Source: §1 · p. 2

2 LARC: Latent Reasoning in Continuous Space

LARC combines curriculum SFT, which progressively introduces continuous latent reasoning, with self-evolving RL that rewards traces for improving target-image prediction. Its trajectories alternate language spans and latent hidden states that condition image generation.

2.1 Overview

  • LARC has two stages: curriculum SFT gradually introduces latent thought by feeding the UMM’s last hidden state as its next input embedding, followed by information-gain RL.The RL reward is derived from internal knowledge.Source: §2.1 · pp. 2–3, Figure 2 · p. 3

2.2 Interleaved Latent Reasoning

  • The trajectory alternates stochastic language spans with continuous hidden-state steps; after termination, the image module reads the full hidden-state trajectory to produce the target image.The model returns to language mode after fixed latent steps, and <eos> ends inference.Source: §2.2 · p. 3

2.3 Stage 1: Supervised Fine-Tuning with Curriculum Learning

  • Curriculum SFT starts with text-only reasoning, activates latent steps progressively, then trains with all steps; text prediction and image flow-matching losses are combined.Gradients pass through latent steps without explicit supervision targets, allowing them to encode representations implicitly.Source: §2.3 · p. 4

2.4 Stage 2: Self-Evolving Reinforcement Learning

  • Information-gain RL rewards latent traces when they raise the reference model’s target-image likelihood over the context-only baseline, while latent-diversity regularization discourages collapse.The reward is framed as conditional mutual information between the trace and target image given context.Source: §2.4 · pp. 4–5

3 Experiments

Experiments evaluate LARC on image generation, image editing, multimodal understanding, test-time scaling, and component ablations. Results report gains over baselines and improvements from latent reasoning and self-evolving reinforcement learning.

3.1 Image Generation Experiments

  • +15%p on Attribute Binding over the baseline is LARC’s largest reported T2I improvement, with the subtask requiring visual realization and compositional planning.Source: §3.1 · p. 6
  • +5.8% on DPG-Bench (85.07 →90.04) and +10.2%p on OneIG-Bench over BAGEL show improved prompt-following fidelity.Source: §3.1 · p. 6
  • +10.2% on LC-T over BAGEL (20.434 → 22.509), with PSNR rising from 24.3 to 25.7 and LPIPS falling from 0.069 to 0.049.Source: §3.1 · p. 6, Table 4 · p. 7

3.2 Text Generation Experiments

  • LARC improves over BAGEL and Uni-CoT across MME, MMB, MMMU, and MIRA, scoring 11.7 versus 7.5 on MIRA.Source: Table 5 · p. 7, §3.2 · p. 7

3.3 Test-time Scaling

  • As latent reasoning budget grows, performance rises monotonically across image generation, editing, and visual reasoning, outperforming best-of-N at matched compute.Source: §3.3 · p. 7

3.4 Component Ablations

  • Latent reasoning improves all four benchmarks over token-level reasoning, including +2.6 on MIRA and +0.65 on LC-T.Source: Table 6 · p. 8, §3.4 · p. 8
  • Interleaved latent reasoning beats the naive strategy by +0.02 on OneIG-Bench and +0.44 on LC-T, with comparable performance elsewhere.Source: Table 7 · p. 8, §3.4 · p. 8
  • Adding Stage 2 self-evolving RL raises T2I generation from 0.83 to 0.88 and visual reasoning from 9.7 to 11.7 beyond curriculum SFT.Source: §3.4 · p. 8

4 Related Work

Related work covers language-model test-time scaling, image-generation architectures, and reinforcement learning for language models. LARC is positioned as continuous-space reasoning for UMMs, trained with outcome-level rewards where ground-truth reasoning trajectories are unavailable.

4.1 Test-time Scaling in LLMs

  • Prior UMM scaling efforts rely on discrete reasoning tokens and lack predictable scaling; LARC instead reasons in continuous space.Source: §4.1 · p. 8

4.2 Image Generation Models

  • UMMs span discrete-token models, hybrid text-and-diffusion systems, and decoupled visual architectures; LARC adds reasoning to an existing backbone.Source: §4.2 · p. 9

4.3 Reinforcement Learning in LLMs

  • Because image generation lacks ground-truth reasoning trajectories, LARC uses GRPO [34] with outcome-level rewards rather than chain-level annotations.Source: §4.3 · p. 9

5 Conclusion

LARC performs reasoning in continuous space for unified multimodal models, using curriculum SFT and information-gain RL. On BAGEL, it achieves state-of-the-art text-to-image performance and clear test-time scaling.

5 Conclusion

  • LARC represents reasoning steps as continuous hidden states and pairs curriculum supervised fine-tuning with self-evolving reinforcement learning driven by an information-gain reward.The RL stage requires no external critic.Source: §5 · p. 9
  • On BAGEL, LARC achieves state-of-the-art text-to-image performance with a 7%p gain over BAGEL.Source: §5 · p. 9
  • LARC is the first reasoning-augmented UMM reported to exhibit clear test-time scaling for image generation.Source: §5 · p. 9

Broader Impact Statement

The broader impacts discuss potential benefits of allocating more inference compute, risks inherited from text-to-image generation, responsible-deployment measures, and training costs.

Broader Impact Statement

  • Additional inference compute can yield higher-fidelity outputs, particularly for prompts involving attribute binding or spatial layout.The authors identify design assistance, accessibility, education, and scientific visualization as potential applications.Source: Broader Impact Statement · p. 10
  • LARC inherits risks of misleading or harmful imagery, bias amplification, and deceptive-content misuse, while improved prompt fidelity also heightens dual-use concerns.The authors state that dataset-provenance risks are inherited from BAGEL and the public datasets used, rather than newly created.Source: Broader Impact Statement · p. 10
  • The authors recommend safeguards including content moderation, watermarking, prompt-level filters, and training-data audits.They also describe a restrictive release license and encourage evaluation protocols for reasoning-augmented generative models.Source: Broader Impact Statement · p. 10
  • Training requires reinforcement learning over a multibillion-parameter backbone, incurring non-trivial compute and carbon costs.Pretrained BAGEL initialization and a curriculum that converges in fewer steps than naive joint training are described as mitigations.Source: Broader Impact Statement · p. 10

A.1 Training Details

Training uses BAGEL and a two-stage curriculum-SFT/RL pipeline, with different expert updates in each stage and a reported compute budget.

A.1 Training Details

  • Training adopts BAGEL and proceeds through curriculum supervised fine-tuning followed by self-evolving reinforcement learning.Source: §A.1 · p. 16
  • During Stage 1, the vision tokenizer is frozen while the understanding and generation experts are jointly trained across phases that increase activated latent steps from 0 to N.The learning rate decays at every phase transition.Source: §A.1 · p. 16
  • During Stage 2, the policy is initialized from MSFT, a copy is frozen as the reference, and only the understanding expert is updated during RL.The generation expert is reused for reward estimation through shared-noise flow-matching loss.Source: §A.1 · p. 16
  • Training used 8 NVIDIA A100s, taking approximately 300 GPU hours for Stage 1 and 96 GPU hours for Stage 2.Source: §A.1 · p. 16
  • Table 9 summarizes hyperparameters for Stage 1 and Stage 2.Source: Table 9 · p. 16

A.2 Evaluation Details

Evaluation covers text-to-image generation, image editing, and multimodal understanding, following each benchmark's official protocol.

A.2 Evaluation Details

  • Inference uses greedy sampling and one image per prompt unless otherwise specified.Source: §A.2 · p. 16
  • Text-to-image evaluation uses GenEval, DPG-Bench, and OneIG-Bench, covering compositional accuracy, dense-prompt judgments, and prompt-image alignment.OneIG-Bench alignment scores are reported.Source: §A.2 · p. 16
  • Image editing is evaluated on CompBench multi-object editing tasks using local CLIP-text and CLIP-image scores, PSNR, SSIM, and LPIPS.Source: §A.2 · p. 16
  • Multimodal understanding is evaluated on MME, MMBench, MMMU, and MIRA, including visual chain-of-thought problems requiring intermediate visual representations.Source: §A.2 · p. 16
  • GPT-4o results use model version gpt-4o-2024-08-06; VLMEvalKit is used for multimodal-understanding evaluation, with official benchmark code for generation and editing.Source: §A.2 · p. 16

B Data Statistics

The reasoning dataset balances text-to-image, image-editing, and interleaved reasoning samples. The section motivates information-gain RL because latent reasoning lacks suitable preference labels and dense external supervision.

B Data Statistics

  • The 1M-sample dataset mixes FLUX-Reason [19], WEAVE [20], and ScaleEdit [21] to jointly represent text-to-image, image editing, and interleaved reasoning.Latent steps are progressively activated at marked reasoning-step boundaries.Source: B Data Statistics · p. 17, Table 10 · p. 17

C.1 Why Self-Rewarding RL?

  • Preference-based RL lacks pairwise labels for latent traces, which are not human-interpretable, and image rewards cannot distinguish traces yielding comparable images.The passage also notes that DPO-based optimization can overfit in-domain preference distributions, limiting generalization beyond the curated set.Source: §C.1 · p. 17
  • Critic-based RL adds a value network that roughly doubles trainable parameters, while accurate latent-trajectory values require unavailable dense supervision.Image-quality estimators or learned reward models inherit the evaluation gap.Source: §C.1 · p. 17
  • LARC uses information gain from a frozen Stage 1 reference policy and GRPO group-relative advantages, avoiding an external evaluator and separate value network.The optimization requirements are a model-derived reward and a policy-gradient estimator without an additional value network.Source: §C.1 · p. 17

C.2 Derivation of the Information-Gain Reward

The information-gain reward measures how much a latent reasoning trace changes the reference model’s likelihood of the target image relative to context alone. LARC estimates this reward with paired flow-matching evaluations and uses GRPO for policy optimization.

C.2 Derivation of the Information-Gain Reward

  • The reward is the reference-model log-likelihood of target image x given context c and latent trace h(τ), minus its likelihood given context alone.Its expectation recovers conditional mutual information; maximizing it is equivalent to maximizing mutual information between the latent trace and target image.Source: §C.2 · pp. 17–18
  • Because the generation expert lacks closed-form marginal image likelihoods, LARC approximates them with a conditional flow-matching objective and substitutes its negative-log-likelihood bound into the reward.The flow-matching loss is evaluated under the frozen reference policy; the sample-wise constant depends only on the target image and noise schedule.Source: §C.2 · p. 18
  • Using the same noise sample and time step cancels the shared sample-wise constant, yielding an unbiased, low-variance paired reward estimate without an extra reward model.The paired evaluation reuses the frozen reference policy from Stage 1.Source: §C.2 · p. 18

C.3 Policy Optimization

  • GRPO samples groups of trajectories, normalizes their total rewards into advantages, and updates the policy with a clipped surrogate objective, without a separate value network.The procedure initializes from the reference policy and uses group-relative advantages.Source: §C.3 · p. 18, §C.2 · p. 18

D.1 Detailed Analysis on Test-time Scaling

LARC’s latent-step scaling is reported to outperform best-of-N at every tested scale with modest additional inference time. The section also reports that removing the latent diversity reward causes latent-step collapse and degrades benchmark performance.

D.1 Detailed Analysis on Test-time Scaling

  • 1.2 seconds of additional time at L = 10 versus L = 1 accompanies LARC’s reported advantage over best-of-N at every tested scale.The comparison measures per-sample clock time on one NVIDIA A100 at batch size 1; LARC uses a single pass, unlike best-of-N’s iterative sampling.Source: §D.1 · p. 19, Table 11 · p. 19

D.2 Additional Ablation Study

  • 0.97 pairwise cosine similarity across latent steps occurs without the diversity reward, while MIRA falls from 11.7 to 9.8; λdiv = 0.1 gives the best trade-off.The passage reports degradation on all three benchmarks when the diversity reward is removed.Source: §D.2 · p. 19, Table 12 · p. 19

D.3 Qualitative Results

  • The qualitative comparisons place BAGEL best-of-N and LARC side by side for text-to-image and image editing at N = L = 1, 4, and 10.Each figure assigns those settings to successive rows.Source: Figure 3 · p. 20, Figure 4 · p. 21

E Limitations and Future Work

LARC is currently instantiated on UMMs with flow-matching image modules, and latent reasoning adds inference latency. The authors identify other generation heads and inference-efficiency strategies as future directions.

E Limitations and Future Work

  • The current framework is instantiated only on UMMs whose image module is a flow-matching head; discrete-generation and other heads remain future extensions.The authors describe extending the principle to those heads as a natural next step.Source: E Limitations and Future Work · p. 22
  • Interleaved latent reasoning adds inference computation and latency compared with the base UMM.Reducing this overhead through efficient decoding, distillation, or adaptive latent allocation is left for future work.Source: E Limitations and Future Work · p. 22
  • The authors leave both broader generation-head support and inference-overhead reduction for future work.Proposed approaches include efficient decoding, distillation, and adaptive latent allocation.Source: E Limitations and Future Work · p. 22
Loading 2610.42149v1…