PaperlayerView exact revision on arXiv

Source-linked AI summary

SRUM: Fine-Grained Self-Rewarding for Unified Multimodal Models

Weiyang Jin, Yuwei Niu, Jiaqi Liao, Chengqi Duan, Aoxue Li, Shenghua Gao, Xihui Liu

arXiv:2510.12784v2cs.CVcs.CL

TL;DR

Unified Multimodal Models can understand prompt-image alignment yet fail to generate faithful images, leaving a gap between understanding and generation. SRUM uses the model’s understanding module to reward generation through global and local feedback. It reports improved results on composition and reasoning benchmarks, while identifying its current approach as a preliminary exploration with room to improve scoring prompts and scale data.

  • Problem

    Current UMMs can correctly judge prompt-image alignment yet fail to generate a faithful image from the same prompt, revealing a gap between understanding and generation.

    Source: §1 · p. 2

  • Method

    SRUM is a post-training feedback framework in which a UMM’s understanding module evaluates generation, combining global compositional assessment with local object-level feedback.

    Source: §1 · p. 2, §3.3 · p. 6

  • Results

    82.18 to 88.37 on T2I-CompBench and 43.82 to 46.75 on T2I-ReasonBench are the reported improvements over a strong baseline.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The findings support self-rewarding as a way for a UMM’s understanding module to guide its own generation, with reported in-domain and out-of-domain generalization.

    Source: §1 · p. 3, §5 · p. 14

    SRUM is a preliminary exploration; its scoring prompts could improve, and the authors plan to scale the method to larger datasets.

    Source: §5 · p. 14

Abstract

from arXiv · show

Recently, remarkable progress has been made in Unified Multimodal Models (UMMs), which integrate vision-language generation and understanding capabilities within a single framework. However, a model's strong visual understanding often fails to transfer to visual generation: it may correctly judge prompt-image alignment while failing to generate a faithful image from the same prompt. This raises a compelling question: Can a model improve itself by using its understanding module to reward its generation module? We introduce SRUM, a self-rewarding post-training framework directly applicable to existing UMMs of various designs. SRUM creates a feedback loop where the model's own understanding module acts as an internal ``evaluator'', providing corrective signals to improve generation without additional human-labeled data or external reward models. To provide comprehensive feedback, SRUM uses a global-local dual reward system: a \textbf{global reward} ensures overall visual semantics and layout, while a \textbf{local reward} refines fine-grained, object-level fidelity. SRUM shows strong generalization, boosting performance on T2I-CompBench from 82.18 to \textbf{88.37} and on T2I-ReasonBench from 43.82 to \textbf{46.75}. Overall, our work establishes a powerful paradigm for enabling a UMM's understanding module to guide and enhance its own generation via self-rewarding.

1 Introduction

Text-to-image models struggle with complex instructions, and Unified Multimodal Models can understand prompt-image alignment better than they can generate faithful images. SRUM uses the model’s understanding module to guide generation through global and local rewards, improving benchmark performance.

1 Introduction

  • Traditional text-to-image models often fail on world knowledge, complex spatial relationships, detailed attribute binding, and compositional reasoning.UMMs integrate understanding and generation in one framework, offering potential synergy across those capabilities.Source: §1 · pp. 1–2
  • Current UMMs can judge alignment between a detailed prompt and a complex image yet fail to generate a faithful image from that prompt.The introduction frames this as a gap between understanding and generation that SRUM aims to bridge, particularly in complex generation domains.Source: §1 · p. 2, Figure 1 · p. 3
  • SRUM uses the UMM’s understanding module as an internal evaluator to improve generation without external reward models, human labels, or additional image data during training.The framework treats generation as the student and understanding as the teacher or evaluator.Source: §1 · p. 2
  • Global rewards assess high-level compositional coherence, while local rewards target object-level details, attribute binding, and spatial arrangements.The paper says a single holistic score fails to provide the fine-grained corrective signals needed for detailed improvement.Source: §1 · p. 2
  • 82.18 to 88.37 on T2I-CompBench and 43.82 to 46.75 on T2I-ReasonBench are the reported gains over a strong baseline.The paper also reports improvements in composition, reasoning, and visual fidelity, with generalization across in-domain and out-of-domain settings.Source: §1 · p. 2

2 Related Works

Related work covers unified-model architectures, post-training approaches for generation, and self-rewarding methods for understanding. SRUM builds on these directions with global and local feedback for a more comprehensive training signal.

2.1 Architectures for Unified Multimodal Models

  • Unified multimodal architectures range from purely autoregressive and sequential AR-diffusion designs to unified transformers and sparse Mixture-of-Transformers; SRUM inherits Bagel’s MoT framework.Bagel’s modality-specialized experts share information through a common attention mechanism.Source: §2.1 · pp. 3–4

2.2 Post-Training Stage in UMMs

  • Post-training methods use reasoning, verification, preference feedback, reconstruction loss, or rule-level rewards, while SRUM scores self-generated samples with the model’s understanding module.The passage notes that several alternatives rely on external models, curated paired data, or task-specific rules.Source: §2.2 · p. 4

2.3 Self-Rewarding in Understanding Models

  • Prior self-rewarding methods for understanding models tend to focus on one feedback dimension, whereas SRUM combines global and local rewards.The dual system is intended to provide a more comprehensive training signal.Source: §2.3 · pp. 4–5

3 SRUM: Self-Rewarding for Unified Multimodal Models

SRUM generates and scores candidate images, then uses the cached feedback in reward-weighted training. Its feedback combines object-level and image-level judgments to guide localized refinement while maintaining overall composition.

3 SRUM: Self-Rewarding for Unified Multimodal Models

  • SRUM generates candidate images and bounding boxes, evaluates overall composition and fine-grained details, then uses cached rewards for targeted, region-specific training.The pipeline is described as reducing reward hacking through its training stage.Source: §3 · p. 5

3.1 Image Candidates and Bounding Box Generation

  • The UMM generates semantically rich images; Bagel can use an external segmentation model for spatial supports, while BLIP3o can use native grounding.In both cases, the UMM verifies boxes, matches prompt-relevant regions, and assigns semantic rewards; the external component is a localization aid, not a reward model.Source: §3.1 · p. 5

3.2 Rewarding Process

  • Local judgment scores object fidelity and artifacts, while global judgment assesses holistic composition and spatial alignment; reward generation yields local rewards and one global reward.The understanding module is frozen and receives no gradients; SRUM generates candidates, caches rewards, and updates trainable generation or flow parameters.Source: §3.2 · p. 6

3.3 Reward-Weighted Training

  • Cached regional reward map R and global scalar α weight the squared error between predicted velocity and the target derived from the original latent.Positive feedback encourages preservation and negative feedback promotes change; the final objective combines the reward loss with a reference constraint weighted by λ_c.Source: §3.3 · pp. 6–7

4 Analysis of Self-Rewarding

SRUM is evaluated across unified multimodal models and benchmarks, with analyses of reward components, generalization, understanding capabilities, and computational efficiency.

4.1 Experimental Setup

  • Experiments post-train Bagel and BLIP3o from pretrained weights, using Bagel for comprehensive analysis and BLIP3o to test generality.Evaluation spans compositional benchmarks, GenEval, WISE, and T2I-ReasonBench.Source: §4.1 · pp. 7–8

4.2 Main Results

  • 88.37 is Bagel+SRUM’s T2I-CompBench overall score in CoT mode, 3.91 points above Bagel with CoT.The table covers T2I and UMM models; SRUM-enhanced models are marked +SRUM.Source: Table 1 · p. 8, §4.2 · p. 8

4.3 Empirical Study

  • Removing local reward drops accuracy by 0.76 in CoT mode and 1.04 in standard mode; global reward, reference constraint, and dense feedback also contribute.The full model has the highest overall accuracy in the reward ablation.Source: §4.3 · pp. 9–10, Figure 3 · p. 10
  • λc = 0.5 is most effective across Bagel’s CoT and non-CoT configurations and is fixed for subsequent experiments.Figure 3’s right panel evaluates accuracy across λc values in both modes.Source: §4.3 · p. 10, Figure 3 · p. 10
  • SRUM outperforms ReCA and UnifiedReward; its internal scorer has stronger evaluator calibration (ρ = .58 vs. .43) and accept/reject separation (0.67 vs. -0.31).Replacing the internal understanding module with external VLMs does not yield stable improvements.Source: §4.3 · pp. 10–11, Table 3 · p. 10
  • Stepwise scoring indicates that think mode and global reward improve early layout, while local rewards improve detail fidelity in later inference steps.The analysis separates layout from detail scores and contrasts SRUM with a global-reward-only sample-reward baseline.Source: §4.3 · p. 11, Figure 4 · p. 11
  • Understanding benchmarks show only marginal fluctuations, while SRUM preserves supportive activation in secondary functional clusters; scoring 6K candidate images takes under 4 H100 GPU-hours.SRUM loss computation takes 12.5 GPU-hours versus 12.4 for SFT.Source: §4.3 · pp. 9, 12, Figure 5 · p. 12, Table 4 · p. 12
  • SRUM generalizes across settings: it scores 0.83 in GenEval Counting, improves other WISE prompt groups when trained on one group, and outperforms SFT and Base on T2I-ReasonBench.WISE gains hold in standard and CoT reasoning modes; T2I-ReasonBench evaluation uses a model trained on T2I-CompBench prompts.Source: §4.3 · pp. 12–14

5 Conclusion

SRUM uses a model’s understanding module to reward its generation module through global and local feedback. Experiments report state-of-the-art benchmark results and robust generalization, while the authors identify remaining scope for development.

5 Conclusion

  • SRUM decomposes self-reward into global and local components for multi-scale alignment and refinement.The framework uses the understanding module to reward generation.Source: §5 · p. 14
  • SRUM reports state-of-the-art results on T2I-CompBench and T2I-ReasonBench, with robust in-domain and out-of-domain generalization.The authors also report empirical validation of the fine-grained reward design.Source: §5 · p. 14
  • The authors characterize SRUM as a preliminary exploration and identify scoring prompts, dataset scale, and a more closed-loop training system as areas for development.The current implementation uses standardized prompt templates and, for some models, external localization aids; the reward comes from the UMM’s frozen understanding module.Source: §5 · p. 14

A Detail Settings

The implementation follows a specified post-training and generation recipe, with localization sources varying by model. The UMM verifies boxes and supplies semantic rewards in either localization setup.

A Detail Settings

  • Post-training follows Bagel stage 4 with AdamW, β1 = 0.9, β2 = 0.95, epsilon 1.0 × 10−15, and a constant learning rate.The epsilon choice is intended to mitigate loss spikes; the constant rate simplifies scaling training data without restarting training.Source: A Detail Settings · p. 21
  • Generation uses diffusion with classifier-free guidance in latent space, and the timestep changes from 1.0 to 4.0 when resolution increases.The timestep adjustment is used to maintain a stable noise-level distribution.Source: A Detail Settings · p. 21
  • Bagel uses SAM for localization, whereas BLIP3o can use native grounding; in both cases the UMM verifies boxes and supplies semantic reward.RefCOCO performance can guide localization-source choice.Source: A Detail Settings · p. 21

B Definition and Calculation of Average Activation Strength

Average Activation Strength quantifies activation in a functional neuron cluster for a task by averaging across cluster neurons and dataset samples. The appendix describes cluster assignment, evaluation, and aggregation procedures.

B Definition and Calculation of Average Activation Strength

  • Average Activation Strength is the mean activation of neurons in a functional cluster, averaged across an entire task dataset.It uses an intra-cluster average for each sample, followed by a dataset-wide average.Source: B Definition and Calculation of… · p. 22
  • A higher Average Activation Strength indicates stronger and broader engagement of a functional cluster on the task being measured.For example, the Understanding Cluster can be evaluated on generation tasks.Source: B Definition and Calculation of… · p. 22
  • The final score Sfinal is the expected sample-level score estimated by averaging Ssample across dataset DT.The appendix gives the dataset-wide expression as an average over samples and cluster neurons; Sfinal is the height of each activation-figure bar.Source: B Definition and Calculation of… · p. 22
  • Cluster construction selects highly active neurons by a percentile threshold and assigns each to understanding or generation according to which task has the higher mean activation.The procedure compares mean activations on understanding and generation datasets, thresholds their per-neuron maxima, and fixes the resulting clusters.Source: B Definition and Calculation of… · p. 23
  • Evaluation records cluster-neuron activations on understanding and generation datasets, averages within each cluster for each sample, then averages those scores across samples.The appendix describes repeating the evaluation and aggregation for each model and task dataset.Source: B Definition and Calculation of… · p. 23

C Data Curation

The data curation process uses T2I-CompBench training instructions to generate images and then score them with UMM understanding capabilities. The passage also situates the evaluators’ capabilities and reward prompting approach.

C Data Curation

  • T2I-CompBench training instructions guide UMM image generation, after which the UMM understanding side evaluates and scores the generated images.The understanding capability is compared to vision-language comprehension in models such as LLaVA, InternVL, and Gemini [67] [72].Source: C Data Curation · p. 24
  • The evaluated models draw on web-scale and canonical vision datasets, often supplemented by captioning and instruction-following data.The reward-prompting strategy is inspired by instruction-based image-editing methods [2] [29] [86].Source: C Data Curation · p. 24

D Failure Cases Study

The failure analysis identifies reasoning, causal multi-image generation, and aesthetics as distinct problem areas. The comparison with nano-banana indicates that the illustrated failure is not limited to the paper’s model.

D Failure Cases Study

  • In one reasoning case, an incorrect language-model answer leaves the generation module able to produce only “apple.”The prompt maps numbers to fruits and asks for a calculation before returning the corresponding fruit.Source: D Failure Cases Study · p. 25
  • Bagel performs poorly on causal multi-image generation because its training data rarely represents causality within a single image.Source: D Failure Cases Study · p. 25
  • Aesthetics are not a primary consideration because the method focuses on reasoning, knowledge, and composition, as illustrated by the “particularly beautiful chair” prompt.Source: D Failure Cases Study · p. 25
  • The illustrated failure also appears in nano-banana, which the passage describes as a systemic problem in generative models.The top row shows the paper’s failure cases and the bottom row shows nano-banana’s.Source: D Failure Cases Study · p. 25
Loading 2510.12784v2…