PaperlayerView exact revision on arXiv

Source-linked AI summary

PairUni: Pairwise Training for Unified Multimodal Language Models

Jiani Zheng, Zhiyang Teng, Kunpeng Qiu, Xiangtai Li, Anran Wang, Yu Tian, Ye Tian, Haochen Wang, Zhuochen Wang

arXiv:2510.25682v3cs.CL

TL;DR

Unified vision-language models must balance heterogeneous understanding and generation supervision during reinforcement learning, where unrelated signals can create optimization conflict. PairUni organizes related samples into pairs and uses pair similarity to modulate optimization; it reports balanced improvements across tested models and generalization to Lumina-DiMOO.

  • Problem

    Unified vision-language models must balance heterogeneous understanding and generation objectives during RL, while unrelated signals and competing gradients can create task interference and unstable updates.

    Source: §1 · pp. 1–2

  • Method

    PairUni constructs aligned or retrieval-based understanding–generation pairs and uses PairGRPO to modulate advantages according to pair similarity.

    Source: §1 · p. 2, §2.1 · p. 3

  • Results

    PairUni achieves balanced improvements on Janus-Pro and generalizes to Lumina-DiMOO, with reported gains across architectures and scales.

    Source: §1 · p. 3, §3.1 · p. 8

  • Takeaways & Limitations

    The findings support pairing data and weighting advantages by pair similarity as a general ingredient for improving understanding and generation together in unified multimodal training.

    Source: §1 · p. 3

Abstract

from arXiv · show

Unified Vision-Language Models (UVLMs) perform both understanding and generation within a single architecture. Since these models rely on heterogeneous data and supervision, balancing both generation and understanding in reinforcement learning (RL) is challenging. To address this challenge, we propose PairUni, a unified framework that reorganizes data into understanding-generation (UG) pairs and aligns optimization accordingly. Specifically, we construct a unified paired dataset by synthesizing aligned instances via cross-modal semantic completion and retrieving semantically related samples. These paired structures expose cross-task semantic correspondences and support consistent policy learning. To leverage this structure, we present PairGRPO, a pair-aware variant based on Group Relative Policy Optimization. It assigns a similarity score to each pair to modulate the advantage, strengthening learning from well-aligned examples and reducing task interference. Extensive experiments across diverse UVLM architectures (Autoregressive and Discrete Diffusion) and scales (1B to 14B) demonstrate that PairUni yields consistent improvements over strong baselines. Notably, our method also demonstrates strong generalization by improving performance on image editing tasks without using any editing-specific data. Codes are available at https://github.com/Haochen-Wang409/PairUni.

1 Introduction

Unified RL for vision-language models must balance heterogeneous understanding and generation objectives, yet unrelated supervision can produce task interference. PairUni addresses this by pairing related data and adjusting optimization according to pair similarity.

1 Introduction

  • Unified RL faces task interference, diverse understanding tasks that resist a single reward design, and limited guidance for selecting and aligning data at scale.The paper also describes competing signals as driving unstable updates and uneven performance gains across tasks.Source: §1 · p. 2
  • PairUni organizes supervision into understanding–generation pairs centered on the same or closely related images, using aligned and retrieval-based pair types.Aligned pairs complete single-task samples into unified quadruples; retrieved pairs connect semantically related understanding and generation samples when exact matches are scarce.Source: §1 · p. 2
  • PairGRPO modulates the advantage by pair similarity, giving aligned pairs full weight and down-weighting retrieved pairs to reduce cross-task interference.The clipped objective and KL regularization preserve GRPO stability.Source: §1 · p. 2
  • PairUni introduces PairUG-16k, a high-quality paired dataset designed to support unified RL fine-tuning.Figure 1 presents aligned generation-understanding examples and a retrieval-and-clustering strategy for constructing the dataset.Source: Figure 1 · p. 2, §1 · p. 3
  • PairUni achieves balanced improvements on Janus-Pro and generalizes to Lumina-DiMOO, supporting paired training with similarity-weighted advantages as an effective approach to unified multimodal learning.The reported conclusion is that understanding and generation improve together rather than trading one for the other.Source: §1 · p. 3

2 Method

PairUni pairs understanding and generation supervision, then uses PairGRPO to align their policy optimization. Its formulation is model-agnostic across autoregressive and discrete-diffusion backbones.

2.1 Pairing Understanding–Generation Pairs

  • PairUni reorganizes heterogeneous understanding and generation data into image-centered quadruples so training items support both capabilities.Each item contains an image, generation prompt, understanding question, and answer.Source: §2.1 · p. 3

2.1.1 Generating Aligned Pairs

  • Aligned pairs complete single-task samples into semantically checked quadruples, then cluster joint image features and retain each cluster’s medoid.Synthesized components are checked against the original data to guard against contradictions.Source: §2.1.1 · pp. 3–4

2.1.2 Constructing Retrieval-Based Pairs

  • Retrieval-based pairs link different images by cosine similarity, selecting top-n understanding matches above a threshold while preventing image reuse.These approximate matches expand supervision when exact aligned pairs are scarce.Source: §2.1.2 · p. 4

2.2 PairGRPO

  • Pairwise GRPO samples multiple trajectories per understanding–generation data pair and normalizes rewards within task-type groups, deriving updates from semantically related examples.The pairing is at the data level, with multiple trajectories possible for either item.Source: §2.2 · p. 5
  • PairGRPO gives aligned pairs weight 1 and scales retrieved-pair advantages by √sp, strengthening well-aligned updates and attenuating weak or noisy ones.The pair-level similarity score is computed using a pretrained image encoder.Source: §2.2 · pp. 5–6

3 Experiments

Experiments report gains on both understanding and generation, then test PairUni across model families and scales and examine the effects of data pairing and similarity weighting.

3.1 Main Results

  • PairUni improves both understanding and generation, avoiding the trade-off typically observed in unified modeling.Source: §3.1 · pp. 6–7
  • At 7B, MMMU reaches 47.0 versus Janus-Pro-7B (41.1), while MMStar rises from 46.5 to 49.5 and MME from 1567.1 to 1597.7.Source: §3.1 · p. 6, Table 1 · p. 7
  • WISE scores are 0.38 (1B) and 0.45 (7B), while GenEval scores are 0.79 (1B) and 0.85 (7B), exceeding Janus-Pro comparators at both scales.Source: §3.1 · p. 7, Table 2 · p. 8, Table 3 · p. 9
  • PairUni gains on discrete-diffusion Lumina-DiMOO and Bagel-14B, and improves editing subtasks without editing-specific training data.Source: §3.1 · p. 8, Table 4 · p. 10

3.2 Analysis: Effectiveness of PairUni Framework

  • Under identical data source and budget, naive unpaired or random mixtures yield 0.71–0.73 on GenEval, while PairUG-16k has more stable training dynamics.The medoid selection strategy also outperforms random selection.Source: §3.2 · p. 9, Figure 4 · p. 9
  • Square-root similarity weighting outperforms linear or no weighting; threshold 0.6 balances training volume against excluding low-similarity pairs.Visual-similarity encoders outperform high-level semantic encoders for retrieval.Source: §3.2 · pp. 9–10

4 Conclusion

PairUni aligns understanding and generation through paired training signals and reports balanced improvements on standard UVLM evaluations. The surrounding related work situates this approach among unified architectures and RL methods, while the appendix outlines further analyses.

4 Conclusion

  • PairUni and PairUG-16k align understanding and generation through paired training signals, with balanced improvements over competitive RL baselines on standard UVLM evaluations using Janus-Pro backbones.Source: §4 · p. 10

Appendix

  • Figure 5 highlights a domain gap between multimodal understanding and image-generation datasets and the need for alignment.Source: Figure 5 · p. 15
  • The appendix covers related work, data analyses, ablations, additional-model generalization, and qualitative understanding cases.Source: Appendix · p. 15

A Related Work

  • Unified vision-language architectures include autoregressive foundations [42] [49], diffusion-based generation [56], Janus-Pro, and Bagel [8]; PairUni targets their reinforcement-learning post-training stage.Source: A Related Work · p. 15
  • Figure 6 presents representative paired cases illustrating the proposed synergy between understanding and image-generation tasks.Source: Figure 6 · p. 16
  • Unlike prior unified RL methods focused on method design, PairUni builds understanding–generation pairs; related approaches include generated QA rewards [26], dual-source rewards [12], Pair DPO [37], and two-stage training [14].Source: A Related Work · p. 16

B.1 Data Distribution of Understanding and Generation Data

The source understanding and generation datasets differ substantially in their category distributions. PairUG-16k combines aligned and retrieved pairs, with retrieved-pair similarities concentrated in a moderate range.

B.1 Data Distribution of Understanding and Generation Data

  • The source splits use Orsta-47k for diverse understanding tasks and BLIP-3o-60k for described generated images, with deduplication and pretraining-data overlap removal.Source: §B.1 · p. 16
  • Understanding data include math- and OCR-intensive items that are rare in generation data, while descriptive object-centric scenes are comparatively overrepresented in generation.Source: §B.1 · pp. 16–17

B.2 Retrieved Pairs Cases

  • Figure 6 presents representative retrieved-pair examples.Source: §B.2 · p. 17

B.3 Distribution of PairUG-16k

  • PairUG-16k contains aligned and retrieved splits; retrieved-pair similarities are mostly 0.55–0.75, and the aligned split covers 17 labeled categories.Source: §B.3 · p. 17

C Additional Ablation Studies

The additional ablations examine design choices in PairUni using Janus-Pro-1B.

C Additional Ablation Studies

  • The ablations cover retrieved-pair weighting, similarity thresholds, K-means medoid selection, and image feature extractors, all evaluated on Janus-Pro-1B.Source: C Additional Ablation Studies · p. 17

C.1 Weighting Strategy for Retrieved Pairs

The ablations favor square-root similarity weighting and a 0.6 retrieval threshold, and report better performance from K-means medoids than random selection.

C.1 Weighting Strategy for Retrieved Pairs

  • Square-root similarity weighting provides the most robust performance boost, balancing high-similarity retrieval contributions without dominating the loss.Source: §C.1 · p. 17

C.2 Similarity Threshold Selection

  • A 0.6 similarity threshold is reported as optimal; 0.5 may introduce degrading noise, while 0.7 reduces training-data diversity and retrieval benefits.Source: §C.2 · p. 18

C.3 K-means Medoid Selection

  • K-means medoids significantly outperform random sampling; the authors hypothesize medoids favor samples compatible with both tasks.Source: §C.3 · p. 18

C.4 Results with different image extractors.

The extractor comparison finds that PE underperforms on both understanding and generation benchmarks, while DINOv3 and ResNet perform comparably. In this setting, visual feature similarity is more critical than semantic abstraction.

C.4 Results with different image extractors.

  • The comparison evaluates Perception Encoder (PE), DINOv3, and ResNet as image feature extractors.These are the three alternatives considered for the final implementation.Source: §C.4 · p. 18
  • PE underperforms on both understanding and generation benchmarks.Source: §C.4 · p. 18
  • DINOv3 and ResNet achieve comparable results, indicating visual feature similarity matters more than semantic abstraction in this setting.Source: §C.4 · p. 18

D Gradient Similarity Analysis

The analysis reports that semantically aligning understanding and generation data is associated with greater gradient agreement and stronger downstream results.

D Gradient Similarity Analysis

E Generalization Capabilities

PairUni improves comprehension and generation in the reported Lumina-DiMOO implementation and improves several benchmarks for a BAGEL model trained with PairUG-16k. The section also describes the preliminary Lumina-DiMOO integration and comparisons of extractor and data-combination choices.

E Generalization Capabilities

  • The Lumina-DiMOO integration is preliminary and may benefit from refinement; it uses rollout size 2 and fixed sampling steps for text and image generation.The implementation uses 2 diffusion steps for text generation and 35 for image generation.Source: E Generalization Capabilities · p. 18
  • Table 10 compares PE, DINOv3, and ResNet, and its caption states that the results justify choosing ResNet for the final implementation.Source: Table 10 · p. 19
  • Figure 8 compares median gradient cosine similarity and benchmark performance across six data-combination scenarios.The caption names PairUG-16k, retrieval-based pairs, low-similarity unpaired data, generation-only data, understanding-only data, and random pairs.Source: Figure 8 · p. 19
  • In the current Lumina-DiMOO implementation, PairUni improves both comprehension and generation.Source: E Generalization Capabilities · p. 19
  • The PairUG-16k SFT model improves consistently on MMMU, POPE, GenEval (s), GenEval (L), and ImgEdit.BAGEL does not support RL training under its default settings, so this evaluation uses supervised fine-tuning with a 1:1 understanding-to-generation data ratio.Source: E Generalization Capabilities · p. 19

F Case Studies on Understanding Tasks

The case studies present examples of PairUni's improved accuracy over Janus-Pro on complex visual scenes. The accompanying GPT-o3 prompts enforce semantic alignment when completing paired training data.

G Prompts for GPT-o3

  • For understanding-to-generation completion, GPT-o3 produces captions consistent with question-answer evidence; for generation-to-understanding, it creates answerable questions about captioned entities and attributes.Both prompt pathways include self-verification for hallucinations or contradictions.Source: G Prompts for GPT-o3 · p. 19

F Case Studies on Understanding Tasks

G Prompts for GPT-o3

Loading 2510.25682v3…