DeCoRL: Decomposed Consistency Reinforcement Learning for Multi-Image Composition

• MICo consistency requires preserving entity identities, attributes, and semantic bindings across references, with ambiguous many-to-one correspondences growing as inputs increase.Source: §1 · p. 2
• SFT often lacks explicit object-level correspondence supervision, contributing to identity confusion or attribute leakage as reference conditions grow.Source: §1 · p. 2
• RL offers a post-training alternative to imitating paired targets by optimizing model preferences through reward signals.Source: §1 · p. 2
• Holistic MICo scoring must resolve competing visual conditions and source bindings, a judgment that remains unreliable even for strong VLM evaluators.Source: §1 · p. 2
• DeCoRL decomposes holistic consistency into tractable checks over instruction-relevant objects, aggregating matched-object scores and penalizing missing objects for RL post-training.Its modular instantiation uses VLM parsing, Grounding DINO localization, and a CoT-trained reward model.Source: §1 · pp. 2–3
• DeCoRL yields consistent MICo consistency improvements on DeCo-Bench and the public MICo-Bench.DeCo-Bench covers compositions with 2–6 reference images.Source: §1 · pp. 2–3
• The paper identifies object-level decomposition as a practical reward-design principle, evaluating consistency locally after establishing instruction-relevant correspondences.Source: §1 · p. 3

• Open-source editing methods were primarily designed for single-image editing, while newer systems began supporting MICo but still struggled as reference counts increased.Source: §2 · p. 3
• Most existing MICo progress relies on large-scale, often synthetic, data followed by SFT, which provides limited explicit signals for cross-image correspondence and consistency.Consistency often degrades in harder cases with many reference images.Source: §2 · p. 3
• RL had been increasingly adopted for image-editing post-training, and the authors propose DeCoRL as a promising direction beyond standard SFT objectives.Source: §2 · p. 3

3 Method

• DeCoRL decomposes holistic MICo consistency into object-level checks, then aggregates their scores into a reward for RL post-training.The decomposition-based reward is not specific to the chosen optimizer.Source: §3 · p. 3, Figure 2 · p. 4

3.1 Task Definition

• MICo takes an editing instruction and reference images as input and generates an edited image that follows the instruction while preserving relevant reference consistency.Source: §3.1 · p. 4

3.2 Multi-Image Supervised Fine-Tuning

• SFT initializes Qwen-Image-Edit2511 for 1–6 references using approximately 470K examples, but does not explicitly supervise cross-reference object correspondences.The training data combines MICo-150K and UniPic 3.0.Source: §3.2 · pp. 4–5

3.3 Single-Object Consistency Reward model

• The single-object reward model is fine-tuned on annotated crop pairs with chain-of-thought explanations and binary consistency decisions, producing continuous scores in [0, 1].Training uses PaCO-RL-Dataset and Qwen3-VL-32B.Source: §3.3 · p. 5

3.4 Instruction-Relevant Object Correspondences

• A VLM identifies instruction-required objects and assigns open-vocabulary labels; Grounding DINO uses those labels to localize reference crops.The resulting aligned object instances support subsequent single-object evaluation.Source: §3.4 · pp. 5–6

3.5 Multi-Object Consistency Reward

• For each labeled object, Grounding DINO finds a generated-image crop; the reward model scores matched reference–generated crops, while an undetected required object receives zero reward.The object scores are aggregated into a multi-object consistency score, though the simple mean does not guarantee balanced improvement across objects.Source: §3.5 · pp. 6–7

3.6 Diffusion NFT Post-Training

• Diffusion NFT optimizes reward-improving predictions while keeping the updated policy close to a reference model for stability.Group-relative advantages are normalized and clipped; an additional VLM reward penalizes unnatural outputs, structural distortions, and compositing artifacts.Source: §3.6 · p. 7

4.1 Training Setup.

• Training fine-tunes Qwen-Image-Edit2511 for 24,000 steps, then applies Diffusion NFT post-training from the SFT checkpoint using LoRA.SFT uses MICo-150K, UniPic 3.0, and Nano-consistent-150K; RL post-training uses MICo-150K.Source: §4.1 · p. 7

4.2 Benchmarks

• DeCo-Bench covers 2–6 reference images with 40 cases per setting; MICo-Bench reports category-level and semantic-consistency and perceptual-quality results.Source: §4.2 · pp. 7–8

4.3 Comparative Results

• DeCo-Bench overall score: DeCoRL reaches 8.16, surpassing open-source baselines and Seedream4.0 (2K) (7.76), while Nano-Banana-Pro leads at 8.69.Source: §4.3 · p. 8, Table 1 · p. 8
• MICo-Bench overall score: DeCoRL achieves 54.03, outperforming the reported open-source baselines and proprietary Nano-Banana and Nano-Banana-Pro.Source: §4.3 · p. 8, Table 2 · p. 8

4.4 Ablation Study

• On DeCo-Bench, SFT raises the overall score from 6.22 to 7.05, and DeCoRL further raises it to 8.16, including 6-image results from 6.15 to 7.61.Source: Table 1 · p. 8, §4.4 · p. 9
• On MICo-Bench, DeCoRL raises the overall score from 37.58 with SFT to 54.03, including Human Centric 28.89→56.21 and De&Re 45.50→64.42.Source: Table 3 · p. 8, §4.4 · p. 9
• Against SFT, DeCoRL improves SR (7.01→7.64), Naturalness (6.54→8.01), and Artifacts (5.76→6.64), while PF is slightly lower (8.76→8.69).Source: Table 3 · p. 8, §4.4 · p. 9

4.5 Visualization

• The visual examples report better consistency from DeCoRL than open-source and closed-source models, including Nano-Banana-Pro, on two six-image cases.Source: §4.5 · p. 10

5.2 Discussion and Future Directions

• DeCoRL establishes instruction-relevant object correspondences, scores matched reference–generated object pairs, and aggregates those scores into a multi-object reward.Source: §5.2 · p. 10
• The method targets localizable objects and depends on accurate grounding and matching; scaling beyond six references requires suitable training data and further evaluation.Future directions include adaptive aggregation, robustness to imperfect crops, and relational or other non-localizable consistency.Source: §5.2 · p. 10

A.1 Training Details

• The single-object consistency reward model is Qwen3-VL-32B-Instruct trained on PaCO-RL-Dataset with LoRA rank 32 for 2,200 steps on 8 H800 GPUs.Source: §A.1 · p. 14
• SFT fine-tunes Qwen-Image-Edit-2511 on Nano-consistent-150k, MICo-150K, and UniPic 3.0 for 24,000 steps on 32 H800 GPUs with batch size 128.Source: §A.1 · p. 14
• RL starts from the SFT checkpoint and trains on MICo-150K with LoRA rank 32; a general-purpose VLM reward penalizes unnatural outputs, structural distortions, and compositing artifacts.Training uses 32 H800 GPUs and group size 16; Algorithm 1 identifies the multi-object and general VLM rewards and the NFT and regularization losses.Source: §A.1 · p. 14

A.2.1 DeCo-Bench.

• DeCo-Bench contains 200 cases, with 40 cases for each setting from 2 to 6 reference images, all drawn from real-world photographs.Source: §A.2.1 · p. 14
• The benchmark targets whether instruction-relevant entities from multiple references are correctly and consistently realized in the generated result.Source: §A.2.1 · p. 14
• Evaluation crops relevant objects from reference and generated images, scores each pair with Gemini-3.1-Pro, and averages the object-level scores for the sample.Source: §A.2.1 · p. 14

A.2.2 MICo-Bench

• MICo-Bench jointly evaluates source preservation and instruction alignment for each multi-image composition case.Its final score is based on these two considerations.Source: §A.2.2 · p. 15
• The benchmark covers Object-Centric (138 cases), Human-Centric (168), HOI (291), and De&Re (300) tasks.These tasks cover object composition, people and human–object interactions, and decomposition and recomposition of real portraits.Source: §A.2.2 · p. 15
• Source preservation weights the fraction of correctly preserved source images, using ArcFace for faces and a VLM for other elements.Semantic Consistency compares the generation with a human-verified reference image to avoid cross-image attention overload, and includes Prompt Following.Source: §A.2.2 · p. 15
• Subject Resemblance measures identity, appearance, and semantic-content match, while Perceptual Quality assesses naturalness and artifacts.Perceptual Quality is evaluated from the generated image alone.Source: §A.2.2 · p. 15
• The benchmark reports scores per task and overall, aggregating across each task’s cases and the full case set.The overall case set is the union of cases across tasks.Source: §A.2.2 · p. 15

A.3 More Visualizations

• The additional DeCo-Bench and MICo-Bench visual examples are presented as showing excellent MICo consistency and visual quality.The examples appear in Figures 6 and 7.Source: §A.3 · p. 15

Independent validation of the object-level reward evaluator

• 0.721 versus 0.354: the trained object-level evaluator outperforms its untuned counterpart in accuracy on EditReward-Bench.This measures agreement with benchmark labels, not reward calibration.Source: B Additional Validation and Discussion · p. 15, Table 5 · p. 19

Grounding and object matching diagnostics

• 94.6% label–crop matching accuracy was obtained under VLM-based verification of 10,000 sampled training examples.The authors caution that this diagnostic proxy is not human-annotated bounding-box localization or same-category instance-matching accuracy.Source: B Additional Validation and Discussion · p. 19

Behavior of best- and worst-case object rewards

• 0.3012 to 0.7423: the worst-case object reward increased more than the best-case reward, which rose from 0.6712 to 0.8412.Both increased over the reported checkpoints under mean aggregation; this does not establish that mean aggregation is optimal.Source: B Additional Validation and Discussion · p. 19, Figure 9 · p. 19

Evaluation scope and potential reward–metric alignment

• DeCo-Bench contains 200 test cases across 2–6 reference images, while MICo-Bench provides complementary evidence beyond this reward-aligned evaluation.The current model was trained for up to six reference images, and reliability beyond that range has not been demonstrated.Source: B Additional Validation and Discussion · p. 19

Choice of RL optimization and auxiliary reward

• The auxiliary VLM reward discourages implausible global compositions and errors that a local crop-based score may miss.Diffusion NFT is the practical optimizer used; the object-level reward design is described as orthogonal to the choice of diffusion RL algorithm, with no matched-training controlled comparison reported.Source: B Additional Validation and Discussion · p. 19

Limitations

• The decomposed reward primarily captures localizable object identity and appearance, not abstract styles, spatial relations, or interaction and binding constraints.The authors also identify possible grounding or same-category matching failures, VLM bias or reward hacking, and a relatively small DeCo-Bench.Source: B Additional Validation and Discussion · p. 19
• The main experiments use one editing backbone and 2–6 reference images, while the pipeline’s multiple large models raise computational and reproducibility costs.A controlled holistic-versus-object-level reward comparison under matched RL training remains future work.Source: B Additional Validation and Discussion · p. 20