Source-linked AI summary
Visual Jev Rewards: Reference-Bound Verification for Multi-Subject Image Generation
Baoteng Li, Wenzhuo Wu, Kongming Liang, Zhanyu Ma
TL;DR
Multi-subject image generation needs rewards that check whether the specified reference subjects participate in requested attributes, actions, and relations. The paper trains a reference-bound Visual Jev verifier and uses its judgments as GRPO rewards. An initial study raises the GPT-5.4 composite score from 41.78 to 52.50 on a manually selected MICo-Bench subset, while the authors note that broader, controlled evaluation is still needed.
Problem
Subject presence and action presence alone do not establish that the specified reference subjects participate in the requested interaction.
Source: §1 · p. 1Method
The framework uses fixed questions and binary visual supervision to train a verifier whose Yes probabilities score reference-bound conditions for GRPO.
Source: §1 · p. 1, §3.3 · p. 4, §3.4 · p. 4Results
With 200 training tasks and 30 updates, the GPT-5.4 composite score rises from 41.78 to 52.50 on the manually selected 897-task MICo-Bench subset.
Source: §6 · p. 8Takeaways & Limitations
The study provides an initial working path from reference-bound visual decisions to generative-model post-training while retaining individual judgments for inspection.
Source: §6 · p. 8The manually screened 897-task subset favors higher-quality examples, and independent human evaluation of trained generators remains necessary.
Source: §7 · p. 8
Abstract
from arXiv · showhide
Multi-subject image generation requires rewards that verify whether requested attributes, actions, and relations hold for the specified reference subjects. Subject presence alone does not establish that the correct subjects participate in a requested interaction. We present reference-bound Visual Jev rewards that turn these visual decisions into generator training signals. Each subject-related question receives a positive label only when the requested condition and the relevant reference identities hold jointly. We construct fixed questions offline, train a Qwen3.5-4B verifier with binary supervision, and directly read Yes probabilities from its language-model head. Their mean supplies a GRPO reward while retaining individual judgments for inspection. Using 200 MICo-150K training tasks and 30 updates, the framework raises a GPT-5.4 composite score from 41.78 to 52.50 on a manually selected 897-task MICo-Bench subset; direct 27B rewards yield 51.84. Each reward is tested in one GRPO run, and offline human evaluation does not establish a statistically significant advantage over direct scoring. The study provides an initial implementation and evaluation of Visual Jev as a reference-bound reward for multi-subject image generation.
1. Introduction
Multi-subject generation needs rewards that verify whether the specified reference subjects participate in requested conditions, not merely whether subjects and actions appear. This work makes reference identity part of each subject-related judgment and turns fixed visual decisions into GRPO rewards.
1. Introduction
- A candidate can contain both reference subjects and show the requested action yet fail the interaction if the wrong subjects participate.The reward must judge whether the specified subjects are participants in the specified conditions.Source: §1 · p. 1
- Separating subject consistency from instruction following can obscure whether the correct subjects participate in the requested interaction.Question-based verification makes requirements explicit, but multi-reference questions must also identify which reference subject each condition concerns.Source: §1 · p. 1
- Reference-bound atomic judgments require both the requested condition and the identities of its participants for a positive answer.The paper expresses this requirement in question wording and supervision labels, then trains a local visual verifier to evaluate the conditions.Source: §1 · p. 1
- Fixed task questions and a task-trained verifier provide averaged Yes probabilities as a GRPO reward, with individual scores retained for inspection.The verifier and question sets remain fixed while the generator is updated; only the generator LoRA is updated during GRPO.Source: §1 · pp. 1–2, Figure 1 · p. 2
2. Related Work
Prior work develops multi-subject composition, fine-grained visual questions, and verification-based image rewards. This paper builds on those approaches by requiring specified reference identities within each relevant subject-related judgment.
2.1. Multi-Subject Generation and Composition
- UNO studies multi-subject generalization, MICo-150K supplies composition data and MICo-Bench, and PSR uses subject-consistency rewards with general rewards [13], [12], [11].The paper distinguishes its focus as jointly judging identity and semantics: requested actions or relations must involve the reference-specified subjects.Source: §2.1 · p. 2
2.2. Atomic Visual Verification and Jev
- TIFA and DSG use decomposed visual questions and dependent judgments, while Jev offers structured decision interfaces and Visual Jev applies answer-probability readout to visual evidence,, [14].This work makes reference identity a necessary condition within each relevant subject-related judgment.Source: §2.2 · p. 2
2.3. Verification Rewards for Image Synthesis
- Flow-GRPO, EditScore, Edit-R1, and AlphaGRPO explore reward-based image generation, with DVReward aggregating normalized Yes/No probabilities [7],, [4], [6].Unlike candidate-specific rationales verified by EVR [9], this work fixes task requirements before generation and trains a verifier to score the same conditions across candidates.Source: §2.3 · p. 3
3. Reference-Bound Visual Jev Rewards
Reference-bound Visual Jev turns visual requirements into a GRPO reward by judging each condition together with the identities of its specified subjects. Fixed questions, a trained verifier, and aggregated Yes scores connect task inputs to generator updates.
3. Reference-Bound Visual Jev Rewards
- Reference-bound Visual Jev makes correct reference identity necessary for a positive subject-related judgment of an attribute, action, or relation.The framework composes these judgments into a reward for generation.Source: §3 · p. 3
3.1. Task Definition
- The task pairs a text prompt with ordered references; the verifier is trained first and fixed during generator optimization.References specify visual identities, while the prompt specifies attributes, actions, relations, and other scene requirements.Source: §3.1 · p. 3
3.2. Atomic Questions Bound to References
- GPT-5.4 constructs atomic questions offline from the prompt and references, and all candidates for a task share those fixed requirements.Questions should be grounded in input requirements, individually answerable, and cover requirements while reducing duplication.Source: §3.2 · p. 3
3.3. Identity-Dependent Judgment Criteria
- A subject-related question is positive only when its condition holds for the correct reference participants; matching subjects elsewhere in the image is insufficient.The identity constraint applies only to subjects involved in that question, and the target-label rule does not guarantee that the verifier always implements it correctly.Source: §3.3 · pp. 3–4
3.4. Visual Jev Implementation and Training
- A Qwen3.5-4B verifier uses LoRA and binary cross-entropy supervision to read candidate-answer probabilities from its language-model head.The verifier is trained separately from the image generator, then frozen for offline scoring and reinforcement learning.Source: §3.4 · p. 4
3.5. Image-Level Reward Aggregation
- The image-level reward is the arithmetic mean of atomic Yes scores, with equal question weights and individual scores retained for diagnosis.The mean can let high scores on other conditions partly compensate for failure on a critical requirement.Source: §3.5 · p. 4
3.6. GRPO with Atomic Rewards
- GRPO updates the generator using group-relative advantages from image rewards while the verifier parameters and task questions remain fixed.The local verifier scores newly generated candidates during training, supplying one scalar reward per image.Source: §3.6 · pp. 4–5
4. Data and Experimental Setup
The experiments use a verifier dataset and a 200-task MICo-150K training set, with fixed questions and specified generator and verifier configurations. Evaluation uses fixed candidates for offline comparisons and a manually screened 897-task MICo-Bench subset for downstream scoring.
4.1. Datasets
- The verifier dataset contains 3,972 binary examples, while GRPO uses 200 MICo-150K tasks with 549 reference images and 1,634 atomic questions.Verifier examples include original candidates, candidate comparisons, and reference-replacement examples; questions for GRPO tasks are constructed offline and fixed during training.Source: §4.1 · p. 5
4.2. Models and Training
- Offline reward and GRPO experiments use the development-selected Qwen3.5-4B verifier checkpoint at update 136; GRPO training completes 30 updates.The direct-reward control uses Qwen3.5-27B, scoring instruction following and subject consistency from 0 to 5 each and dividing their sum by 10.Source: §4.2 · p. 5
4.3. Evaluation
- Offline scoring compares Visual Jev with direct Qwen3.5-4B and Qwen3.5-27B scoring on fixed candidates and blinded human rankings; downstream evaluation covers 897 manually selected MICo-Bench tasks.The downstream GPT-5.4 composite averages scores for naturalness, artifacts, instruction following, and subject similarity across the selected tasks.Source: §4.3 · p. 5
5. Experimental Results
The experiments first test Visual Jev as a GRPO reward, then characterize its verifier decisions, human-preference agreement, and aggregation behavior.
5.1. GRPO with Visual Jev Rewards
- 52.50 versus 41.78: after 30 updates, Visual Jev raises the GPT-5.4 composite score by 10.72 points on 897 selected MICo-Bench tasks.Direct 27B rewards score 51.84; each reward is tested in one run, and the comparison does not isolate the design components.Source: §5.1 · pp. 5–6, Table 1 · p. 6, Figure 2 · p. 6
- Visual Jev beats baseline in all four categories; against direct 27B rewards, it gains 0.44 on Object, 0.34 on Human, and 1.75 on HOI, while losing 0.09 on De&Re.The larger HOI gain is consistent with the focus on subjects and interactions, but the experiments do not establish that identity-dependent judgments caused it.Source: §5.1 · pp. 5–6, Figure 2 · p. 6
5.2. Atomic Verification
- On 936 test examples, BCE reaches 96.15% accuracy and 96.04% balanced accuracy, versus 82.05% and 84.14% for untuned Qwen3.5-4B.Its Brier score falls from 0.1332 to 0.0356; the fixed-budget comparison uses update 544, while reward experiments use the development-selected update-136 checkpoint.Source: §5.2 · p. 6, Table 2 · p. 7
5.3. Human Preference Agreement
- A post-hoc four-candidate example shows Visual Jev selecting the human’s first choice, while per-question probabilities expose identity and relation failures for inspection.The direct scorer prefers a candidate ranked last by the human; the example illustrates scoring evidence, not overall performance.Source: Figure 3 · p. 7, §5.3 · p. 7
- Visual Jev reaches 63.49% strict preference agreement, versus 60.32% for direct 27B and 59.13% for direct 4B scoring on fixed candidates.The paired-bootstrap intervals for agreement differences against both direct scorers cross zero; severe identity risk does not improve correspondingly.Source: §5.3 · p. 7, Table 3 · p. 7
5.4. Reward Aggregation
- On validation, category mean and the worst-category mixture each reach 65.08% agreement, versus 63.49% for atomic mean; development results tied, and a prespecified tiebreak selected atomic mean.The sample is too small to establish a consistently optimal rule, so GRPO retains the frozen atomic mean.Source: §5.4 · p. 8, Table 4 · p. 8
6. Conclusion
The paper implements reference-bound Visual Jev rewards that require correct reference identities for positive subject-related judgments. An initial GRPO study improves the GPT-5.4 composite score on a manually selected MICo-Bench subset, while further evaluation is needed to establish stability and isolate the source of improvement.
6. Conclusion
- Reference-bound Visual Jev makes correct participant identities necessary for positive subject-related judgments of visual conditions.Fixed task questions, a task-trained visual verifier, and direct answer-probability readout produce an image-level reward while preserving individual decisions for inspection.Source: §6 · p. 8
- 41.78 to 52.50: an initial GRPO study with 200 training tasks and 30 updates raises the GPT-5.4 composite score on the manually selected 897-task MICo-Bench subset.Larger independent evaluations and controlled ablations are needed to establish when the design offers a stable advantage and how much improvement comes from identity binding.Source: §6 · p. 8
- The study presents an initial working path from reference-bound visual decisions to generative-model post-training, but does not establish a stable advantage or isolate identity binding’s contribution.The GRPO comparison uses 200 training tasks and 30 updates, with larger independent evaluations and controlled ablations identified as needed.Source: §6 · p. 8
7. Limitations
The evaluation is preliminary: each GRPO method is tested in one run, and offline human validation is limited. The study also leaves component contributions, broader generalization, and the full-benchmark performance unresolved.
7. Limitations
- One GRPO run per method and 30 updates, alongside 21 valid offline human-validation tasks, limit the strength of conclusions about reward performance.Preference-agreement confidence intervals cross zero, and the offline comparison does not reduce severe identity risk.Source: §7 · p. 8
- The complete-system comparison does not isolate verifier training, question decomposition, identity-dependent labels, or aggregation as sources of the observed results.Repeated training and controlled component ablations are needed to test stability and identify the contribution of design choices.Source: §7 · p. 8
- Reward quality depends on question coverage and binary-annotation quality, and generalization to broader subjects, tasks, and reference configurations remains unevaluated.GPT-5.4 contributes to question construction, annotation, and downstream evaluation; the manually screened 897-task subset also limits conclusions about the full benchmark.Source: §7 · p. 8