Source-linked AI summary
Multimodal RewardBench 2: Evaluating Omni Reward Models for Interleaved Text and Image
Yushi Hu, Reyhane Askari-Hemmat, Melissa Hall, Emily Dinan, Luke Zettlemoyer, Marjan Ghazvininejad
TL;DR
Reward models are important in LLM development, but their evaluation for omni models handling interleaved text and images remains underexplored. MMRB2 benchmarks four multimodal tasks with expert-annotated preference pairs and evaluates existing judges and reward models. Gemini 3 Pro reaches 74–80% agreement on multimodal generation tasks, while disagreement remains substantial and human evaluation remains essential.
Problem
Reward modeling for omni models remains underexplored, and no existing benchmark measures their quality across interleaved text-and-image sequences.
Source: §1 · p. 1Method
MMRB2 provides four-task evaluation using expert-annotated preference pairs curated from challenging prompts and state-of-the-art model and agent responses.
Source: §1 · p. 2Results
74–80% agreement is achieved by Gemini 3 Pro on multimodal generation tasks, while 20–26% disagreement remains and human evaluation remains essential.
Source: §5 · p. 12Takeaways & Limitations
MMRB2 establishes a challenging benchmark intended as a foundation for future omni reward-modeling, evaluation, and post-training research.
Source: §5 · p. 12MMRB2 focuses on core settings and overall human preferences in interleaved text–image scenarios, with extensions proposed for additional dimensions, task formats, and modalities.
Source: §5 · p. 12
Abstract
from arXiv · showhide
Reward models (RMs) are essential for training large language models (LLMs), but remain underexplored for omni models that handle interleaved image and text sequences. We introduce Multimodal RewardBench 2 (MMRB2), the first comprehensive benchmark for reward models on multimodal understanding and (interleaved) generation. MMRB2 spans four tasks: text-to-image, image editing, interleaved generation, and multimodal reasoning ("thinking-with-images"), providing 1,000 expert-annotated preference pairs per task from 23 models and agents across 21 source tasks. MMRB2 is designed with: (1) practical but challenging prompts; (2) responses from state-of-the-art models and agents; and (3) preference pairs with strong human-expert consensus, curated via an ensemble filtering strategy. Using MMRB2, we study existing judges for each subtask, including multimodal LLM-as-a-judge and models trained with human preferences. The latest Gemini 3 Pro attains 75-80% accuracy. GPT-5 and Gemini 2.5 Pro reach 66-75% accuracy, compared to >90% for humans, yet surpass the widely used GPT-4o (59%). The best performing open-source model Qwen3-VL-32B achieves similar accuracies as Gemini 2.5 Flash (64%). We also show that MMRB2 performance strongly correlates with downstream task success using Best-of-N sampling and conduct an in-depth analysis that shows key areas to improve the reward models going forward.
1 Introduction
Reward modeling for omni models remains underexplored, in part because no benchmark covers their interleaved text-and-image sequences and diverse evaluation challenges. MMRB2 addresses this gap with four subtasks and expert-annotated preference pairs, then evaluates existing judges and reward models, finding substantial performance gaps and error patterns.
1 Introduction
- Omni reward modeling remains underexplored, and existing benchmarks are lacking for evaluating models that process arbitrarily ordered sequences of text and images.Images are difficult to verify automatically, preference annotation requires carefully designed protocols, and the applications call for diverse task coverage.Source: §1 · p. 1
- MMRB2 covers text-to-image, image editing, interleaved generation, and multimodal reasoning with 1,000 expert-annotated preference pairs per subtask.Its prompts span 21 existing and newly created tasks; responses come from state-of-the-art models and specialized agents, and pairs have over 90% human-expert agreement while remaining challenging for current judges.Source: §1 · p. 2
- 74–80% accuracy across all subtasks is reached by Gemini 3 Pro, while GPT-5 and Gemini 2.5 Pro reach 66–75% and humans exceed 90%.GPT-4o attains 51–65% accuracy across all subtasks.Source: §1 · p. 3
- 58.3% and 54.0% accuracy on text-to-image evaluation are achieved by VQAScore and ImageReward, below Qwen3-VL-32B at 64.1% and Gemini 3 Pro at 74.4%.Human-preference training improves performance over heuristic metrics, but these models still fall short of frontier MLLMs.Source: §1 · p. 3
- MMRB2 performance strongly correlates with downstream task performance when reward models select outputs using Best-of-N.The reported comparisons cover GenAI-Bench, GEdit-Bench, ISGBench, and EMMA.Source: §1 · p. 3
- Judges show up to 12% higher agreement with human preferences on different-model pairs than same-model pairs, alongside reasoning-task performance gaps of 27.7–49.3% by preferred response modality.In reasoning pairs, judges favor image-containing responses over text-only preferred responses.Source: §1 · p. 3
2 Related Work
Prior multimodal reward-model work includes task-specific preference models for visual generation and broader cross-task approaches, while evaluation benchmarks and judge training largely focus on image-text-to-text tasks. MMRB2 targets the remaining evaluation gap for interleaved text and images.
2 Related Work
- ImageReward, HPSv2 and v3, and PickScore learn human preferences for text-to-image generation; EditScore and EditReward apply similar frameworks to image editing.UnifiedReward trains a single multimodal reward across image, video, and understanding tasks, while most multimodal reward models remain task-specific.Source: §2 · p. 4
- RewardBench, RewardBench 2, VL-RewardBench, and Multimodal RewardBench assess reward models across text or image-text-to-text settings, while most evaluations offer little insight into interleaved text and image.VQAScore and related generation metrics probe image-text alignment, and their annotations are often used to evaluate reward models.Source: §2 · p. 4
- MMRB2 provides a unified, challenging framework for assessing omni-model reward modeling beyond existing image-text-to-text-focused benchmarks.The related work also describes efforts to improve reward modeling through judge training and test-time scaling.Source: §2 · p. 4
3 Multimodal RewardBench 2
MMRB2 evaluates omni reward models against human preferences across four multimodal task types, using prompts and chosen/rejected response pairs.
3.1 Tasks in MMRB2
- MMRB2 covers text-to-image, image editing, interleaved generation, and multimodal reasoning, asking reward models to select preferred images, edits, outputs, or reasoning trajectories.Interleaved outputs are judged for coherence, global planning, and coordination between visual and textual content.Source: §3 · p. 4, §3.1 · pp. 4–5
3.2 Prompt and response collection
- Prompts are stratified by difficulty and subtask from benchmark test splits, supplemented with new tasks; each task contributes 1,000 prompts and responses from 7–11 models.When strong models struggled on interleaved generation and multimodal reasoning, the authors also constructed agents that could call Python and image tools.Source: §3.2 · p. 5
3.3.1 Image generation, editing & interleaved tasks
- Nine multimodal judges assess each pair in both orders, and pairs with at least 90% majority agreement across the orderings are filtered as easy.The diverse judge ensemble and high threshold are intended to limit bias from any single judge.Source: §3.3.1 · p. 6
3.3 Human preference annotations
- Three blinded annotators assess each response using task-specific criteria, then provide an overall preference; high-disagreement, ambiguous, and tied pairs are excluded.Pairs are removed when rating spread exceeds four points, the average falls within 3.0–4.0, or the majority vote is tied.Source: §3.3.1 · p. 6
3.3.1 Image generation, editing & interleaved tasks
- 95.6% overall inter-annotator agreement (excluding ties) was achieved across the three generation tasks, with rates of 95.3%, 96.3%, and 95.2%, respectively.After filtering, 1,000 pairs per task remained, approximately 50% of the initial set.Source: §3.3.1 · p. 6
3.3.2 Multimodal reasoning task
- For multimodal reasoning, annotators verify responses individually, then pairs contrast unanimously correct answers with sound reasoning against responses with flawed reasoning or incorrect answers.Samples are balanced between text-only reasoning and reasoning with both images and text.Source: §3.3.2 · pp. 6–7
3.4 Evaluation Method
- Each judge evaluates both response orders, and each judgment is scored against the human preference, including tie-to-tie matches.Human preferences use majority votes for Tasks 1–3 and unanimous agreement on reasoning and answer correctness for Task 4.Source: §3.4 · p. 7
4 Experiments
The experiments compare multimodal judges and preference-trained evaluators, then examine downstream prediction, pairwise error patterns, and test-time scaling. Gemini 3 Pro performs best across tasks, while the analyses identify substantial task-specific difficulty and only modest scaling gains.
4.1 Performance of MLLM-as-a-judge
- Judges include API-based and open-source models evaluated with task-specific prompts and detailed rubrics, using positional-consistent dual evaluation to mitigate positional bias.Source: §4.1 · p. 7
- Gemini 3 Pro leads across all tasks, while multimodal reasoning is hardest for other judges: top API models reach 52–70% agreement on reasoning versus 63–75% on generation tasks.The Table 2 caption identifies Gemini 3 Pro as best across tasks and Qwen3-VL-32B as the best open-source model.Source: §4.1 · p. 7, Table 2 · p. 8
4.2 Performance of supervised reward models
- Preference training improves evaluator accuracy: EditReward reaches 67.2 on editing versus 57.8 for the Qwen2.5-VL-7B judge, a +9.4% gain.The evaluated preference-trained models include reward models sharing base architectures with MLLM baselines and CLIP-based models.Source: §4.2 · p. 9
- Most preference-trained reward models still underperform Qwen3-VL-32B, and earlier-generation training data may contribute to weaker accuracy on more capable recent outputs.EditReward is a stated exception on editing, where it scores 67.2 versus 66.4.Source: §4.2 · p. 9
4.3 Correlation with downstream tasks
- MMRB2 scores strongly correlate with downstream best-of-N performance, with Pearson’s r above 0.8 for all tasks and models.The analysis uses two base models per task and excludes GPT-4o and GPT-4.1; it removes ISG-Bench preference pairs when scoring interleaved generation to avoid leakage.Source: Figure 4 · p. 9, §4.3 · p. 10
4.4 Fine-grained analysis of errors
- Across image generation, editing, and interleaving, top judges agree more with humans on different-model than same-model pairs, with gaps of 5–13 percentage points.For image generation, Gemini 3 Pro reaches 79.7% on different-model pairs and 70.4% on same-model pairs.Source: Table 4 · p. 10, §4.4 · p. 10
- On multimodal reasoning mixed-modality pairs, judges favor image-containing responses; GPT-5’s accuracy differs by 49.3 points depending on which modality annotators preferred.The reported comparison is 88.2% when annotators prefer the image-containing response versus 38.9% when they prefer text.Source: Table 5 · p. 11, §4.4 · p. 11
4.5 Test-time scaling of rewards
- At K=9, test-time scaling raises Gemini 2.5 Pro’s accuracy from 71.3% to 72.5%, while Qwen3-VL models show no measurable improvement.GPT-4o, Gemini 2.5 Flash, GPT-5, and Gemini 2.5 Pro improve by roughly 0.8–1.2% at K=9.Source: Figure 5 · p. 11, §4.5 · p. 12
5 Conclusion
MMRB2 is a four-task benchmark for omni reward models, and Gemini 3 Pro achieves 74–80% agreement on multimodal generation tasks. The benchmark focuses on core interleaved text–image settings, leaving additional evaluation dimensions and modalities for extension.
5 Conclusion
- MMRB2 is the first comprehensive benchmark for omni reward models, spanning text-to-image, image editing, interleaved generation, and multimodal reasoning.Source: §5 · p. 12
- Gemini 3 Pro achieves 74–80% agreement on multimodal generation tasks, but its 20–26% disagreement with humans means human evaluation remains essential.The conclusion notes that GPT-5 and other models lag significantly behind Gemini 3 Pro.Source: §5 · p. 12
- MMRB2 focuses on core settings and overall human preferences in interleaved text–image scenarios; safety- and bias-sensitive preferences, multilingual and agentic tasks, video, and audio are extensions.Source: §5 · p. 12
A.1 Performance by task source and pair type
Pairwise judge agreement varies by source benchmark within each task, with especially pronounced differences in image editing and multimodal reasoning.
Image Generation
- Realunify scores 48–78.5%, oneigbench reaches 73–74% for top models, and wise yields the lowest scores at 47–66%.Source: §A.1 · p. 18, Table 6 · p. 18
Image Editing
- Text-based editing benchmarks score 54–83% and 48–83%, versus 49–72% for emu-edit and 51–71% for multi-image editing.The text-based ranges are for text and risebench, respectively.Source: §A.1 · p. 18, Table 7 · p. 19
Interleaved
- Isgbench scores 76–79% for frontier models, and leading judges’ scores across interleaved benchmarks cluster within 5–8 percentage points.Source: §A.1 · p. 18, Table 8 · p. 19
Reasoning
- Muirbench scores 36–76%, while vstar scores 30–52%; blink and mindcube cluster at 44–55% for most models.The source describes blink and mindcube as representing a baseline reasoning difficulty.Source: §A.1 · p. 18
A.2 Win rate analysis on generations
The analysis compares human-preference win rates for generators on the same outputs evaluated by judges. Leaders vary across image generation, image editing, and interleaved generation.
Image Generation
- 60.4%: GPT-Image-1 leads text-to-image generation, while Stable Diffusion 3.5 Large (41.0%) and FLUX (36.8%) trail by 19–24 points.Imagen 4 follows at 57.4%, Imagen 4 Ultra at 56.5%, and Gemini 2.5 Flash at 54.3%.Source: §A.2 · p. 19, Table 10 · p. 20
Image Editing
- 59.2%: Gemini 2.5 Flash leads image editing, ahead of GPT-Image-1 (53.2%) and purpose-built Imagen Edit (35.2%).FLUX-Kontext reaches a 49.0% win rate as an open-source solution.Source: §A.2 · pp. 19–20, Table 10 · p. 20
Interleaved Generation
- 57.1%: GPT-Gemini Agent narrowly leads interleaved generation, followed by GPT-Image Agent (56.9%) and Gemini 2.5 Flash (53.2%).GPT-FLUX Agent reaches 40.4%; the passage notes its performance depends critically on component model quality.Source: §A.2 · p. 20, Table 10 · p. 20
B.1 Tasks 1-3
Tasks 1–3 use paired-response judgments with task-specific quality criteria, pointwise ratings, and an overall preference that is aggregated across annotators.
B.1 Tasks 1-3
- Annotators compare two responses to a prompt, rate each response on task-specific dimensions, then record an overall preference and rationale.The interface also includes a prompt-quality check.Source: §B.1 · pp. 20, 22
- Ratings cover instruction faithfulness and image quality; Tasks 2–3 add input-image faithfulness, while Task 3 adds cross-generation congruence, text-image alignment, and generated-text quality.Source: §B.1 · p. 22, §B.1.1 · p. 24
B.1.1 Details of each question
- The image-quality rubric ranges from severe flaws to no noticeable technical or logical flaws, while congruence concerns consistency of recurring elements, style, and context.Source: §B.1.1 · pp. 23–24
- Overall preferences use a 7-point Likert scale, mapped to A-preferred, B-preferred, or tie, with the final pair preference decided by majority vote across three annotators.Automated checks flag disagreements for manual review, including comparison of sibling tasks and correction when needed.Source: §B.1.1 · p. 25
B.2 Task 4
Task 4 evaluates multimodal reasoning by annotating answer and rationale correctness, then constructing preference pairs from responses with contrasting correctness or reasoning quality.
B.2 Task 4
- Annotators judge whether the model’s reasoning or rationale is correct and consistent, distinguishing sound, improvable, or flawed reasoning, absent reasoning, and incorrect or unverifiable answers.Source: §B.2 · p. 25
- Preferred responses require unanimous agreement that the answer is correct and reasoning has no major errors; rejected responses include incorrect reasoning or an incorrect answer with reasoning.Responses with annotator disagreement about reasoning accuracy are discarded.Source: §B.2 · p. 25
- Pairs may contain two text-only responses, two image-and-text responses, or one of each; no model response is duplicated across pairs.Table 11 breaks pair counts down by modality combination and pair type.Source: §B.2 · p. 25, Table 11 · p. 26
C Details for Prompts, Response Generation, and MLLM-as-a-judge
This appendix details prompt construction and response generation for MMRB2’s multimodal tasks, including interleaved text–image sequences. It also describes task-specific MLLM-as-a-judge prompts, rubrics, scoring guidance, and structured outputs for multimodal comparisons and reasoning.
C.1 Task Prompts
- The text-heavy editing task draws on 200 real DreamBench++ images (Peng et al., 2025) and 500 synthetic images from GPT-Image (OpenAI, 2025c) and Gemini-2.5-Flash-Image (Google DeepMind, 2025a), all manually inspected.Source: §C.1 · p. 26
PROMPT
- 114 text-heavy editing examples target practical edits that require rendering explicitly specified text, with instructions containing 10–50 words.The prompt asks for diverse, realistic edits such as posters, comics, or slides.Source: PROMPT · p. 27
- 178 multi-image editing examples comprise 79 with two input images and 99 with three, paired with an instruction to produce one output image.The output image is not included in the benchmark.Source: PROMPT · pp. 27–28
C.2 Response generation
- Responses use a unified text-image format and official default sampling settings, with temperature 1.0 in most cases.The interleaved-task system prompt directs agents to place text and images in the requested order, such as one text-image pair per step.Source: §C.2 · pp. 28–29
- To address model failures such as producing the wrong number of images, the authors collect outputs from multimodal tool-using agents following Visual Sketchpad (Hu et al., 2024).GPT-4.1 (OpenAI, 2025a), o3 (OpenAI, 2025d), and GPT-5 (OpenAI, 2025b) serve as LLM backbones; tool outputs return text and images for further planning and reasoning.Source: §C.2 · p. 28
- Agent variants use different image tools to broaden the range of interleaved outputs.They pair FLUX.1-dev and FLUX.1-Kontext (Labs et al., 2025), Imagen-4-Ultra (Google DeepMind, 2025d) and Imagen-3-Edit (Baldridge et al., 2024), GPT-Image-1 (OpenAI, 2025c), or Gemini 2.5 Flash Image (Google DeepMind, 2025a) for generation and editing.Source: §C.2 · p. 28
- For multimodal reasoning, the system prompt enables Python and image-generation tools to inspect or enhance visual reasoning, including cropping images to examine small details.It also describes conditioning image generation on existing images, for example to produce another view of an object.Source: §C.2 · p. 30
- Agents can use `python_exec` to load, crop, save, or annotate images, while `save_image` returns an image and its index because the sandbox cannot display images directly.Annotations can include lines, boxes, and labels to aid image reasoning.Source: §C.2 · pp. 30–31
- The `generate_image` tool accepts a detailed 15–30-word prompt and optional image references; prior images must be explicitly referenced to condition on them.It can generate or edit images conditioned on multiple images.Source: §C.2 · pp. 30–31
- Agents have at most 15 turns, and their outputs are parsed into interleaved text–image sequences using image tags indexed across the dialogue.The instructions encourage showing both the reasoning process and final result through interleaved text and images.Source: §C.2 · p. 31
C.3 MLLM-as-a-judge details
- The evaluation instructions call for step-by-step comparison of both responses on a 1–6 scale and a confidence rating from 0.0 to 1.0, with most comparisons assigned confidence scores of 0.2–0.5.Source: §C.3 · p. 32
- For image-generation comparisons, the judge assesses prompt faithfulness, rendered-text quality, source-image faithfulness when applicable, and consistency across generated images when multiple images appear.The rubric also evaluates text–image alignment, linguistic text quality, and overall technical and aesthetic quality.Source: §C.3 · p. 32
- The image-generation judge also rates overall quality, including realism, coherence, and visual artifacts or distortions.It compares responses to the same prompt, considering the prompt, any input image, and both generated responses.Source: §C.3 · pp. 31–32
- For image editing, the rubric covers adherence to editing instructions, preservation of input-image elements, overall image quality, and rendered text when present.Input-image preservation includes composition, lighting, style, and background elements.Source: §C.3 · p. 34
- For interleaved generation, the judge evaluates instruction faithfulness, image faithfulness when an input image exists, and the quality of both modalities and their coherence.The prompt specifies a fine-grained rubric and detailed comparative reasoning for each criterion.Source: §C.3 · p. 35
- The multimodal rubric evaluates image quality, cross-image congruence, text-image alignment, generated-text quality, and rendered-text correctness, with conditional “Not Applicable” responses.The rubric also includes separate text- and image-faithfulness fields in its output schema.Source: §C.3 · pp. 35–36
- Judges assign a 1–6 score from A significantly better to B significantly better, with middle scores representing marginal or negligible differences.Scores 4 and 3 respectively indicate A or B is slightly better or roughly equivalent.Source: §C.3 · p. 36
- Confidence should usually be 0.2–0.5, default to 0.3–0.5, and use scores above 0.6 sparingly because quality judgments are subjective.The prompt reserves very high confidence for obvious, ambiguity-free cases and says to use it in fewer than 10% of cases.Source: §C.3 · p. 36
- The multimodal judge must return criterion-level reasoning, a comparison summary, score, better-response label, confidence, and confidence rationale in structured JSON.The score is an integer from 1 to 6; the better-response field is A or B, and confidence is a float from 0.0 to 1.0.Source: §C.3 · pp. 36–37
- For reasoning comparisons, the judge weighs helpfulness, relevance, accuracy, depth, and detail while avoiding position, response-length, and assistant-name bias.It is instructed to compare the responses, briefly explain its choice, and judge objectively.Source: §C.3 · p. 37
- The reasoning judge returns its explanation and better-response choice in JSON, with the choice specified as A or B.Source: §C.3 · p. 37
- The appendix states that the MLLM judge prompts are very close to the rubrics used for human annotations.Source: §C.3 · p. 37
D Limitations and Future Directions
MMRB2’s current release is a focused benchmark for text–image omni-model reward evaluation, covering core generation, editing, interleaving, and image-reasoning tasks. The authors outline extensions across modalities, interaction formats, data sources, evaluation dimensions, and judge ensembles.
Scope and focus
- The current release focuses on core text–image omni-model tasks and overall human preference rather than finer-grained dimensions.Its task scope includes text-to-image generation, image editing, interleaved text–image generation, and multimodal reasoning over images.Source: D Limitations and Future Directions · p. 37
Modalities and task formats
- The construction pipeline is modality-agnostic, but current tasks are predominantly single-turn; video, audio, 3D, multi-turn, and agentic extensions are proposed.The same construction recipe could be applied to additional modalities as use cases and tools become more standardized.Source: D Limitations and Future Directions · p. 37
Data sources and coverage
- Prompts come primarily from established benchmarks and designed variants, leaving in-the-wild, domain-specific, and multilingual settings for complementary extensions.The authors describe MMRB2 as a backbone that specialized or application-driven subsets can build upon.Source: D Limitations and Future Directions · p. 37
Evaluation dimensions
- The present benchmark measures overall instruction-level preference; safety-, bias-, adversarial-robustness, and fairness-sensitive subsets would require adapted prompts and guidelines.The authors expect such specialized subsets to broaden MMRB2’s applicability for alignment and safety research.Source: D Limitations and Future Directions · p. 38
Evolving judges and benchmarks
- The filtering uses contemporary judges, while its modular design supports refreshing judge ensembles, adding model families, and adding tasks as systems evolve.The authors describe this design as preserving compatibility with the benchmark’s core principles.Source: D Limitations and Future Directions · p. 38
E Examples
The examples show preference rationales across text-to-image, image editing, interleaved generation, and multimodal reasoning, including instruction fit, content accuracy, and consistency.
Text-to-image
- Text-to-image preferences favor a railway underpass over an output that is not one, and the requested number of people over an image with only five.The examples compare GPT-Image-1 with FLUX, and two outputs from Gemini 2.5 Flash Image.Source: Figure 8 · p. 38, Figure 9 · p. 38
Image editing
- Image-editing preferences favor preserving important text despite small-text errors, and better instruction following with an anime-styled backpack.The examples compare Gemini 2.5 Flash Image with GPT-Image, and two outputs from Gemini 2.0 Flash Image.Source: Figure 10 · p. 39, Figure 11 · p. 39
Interleaved generation
- Interleaved-generation preferences favor instruction adherence and consistency with the original image, as well as natural-looking cats that change across images while remaining consistent.The examples compare outputs from the GPT-Image agent and two outputs from Gemini 2.5 Flash Image.Source: Figure 12 · p. 40, Figure 13 · p. 42
Multimodal reasoning
- Multimodal-reasoning preferences favor correct reasoning and answers, and complete analysis across images; the examples compare Gemini 2.5 Pro with GPT-4.1 and sketchpad agents using GPT-5 and o3 backbones.The rejected responses contain an incorrect description of a circle and omit analysis for the third image, respectively.Source: Figure 14 · p. 43, Figure 15 · p. 45
Related papers
This paper cites
- Learning Transferable Visual Models From Natural Language Supervision2021
- DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning2025
- Self-Consistency Improves Chain of Thought Reasoning in Language Models2022
- Let's Verify Step by Step2023
- Self-Instruct: Aligning Language Models with Self-Generated Instructions2022
- CLIPScore: A Reference-free Evaluation Metric for Image Captioning2021
Cited by
- How Far Can Coding Agents Go in Visual Generation?2026
- DuplexBridge: Coordinating Understanding and Generation for Unified Image Editing2026
- Unsupervised Multimodal Reward Modeling2026
- Perceptually Grounded and Semantics-Aware Evaluation for Holistic Co-Speech Gesture Generation2026
- Same Trajectory, Contradictory Rewards (ROBORMBENCH): Paraphrase Fragility in Vision Language Reward Models2026
- RubricRM: Generative Reward Modeling via Dynamic Rubrics for Image Generation and Editing2026