PaperlayerView exact revision on arXiv

Source-linked AI summary

OmniVerifier-M1: Multimodal Meta-Verifier with Explicit Structured Recalibration

Xinchen Zhang, Bowei Liu, Jiale Liu, Chufan Shi, Yizhen Zhang, Junhong Liu, Youliang Zhang, Zhiheng Li, Yujiu Yang, Ling Yang

arXiv:2605.28805v1cs.CLcs.AIcs.CVcs.LG

TL;DR

Fine-grained verification of visual outcomes remains difficult because binary judgments and coarse critiques provide limited rationale. OmniVerifier-M1 addresses this with symbolic meta-verification and decoupled reinforcement-learning objectives, outperforming textual rationales and joint training while supporting region-level correction.

  • Problem

    Multimodal systems need precise, fine-grained, reliably evaluable verification, but binary judgments and coarse critiques provide limited informative feedback.

  • Method

    OmniVerifier-M1 trains a generalist multimodal verifier using symbolic localization feedback and decoupled reinforcement-learning objectives for binary judgment and meta-verification.

  • Results

    Symbolic outputs outperform textual explanations, while decoupled objectives outperform joint training and yield consistently higher verification accuracy.

  • Takeaways & Limitations

    Symbolic meta-verification enables rule-based reinforcement learning and supports robust, fine-grained verification with region-level self-correction.

  • Takeaways & Limitations

    Model-based meta-verifier rewards remain vulnerable to reward hacking and may degrade reasoning performance despite higher rewards.

Abstract

from arXiv · show

Visual outcomes are increasingly central to multimodal large language models, making reliable and fine-grained verification essential for scaling generalist foundation models. In this work, we investigate multimodal meta-verification, which leverages verifier-generated rationales rather than decision-only signals, and explore how to effectively incorporate meta-verification feedback into multimodal verifier training. We identify two key findings. First, symbolic verifier outputs (e.g., bounding boxes) outperform textual explanations as meta-verification rationales, enabling efficient rule-based reinforcement learning rewards while avoiding reliance on model-based rewards from auxiliary judge models. Second, decoupling reinforcement learning objectives for binary judgment and meta-verification substantially outperforms joint reward optimization, due to intrinsic differences in output structure and learning dynamics. Based on these insights, we train OmniVerifier-M1, a generalist visual verifier leveraging symbolic meta-verification and decoupled reinforcement learning. OmniVerifier-M1 provides robust verification and fine-grained error localization, and further enables M1-TTS, a verifier-driven agentic generation system achieving dynamic region-level self-correction. This approach paves the way for more reliable, interpretable, and fine-grained multimodal verification, supporting safer and more controllable foundation model deployment.

1 Introduction

As visual outcomes become central to multimodal understanding and generation, reliable verification remains limited by coarse binary feedback. This work introduces multimodal meta-verification with symbolic rationales and decoupled reinforcement learning, yielding OmniVerifier-M1 and region-level self-correction through M1-TTS.

  • Motivation: Visual outcomes connect multimodal understanding and generation across agentic tool-use and native generative processes.They are increasingly important targets for multimodal systems.
  • Motivation: Existing visual verification remains early-stage, while binary decisions without detailed critiques provide coarse feedback for precise judge-model improvement.Prior image reward models largely target traditional text-to-image generation, and binary-only feedback limits granularity.
  • Meta-Verification: Multimodal meta-verification extends verification beyond binary judgments by using verifier-generated explanations to provide more informative and restrictive learning feedback.The approach examines verifier-generated rationales and explanations as supervision for verifier training.
  • Key Findings: Symbolic verifier outputs outperform textual explanations as meta-verification rationales, enabling scalable rule-based reinforcement-learning rewards without auxiliary judge models or reward hacking.Bounding boxes and points provide structured rationales suited to visual representations, whereas textual rationales slow feedback and increase reward-hacking risk.
  • Key Findings: Decoupling reinforcement-learning objectives for binary judgment and meta-verification substantially outperforms joint reward optimization.Joint rewards offer little improvement in judgment accuracy because the tasks differ in output structure, difficulty, and learning dynamics.
  • Systems: OmniVerifier-M1 combines symbolic meta-verification with decoupled training for robust verification and fine-grained error localization, while M1-TTS enables region-level self-correction across diverse generative foundation models.The verifier is adaptable to diverse multimodal foundation models and functions as a fine-grained multimodal optimizer.

2 Related Work

Prior work contrasts scalar reward models with generative verifiers that provide interpretable critiques for test-time computation and reinforcement learning. For visual generation, verifier–generator loops progressively refine outputs when one-shot generation is insufficient.

  • Generative Verifier or Reward Models: Generative verifiers provide interpretable, generative critiques, unlike traditional reward models that output only scalar rewards.This makes them promising for scaling test-time computation or reinforcement learning.
  • Iterative Refinement for Visual Generation: Visual generation methods combine a visual verifier with a generative model in a generate-reflect-refine loop.These approaches address the difficulty of achieving perfect results in a single attempt, particularly for complex compositional generation and world-knowledge reasoning.

3 Problem Formulation

Section 3 formulates RLVR training for pointwise multimodal verifiers that must both judge visual outcomes and produce transparent, fine-grained critiques. It contrasts decision-only supervision with meta-verification feedback, including rationale evaluation and later comparisons of rationale formats and reward coupling.

  • 3 Problem Formulation: The training goal is a verifier that determines prompt satisfaction while producing transparent, fine-grained, actionable critiques for reflection and refinement.
  • 3.1 Baseline RLVR Training for Multimodal Verifiers: The dataset consists of image-prompt pairs xi = (Ii, Pi) with binary ground-truth judgments yi ∈ {True, False}.
  • 3.1 Baseline RLVR Training for Multimodal Verifiers: A visual verifier πθ maps an image and prompt to textual output o, from which a binary decision ˆy is deterministically parsed using a predefined format.
  • 3.1 Baseline RLVR Training for Multimodal Verifiers: Format Reward Rf(·) enforces an explicit reasoning step by requiring intermediate analysis inside <think> and </think> tags before the final judgment.
  • 3.1 Baseline RLVR Training for Multimodal Verifiers: Decision-level accuracy supervision can guide coarse judgments but is exploitable through guessing or superficial patterns, failing to encourage fine-grained, interpretable, reliable verification.
  • 3.2 Meta-Verification Enhanced RLVR Training: Meta-verification augments the output with an explanation e required only when ˆy = False, thereby extending supervision beyond binary decisions.
  • 3.2 Meta-Verification Enhanced RLVR Training: Subsequent analysis compares rationale forms and reward coupling, finding that symbolic rationales with decoupled reinforcement learning objectives perform substantially better.
  • 3.2 Meta-Verification Enhanced RLVR Training: The meta-verification reward Rmeta = Mϕ(I, P, ˆe) ∈ R evaluates whether generated rationales identify genuine visual issues, encouraging faithful and informative explanations.

4 Symbolic Rationales for Rule-Based Multimodal Meta-Verification

The section shows that symbolic verifier outputs, especially bounding boxes, enable reliable rule-based meta-verification rewards while avoiding model-based reward-model vulnerabilities. They achieve performance comparable to textual explanations with greater training efficiency.

  • Drawbacks of Model-Based Meta-Verifiers: Model-based meta-verifiers are highly vulnerable to reward hacking in dynamic reinforcement learning settings.
  • Revisiting Rule-Based Meta-Verifiers: Images provide structured, spatially grounded representations in which verification errors can be captured through symbolic outputs such as bounding boxes.
  • Experimental Setup: The experiments train OmniVerifier-7B and Qwen3-VL-8B with DAPO using binary judgments, textual explanations, and bounding-box meta-verification targets.
  • Experimental Analysis: Symbolic bounding-box signals produce training, in-domain, and ViVerBench accuracy trends remarkably similar to textual explanations, supporting rule-based IoU rewards as a reliable proxy.
  • Experimental Analysis: Symbolic outputs reduce GPU memory usage, per-sample reward computation time, and per-step training time while maintaining comparable inference efficiency and response lengths.
  • Finding 1: Symbolic verifier outputs beat textual ones, unlocking rule-based RL rewards in meta-verification.

5 Decoupled Reinforcement Learning Incentivizing Meta-Verification

This section shows that joint reinforcement learning makes meta-verification rewards conditional on correct binary judgments, producing weak, sparse, and noisy learning signals. Decoupling the two objectives with independent reward models stabilizes meta-verification training and consistently improves performance over joint training.

  • Limitations of Joint Training: Joint training activates meta-verification rewards only when both the model prediction and ground-truth label are False, creating conditional and discontinuous gradient flow.Binary judgment and meta-verification also differ in output structure and optimization landscape.
  • Limitations of Joint Training: When pacc(θ) ≪1 early in RL training, the meta-verification gradient is approximately zero, so smaller or less capable models struggle to optimize error localization.The verifier must first learn correct binary judgments before receiving reward signals about where errors occur.
  • Decoupled Training: Decoupled training assigns binary judgment and meta-verification to separate tasks with independent reward models, using the full dataset for judgment and False-labeled samples for grounding.This design separates the objectives rather than coupling their rewards sequentially.
  • Gradient Analysis: Var(Gjoint) ≥ pacc(θ) Var(Gdec), with strict inequality when E[Gdec] ≠ 0 and pacc(θ) ∈ (0, 1).The joint meta-verification gradient is gated by binary-judgment accuracy, suppressing expected magnitude and adding variance; its signal-to-noise ratio is reduced by pacc(θ).
  • Experimental Analysis: Decoupled training consistently outperforms joint training on OmniVerifier-7B and Qwen3-VL-8B, especially on ViVerBench Bounding Box, Counting, and Pointing tasks.The reported improvement is attributed to more stable meta-verification gradients that support more precise reasoning and localization.

6 Multimodal Verifier for Agentic Generation

OmniVerifier-M1 is a generalist multimodal verifier that uses symbolic bounding boxes and decoupled reinforcement training for rule-based meta-verification. Its fine-grained feedback powers M1-TTS, an agentic system for iterative, region-level image correction.

  • OmniVerifier-M1: OmniVerifier-M1 is built on Qwen3-VL-8B and combines symbolic bounding-box rationales with rule-based meta-verification feedback through decoupled reinforcement training.This design follows the paper’s two experimental findings about symbolic rationales and decoupled reinforcement objectives.
  • OmniVerifier-M1: 0.68: OmniVerifier’s score on ViVerBench, with notable gains on Object, Attribute, Spatial Relationship, and Bounding Box text-to-image verification tasks.The approach also significantly reduces training overhead and is presented as a generalizable reinforcement-learning framework for meta-verification.
  • M1-TTS: M1-TTS uses OmniVerifier-M1 and a unified multimodal model to perform fine-grained image world modeling through verifier-guided generation and editing.The verifier localizes erroneous image regions and provides actionable feedback rather than only coarse, global text-level explanations.
  • M1-TTS: The Verifier Agent outputs spatial actions with symbolic bounding boxes and semantic actions with explicit editing instructions, enabling targeted correction of misaligned regions.These structured actions reduce the unified model’s perception and reasoning burden during image editing.
  • M1-TTS: M1-TTS performs dynamic multi-round optimization, iteratively refining images until the verifier outputs True or the maximum iteration count is reached.The system injects world knowledge into self-refinement and compensates for limitations in visual perception and spatial localization; experiments use a maximum of 10 steps on RePlan and GPT-Image-1.5.
  • M1-TTS: Bounding boxes and explanatory signals provide precise guidance for prompt-misaligned or severely erroneous regions, especially in complex images with similar-attribute objects.The reported gains stem from injecting world knowledge and using interactive bounding-box outputs to guide generative refinement.

7 Conclusion

The paper presents Multimodal Meta-Verification, extending verifier training beyond binary judgments through symbolic localization feedback. It finds that symbolic rationales and decoupled reinforcement-learning objectives improve verification training.

  • Multimodal Meta-Verification extends verifier training beyond binary judgments by leveraging symbolic localization feedback.
  • Symbolic outputs provide structured, efficient rationales that outperform textual explanations and enable rule-based reinforcement learning while mitigating reward hacking.
  • Decoupled reinforcement-learning objectives for binary judgment and meta-verification facilitate more robust and efficient optimization.

Impact Statements … A.3 Proof of Throrme 5.3

The paper introduces OmniVerifier-M1 and M1-TTS while presenting a meta-verification training framework. Its appendices formalize joint-training gradients, accuracy-dependent explanation gradients, and variance behavior, alongside an impact statement anticipating no immediate unusual societal harms.

  • Impact Statements: OmniVerifier-M1 and M1-TTS are introduced as part of a robust framework for training multimodal verifiers with meta-verification feedback.
  • Impact Statements: The authors intend the methods and findings to advance efficient, scalable machine-learning systems, without anticipating immediate negative societal impacts beyond those associated with more capable language models.
  • A.1 Proof of Lemma 5.1: The joint-training objective multiplies accuracy and meta-verification rewards, yielding a policy-gradient estimator involving Racc(ŷ, y) · Rmeta(e).The joint log-probability factorizes into prediction and conditional meta-verification terms, log πθ(ŷ | x) + log πθ(e | x, ŷ).
  • A.1 Proof of Lemma 5.1: The explanation-related joint gradient is weighted by both prediction accuracy and the meta-verification reward.The displayed term is Ex∼D,(ŷ,e)∼πθ [Racc(ŷ, y) · Rmeta(e)∇θ log πθ(e | x, ŷ)].
  • A.2 Proof of Theorme 5.2: Because Racc(ŷ, y) is the indicator 1[ŷ = y], the explanation-gradient bound retains only correct predictions and is proportional to pacc(θ).The bound is pacc(θ) · C, where C is a finite upper bound on the meta-verification gradient term.
  • A.2 Proof of Theorme 5.2: The proof concludes that the explanation-gradient term is bounded by pacc(θ) · C, with C defined as the supremum of the expected gradient norm over inputs.
  • A.3 Proof of Throrme 5.3: For Theorme 5.3, the joint gradient estimator is represented as Gjoint = I · Gdec, where I is Bernoulli with parameter pacc(θ).The proof computes the first and second moments and applies the vector variance identity Var(Z) = E[∥Z∥2] − ∥E[Z]∥2.
  • A.3 Proof of Throrme 5.3: The variance inequality is strict when pacc(θ) ∈ (0, 1) and E[Gdec] ≠ 0, because pacc(θ)(1 − pacc(θ))∥E[Gdec]∥2 is positive.Equality holds if and only if that term equals zero.

A.4 Proof of Corollary 5.4 … B.2 Evaluation of the Verifier’s Localization Accuracy

The proof establishes a strict SNR inequality for the joint gradient estimator under nonzero decoding signal and imperfect accuracy, while additional experiments validate symbolic localization rewards and precise error localization. These experiments use binary rule-based rewards and evaluate localization on held-out synthetic and real-world samples.

  • A.4 Proof of Corollary 5.4: The proof derives the first moment and variance of the joint gradient estimator from Theorem 5.3.
  • A.4 Proof of Corollary 5.4: Substituting these quantities into SNR(Gjoint) yields the corollary’s signal-to-noise expression.
  • A.4 Proof of Corollary 5.4: For pacc(θ) ∈ (0, 1) and ∥E[Gdec]∥ > 0, the denominator exceeds Var(Gdec), confirming a strict inequality.The positive term is (1 − pacc(θ))∥E[Gdec]∥2.
  • B Additional Experiments: Rule-based symbolic point rewards provide an effective alternative to model-based textual explanations for meta-verification under joint training.A point receives reward 1 when it falls inside the ground-truth box and 0 otherwise.
  • B.1 Symbolic Point as Meta-Verification Signals: The point-based reward uses the same binary formulation as bounding-box rewards, replacing IoU thresholding with inside-versus-outside box membership.Bounding-box rewards threshold IoU at 0.6 rather than using continuous IoU directly.
  • B.2 Evaluation of the Verifier’s Localization Accuracy: The localization test set contains 400 held-out False samples: 200 synthetic and 200 real-world examples.Localization success is determined by predicted-to-ground-truth box IoU using the training-consistent threshold of 0.6.
  • B.2 Evaluation of the Verifier’s Localization Accuracy: The reported significant improvement shows that decoupled symbolic rule-based RL teaches precise spatial error localization and provides reliable, fine-grained guidance to the UMM agent.

B.3 Impact of Batch Size on Decoupled Training versus Joint Training · C Data Construction Pipeline

The paper shows that decoupled training outperforms joint training at matched batch sizes because it separates supervision and reduces reward interference. Its automated data pipelines generate meaningful bounding boxes through prompt modification or image inpainting.

  • B.3 Impact of Batch Size on Decoupled Training versus Joint Training: Experiments compare larger-batch joint training with matched-batch decoupled training, and smaller-batch decoupled training with matched-batch joint training.These comparisons test whether gains arise from training strategy rather than batch size.
  • B.3 Impact of Batch Size on Decoupled Training versus Joint Training: Decoupled training consistently outperforms joint training under the same batch size, showing gains stem from decoupled optimization rather than larger batches.The comparison covers OmniVerifier and Qwen3-VL-8B on ViVerBench and RefCOCO.
  • B.3 Impact of Batch Size on Decoupled Training versus Joint Training: Joint training reuses the same 0.5B negative samples for judgment and grounding, whereas decoupled training assigns them to objective-specific supervision.This separation applies even though joint training uses batch size B and decoupled training uses total batch size 1.5B.
  • B.3 Impact of Batch Size on Decoupled Training versus Joint Training: Decoupled training reduces interference between objectives and provides denser, more stable learning signals than joint training’s sparse, entangled rewards.The improved signals lead to better performance.
  • C Data Construction Pipeline: The automated data construction pipelines associate every false sample with a meaningful, well-defined bounding box and derive all training data from OmniVerifier.This design supports fair comparison with the verifier and demonstrates the advantages of meta-verification.
  • C Data Construction Pipeline: In the image-fixed, prompt-modified pipeline, GPT-5 creates a true prompt, alters objects, attributes, or spatial relations, and generates ground-truth boxes for modified regions.The image remains fixed while the prompt is changed to construct a mismatched sample.
  • C Data Construction Pipeline: In the prompt-fixed, image-inpainting pipeline, SAM 2.1 segments objects, one object is selected by mask area, and inpainting removes it to create a false image.GPT-5 then generates a detailed fixed prompt from the true image, yielding accurate and meaningful bounding boxes.

D Limitations and Future Works

The paper identifies two remaining limitations, emphasizing the need to validate its verifier-training paradigm across larger and architecturally diverse backbones while noting unresolved drawbacks of joint training. It also flags M1-TTS as an area requiring further attention, although the supplied passage provides no further detail.

  • M1-TTS: The discussion identifies M1-TTS as a remaining limitation area, but the supplied passage does not specify the underlying issue.The passage only states that M1-TTS performance is still strong before ending.
  • Verifier Training: OmniVerifier-M1’s training paradigm requires validation on larger-scale backbone models and backbones with different architectures.The authors specifically propose evaluating larger models and architectures such as MoE.
  • Verifier Training: Larger models may slightly mitigate joint training’s disadvantages through higher early-training binary judgment accuracy, but do not resolve them.This motivates testing the approach beyond the currently evaluated backbones.
Loading 2605.28805v1…