Mitigating Perceptual Judgment Bias in Multimodal LLM-as-a-Judge via Perceptual Perturbation and Reward Modeling

• Reliable multimodal evaluation is important for decision-critical applications and scalable alternatives to costly, inconsistent human assessment, while automated judges can also provide reward signals for model improvement.Source: §1 · p. 1
• Baseline MLLM judges exhibit Perceptual Judgment Bias through insufficient visual understanding and response anchoring, favoring plausible text over direct visual evidence.Source: §1 · pp. 1–2
• Baseline judges have overall error rates above 20%; accuracy exceeds 0.8 for compound errors but drops by over 10% when only perceptual inconsistency is present.Source: §1 · p. 2
• PPJD injects controlled visual inconsistencies into otherwise correct responses, forming quadruplets that disentangle perceptual failures from reasoning errors and provide fine-grained, verifiable supervision.Source: §1 · p. 2
• The structured reward requires judges to distinguish fully correct responses from perceptual errors and explicitly grades cases combining perceptual and reasoning errors, making perceptual verification a prerequisite for high rewards.Source: §1 · p. 2
• Perception-Judge improves batch-level metrics by up to 11% on Qwen3-VL-4B-Thinking and single-score prediction accuracy by 15% on Flex-Judge-7B; it matches leading proprietary score-based evaluators and surpasses many in batch-level consistency.Source: §1 · p. 2

MLLM-as-a-Judge & Visual Reasoning

• Prior work extends LLM-as-a-Judge evaluation of generated responses to vision-language settings, but MLLM judges often rely on supervised fine-tuning and exhibit perceptual bias and grounding difficulties.Prior visual-reasoning approaches include training-free methods, RL-based optimization, and data-centric strategies.Source: §2 · p. 3

Group Relative Policy Optimization (GRPO)

• GRPO estimates relative advantages across grouped responses, avoiding PPO’s explicit value network and enabling stable, value-free optimization under sparse or noisy rewards (Shao et al., 2024).The passage frames reinforcement learning as an approach to align models with human feedback through verifiable or preference-based rewards.Source: §2 · p. 3
• GRPO samples n responses per prompt, normalizes rewards by their group mean and standard deviation, then updates the policy with a clipped PPO-style objective and KL regularization.The likelihood ratio compares current and old policies, while β controls regularization; intra-group normalization makes updates invariant to reward magnitude and robust to inter-task reward variance.Source: §2 · p. 3

3.1. Problem Definition: MLLM-as-a-Judge

• The task evaluates K multimodal responses to an instruction and visual input, producing scalar scores, pairwise preferences, or ranked candidate sequences aligned with human judgments.The evaluation is intended to reflect both perceptual grounding and reasoning consistency.Source: §3.1 · pp. 3–4

3.2. Perceptual Judgment Bias

• Baselines incur overall errors of 30.5% and 23.5%, with response-anchored failures comparable to or larger than perception failures.The analysis distinguishes judges that misperceive the image from judges that perceive it correctly but fail to apply that evidence when evaluating responses.Source: Table 1 · p. 4, §3.2 · p. 4
• Baselines distinguish correct responses from those with both perceptual and reasoning errors more reliably than from plausible responses with only perceptual errors.The proposed target order rc ≻ rrp ≻ rrp+r is intended to make judges discriminate isolated perceptual errors as well as compounded failures.Source: §3.2 · p. 4, Figure 2 · p. 4

3.3. Data Generation via Perceptual Perturbation

• PPJD turns each chosen response into a quadruplet with a visually perturbed variant that preserves reasoning and a second variant that also degrades reasoning.Starting from MMPR (Wang et al., 2024), the method perturbs visual attributes, verifies that the intended change occurred, and targets the order rc ≻ rrp ≻ rrp+r.Source: §3.3 · p. 5

3.4. Verifiable Reward Modeling

• GRPO training combines format-gated structural validity with a triplet-level ranking reward to promote globally consistent judgments (Shao et al., 2024).The ranking reward compares the predicted ordering of the correct, perception-perturbed, and fully degraded responses with the target ordering using weighted Levenshtein distance.Source: §3.4 · p. 6

4.1. Experimental Setup

• The setup uses 3k PPJD preference pairs from MMPR v1.2 and evaluates on 14 benchmark tasks using Pearson correlation, pairwise accuracy/F1/recall, and normalized Levenshtein distance.The PPJD training set excludes samples overlapping with the evaluation split; the benchmark covers general VQA, mathematical reasoning, and scientific knowledge.Source: §4.1 · pp. 6–7

4.2. Performance Evaluation

• The method improves over Qwen3-VL-4B-Thinking by 4% in pairwise evaluations, 11% in batch-level evaluations, and 12% in single-score grading tasks.The single-score gain is achieved without explicit score labels during training.Source: Table 2 · p. 7, §4.2 · p. 7
• The fine-tuned model exceeds open-source MLLMs, performs comparably to GPT-4o in single-score prediction, and outperforms most proprietary evaluators in batch-level evaluation.The authors associate its batch-level performance with stronger global consistency across multi-response comparisons.Source: §4.2 · pp. 7–8
• The method mitigates perceptual and reasoning biases and gains consistently across all three evaluation settings on InfographicVQA and MathVista.These benchmarks require precise OCR and mathematical reasoning; the authors interpret the gains as evidence of improved perception-related reasoning.Source: §4.2 · p. 8
• Although LLaVA-Critic scores higher in pairwise evaluation, the proposed model achieves competitive performance using a disjoint 3k dataset rather than LLaVA-Critic-113k.The authors present this as evidence of data efficiency with substantially less supervision.Source: §4.2 · p. 8
• Qualitative examples show the model using relevant visual cues in its reasoning and penalizing responses that omit visual content, unlike the baselines.The comparison covers Flex-Judge and Qwen3-VL-4B-Thinking.Source: Figure 4 · p. 8, §4.2 · p. 8

4.3. Ablation Study

• With pairwise reward held constant, PPJD improves consistently across all metrics over MMPR-v1.2 by encoding perceptual corrections in training pairs.The ablation compares configurations trained on 10k samples and supports more reliable evaluation than generic chosen/rejected pairs.Source: Table 3 · p. 8, §4.3 · p. 8
• On PPJD, batch-ranking reward outperforms pairwise reward on score-based and pairwise metrics without explicit score or pairwise labels.The batch objective enforces globally consistent candidate ordering and can induce accurate local preferences and calibrated score predictions.Source: Table 3 · p. 8, §4.3 · p. 9

5. Conclusion

• Experiments across diverse benchmarks show improvements in perceptual fidelity, ranking coherence, and alignment with human judgments.The conclusion presents these results as a foundation for robust, interpretable, and scalable multimodal evaluation.Source: §5 · p. 9
• Perceptual Judgment Bias occurs when multimodal judges prioritize linguistic plausibility over visual correctness, diverging systematically from human evaluation.The work identifies this as a fundamental failure mode in multimodal LLM-based judges.Source: §5 · p. 9
• The proposed framework combines PPJD with unified GRPO optimization and structured batch reward modeling for verifiable supervision and globally consistent ranking without explicit pairwise labels.The approach is perception-guided and supports fine-grained supervision.Source: §5 · p. 9

Impact Statement

• The framework aims to improve automated multimodal evaluators’ reliability and fairness by grounding judgments in visual evidence and aligning them more closely with human judgment.The paper notes this is increasingly relevant as MLLMs are used where accurate visual understanding and faithful evaluation are critical.Source: Impact Statement · p. 9
• Because residual biases may persist in subjective, ambiguous, or out-of-distribution cases, the authors recommend using MLLM judges as decision-support tools with human review and domain-appropriate auditing in high-stakes settings.They caution that risks may be amplified if automated judges replace human oversight or are treated as definitive authorities.Source: Impact Statement · p. 9

A. Limitations & Future Work

• Performance still trails expert annotators in challenging cases, and multi-round training requires more computation than DPO.The authors identify refining the reasoning procedure and broadening preference signals as future directions, while aiming to reduce computational overhead without sacrificing robustness.Source: A. Limitations & Future Work · p. 13
• Supervisory signals depend on proprietary models; future work could use self-distillation with capable open models such as Qwen (Bai et al., 2023) and extend the method to larger scales.Source: A. Limitations & Future Work · p. 13
• Subtle ambiguities in multimodal interpretation and residual reasoning noise can still affect subjective evaluations, despite reduced variability and filtering.Source: A. Limitations & Future Work · p. 13

B. Related Work

• The RL-based framework uses verifiable rewards to reinforce perception-grounded multimodal judgment, addressing judges’ tendency to favor linguistic plausibility over perceptual correctness.Existing multimodal-judge approaches mainly rely on supervised fine-tuning.Source: B. Related Work · p. 13
• Visual-reasoning approaches address under-attention to visual information through attention reweighting, visual-token rewards, or inserted visual questions (Chen et al., 2025b; Jian et al., 2025; Tian et al., 2025).Source: B. Related Work · p. 13
• Unlike prior work that improves perceptual grounding during reasoning, this work focuses on perception itself in multimodal judgment.Source: B. Related Work · p. 13

C. Experimental Details

• GRPO fine-tuning and multimodal-judgment benchmark evaluations used a standardized experimental setup unless otherwise specified.Source: C. Experimental Details · pp. 13–14

C.1. Code & Reproduction

• The PyTorch training and evaluation pipeline uses the verl framework (Sheng et al., 2024), and its code is available on GitHub.Source: §C.1 · p. 14

C.2. Implementation Details & Hyperparameters

• Perception-Judge starts from Flex-Judge-VL and Qwen3-VL and is trained with GRPO for one epoch on PPJD, using five rollouts per prompt and a global batch size of 512.Learning rates are 2.5 × 10−7 for Flex-Judge-VL and 7.5 × 10−7 for Qwen3-VL; training uses 8 H100 GPUs and 8 H200 GPUs, respectively.Source: §C.2 · p. 14

C.3.1. EVALUATION SETTINGS

• Evaluation compares judges with human preferences through single-score grading, pairwise comparison, and batch ranking, measured by Pearson agreement, accuracy, and normalized Levenshtein distance, respectively.Pairwise response order is randomized, and batch rankings compare predicted and human permutations.Source: §C.3.1 · p. 14

C.3.2. EVALUATION PROMPTS

• The evaluation prompts convert single-score outputs from a 1-to-10 scale to 1-to-5, derive pairwise preferences or ties from scores, and turn batch scores into rankings.The three prompt variants are shown in Figures S2–S4.Source: §C.3.2 · p. 15

C.3.3. BIAS MITIGATION

• Order randomization reduces positional bias in pairwise and batch settings, while tie handling avoids forced choices when available or requires a random choice otherwise.These strategies follow prior evaluations (Chen et al., 2024; Ko et al., 2025).Source: §C.3.3 · p. 15

C.4. Baselines

• Baselines include LLaVA-Critic (Xiong et al., 2025), Qwen2.5-VL (Bai et al., 2023), Flex-Judge-VL (Ko et al., 2025), Prometheus-V (Lee et al., 2024), and Qwen3-VL (Yang et al., 2025).They range from judge-tuned or rubric-based evaluators to general-purpose multimodal models used for zero-shot or prompt-based assessment; LLaVA-Critic may overlap with evaluation benchmarks.Source: §C.4 · pp. 15–16

D.1. PPJD Statistics

• PPJD contains 3K instances spanning visual question answering, multi-step reasoning, math and geometry, document extraction, and image-quality assessment.The section points to Table S1 for category-level statistics.Source: §D.1 · p. 16, Table S1 · p. 16

D.2. Detailed Dataset Generation Pipeline

• PPJD examples pair an image-instruction input with a validated correct answer and two rejected candidates, after filtering MMPR preferred answers against the original ground-truth labels.The validated answer is retained only when it matches the ground truth, so preference alone does not establish correctness.Source: §D.2 · p. 16
• The pipeline extracts at most six visually verifiable attributes relevant to the instruction but absent from it, excluding semantic or speculative attributes.Attributes include counts, shapes, colors, textures, spatial relations, and salient elements such as chart text, axes, or legends.Source: §D.2 · p. 16
• PPJD generates one negative by minimally changing perceptual attributes and another by coupling altered visual cues with a reasoning error, retaining only candidates that diverge from ground truth.The joint error can involve an incorrect comparison, causal interpretation, chart trend, or high-level conclusion; both candidates are intended to remain coherent and visually grounded.Source: §D.2 · p. 17

E. Reward Analysis of Perception-Judge

• Both judge models rapidly approach near-perfect format reward, while their batch-ranking reward improves steadily before refinement and validation closely tracks training.Qwen3 achieves a consistently higher batch-ranking reward trajectory than Flex; the passage reports a negligible train-validation gap across both metrics.Source: Figure S8 · p. 18, E. Reward Analysis of Perception-Judge · p. 18

F.1. Generalization to a larger model scale

• The proposed approach yields consistent performance gains for the larger Flex-Judge-VL-32B and Qwen3-VL-8B-Thinking backbones.The scale-up experiments use an additional rank-32 LoRA adapter with existing Flex Judge LoRA modules fixed for the 32B model, and full fine-tuning for Qwen3-VL-8B-Thinking.Source: §F.1 · p. 18

F.2. Score Evaluation on Spearman

• Perception-Judge achieves substantially higher Spearman rank correlation with human judgments than baseline multimodal judges.The passage links the improvement to suppressing visually inconsistent responses and maintaining coherent scoring across perceptual tasks.Source: §F.2 · p. 18

F.3. Pair Evaluation on F1 and Recall

• Perception-Judge surpasses Qwen2.5-VL-7B and Flex-Judge-VL-7B in both recall and F1 on pairwise identification of human-preferred responses.The passage attributes this to improved sensitivity to visually grounded correctness and reduced susceptibility to fluent but perceptually incorrect outputs.Source: §F.3 · p. 18

F.4. Batch Evaluation on Edit Distance

• Perception-Judge has lower edit distance and higher Kendall’s tau and NDCG, indicating more stable rankings that better reflect perceptual fidelity.The passage says comprehensive metric summaries appear in Table S3, but supplies no numerical values.Source: §F.4 · pp. 18, 20, Table S3 · p. 20
• The 32B-scale evaluation covers all datasets and score-based, pairwise, and batch-level metrics.Each judgment is sampled three times and averaged to reduce variance; models marked ‡ use LoRA training (Hu et al., 2022).Source: Table S2 · p. 19
• The complementary 7B-scale evaluation includes Spearman correlation, F1 score, Recall, Kendall’s tau, and NDCG.Table S3 is identified as the source of the comprehensive metric summaries.Source: §F.4 · pp. 18, 20, Table S3 · p. 20

G.1. MLLM-as-a-Judge Qualitative Results

• The examples identify inaccurate visual interpretation and over-reliance on response-anchored cues as recurring errors in Qwen2.5-VL-7B (Bai et al., 2023), Flex-Judge-VL-7B (Ko et al., 2025), and Qwen3-VL-4B-Thinking (Yang et al., 2025).The examples are shown in Figures S9, S12, S13, and S14.Source: §G.1 · p. 20
• The examples provide direct references for Perception-Judge’s improvement in evaluating responses against visual input.They demonstrate correction of systematic perceptual failures observed in baseline multimodal judges.Source: §G.1 · p. 20
• Baseline judges sometimes assign high scores to logically fluent responses that are perceptually incorrect.Perception-Judge instead identifies discrepancies between the visual input and candidate responses.Source: §G.1 · p. 20
• Perception-Judge penalizes descriptions misaligned with the image and favors perceptually grounded reasoning.These behaviors are illustrated by the qualitative cases.Source: §G.1 · p. 20
• The qualitative cases include miscounted objects, incorrect color or spatial attributes, and hallucinated visual elements.These are examples of scenarios in which baseline judgments can diverge from visual evidence.Source: §G.1 · p. 20
• Perception-Judge more reliably aligns evaluations with visual evidence and mitigates biases found in existing MLLM-as-a-Judge systems.This is the collective conclusion drawn from the qualitative examples.Source: §G.1 · p. 20

H.1. Detailed Results of Ablation Study

• PPJD consistently improves over MMPR across diverse benchmarks, while batch reward provides more stable, globally aligned supervision than pair reward.The reported gains are especially pronounced on datasets requiring precise perceptual grounding; batch reward improves alignment across score-based, pairwise, and ranking-oriented metrics.Source: §H.1 · p. 20, Table S4 · p. 21
• Pair reward trains local perceptual preferences by comparing responses across two degradation levels, with each comparison pair sampled equally often.The target preferences are rc ≻ rrp and rrp ≻ rrp+r; the stochastic formulation promotes consistent preference learning across perturbation levels.Source: §H.1 · p. 21

H.2. Ablation studies on KL penalty

• The KL coefficient yields peak Flex-Judge-7B performance at 1e-2, balancing exploration and stability.The KL penalty regularizes GRPO by limiting deviation from the reference policy.Source: §H.2 · p. 21, Table S5 · p. 21

I. Failure Cases

• Perception Judge can fail on subtle or atypical visual distinctions, especially with implicit 3D structure or complex spatial relationships.The paper attributes these cases to broader perception limits of current multimodal backbones rather than to its training objective.Source: §I · p. 21, Figure S15 · p. 28
• Complex multi-step reasoning can still produce incorrect judgments, indicating a need for stronger base-model reasoning when visual perception alone is insufficient.Source: §I · p. 21, Figure S15 · p. 28