Source-linked AI summary
MM-JudgeBias: A Benchmark for Evaluating Compositional Biases in MLLM-as-a-Judge
Sua Lee, Sanghee Park, Jinbae Im
TL;DR
MLLM judges are increasingly important, yet their reliability and biases remain insufficiently understood, particularly when multimodal evidence is incomplete or misaligned. This paper defines Compositional Bias and introduces MM-JudgeBias, which uses controlled perturbations and two complementary metrics to evaluate it. Across 26 MLLMs, compositional bias is pervasive, with models often failing to integrate query, image, and response evidence or remain stable under irrelevant perturbations.
Problem
The reliability and fundamental biases of MLLM-based judges remain underexplored, especially for multimodal evaluation requiring integration of visual, textual, and response information.
Method
MM-JudgeBias defines nine compositional bias types across Integrality, Congruity, and Robustness and measures them with controlled perturbations, Bias-Deviation (BD), and Bias-Conformity (BC).
Results
Experiments on 26 state-of-the-art MLLMs show pervasive compositional bias, including failures to integrate query, image, and response evidence and to remain stable under irrelevant perturbations.
Takeaways & Limitations
Judgment reliability is distinct from general task performance, since stronger reasoning ability or larger model scale does not necessarily produce more reliable multimodal judgment.
Takeaways & Limitations
The study focuses on multimodality-specific biases and primarily evaluates vision-language interactions in a pointwise scoring paradigm.
Abstract
from arXiv · showhide
Multimodal Large Language Models (MLLMs) have been increasingly used as automatic evaluators-a paradigm known as MLLM-as-a-Judge. However, their reliability and vulnerabilities to biases remain underexplored. We find that many MLLM judges fail to reliably integrate key visual or textual cues, yielding unreliable evaluations when evidence is missing or mismatched, and exhibiting instability under semantically irrelevant perturbations. To address this, we systematically define Compositional Bias in MLLM-as-a-Judge systems and introduce MM-JudgeBias, a benchmark for evaluating it. MM-JudgeBias introduces controlled perturbations across Query, Image, and Response, and evaluates model behavior via two complementary metrics: Bias-Deviation (BD) for sensitivity and Bias-Conformity (BC) for stability. Our dataset of over 1,800 curated and refined multimodal samples, drawn from 29 source benchmarks, enables a fine-grained diagnosis of nine bias types across diverse tasks and domains. Experiments on 26 state-of-the-art MLLMs reveal systematic modality neglect and asymmetric evaluation tendencies, underscoring the need for more reliable judges.
1 Introduction
MLLMs are increasingly used as judges, but their reliability remains underexplored because they may fail to integrate visual, textual, and response evidence. MM-JudgeBias formalizes this Compositional Bias and evaluates it with controlled perturbations, complementary metrics, and broad model coverage.
- MLLMs increasingly serve as automatic judges for multimodal generations, extending beyond their role as task solvers.
- The reliability and fundamental biases of MLLM-based judges remain underexplored despite progress in multimodal judging benchmarks.
- Compositional Bias describes judges failing to integrate query, image, and response evidence, instead relying on partial, misaligned, or spurious compositions.
- MM-JudgeBias defines nine bias types across Integrality, Congruity, and Robustness to measure multimodal judgment reliability.
- Bias-Deviation (BD) measures sensitivity to disruptions in Integrality and Congruity, while Bias-Conformity (BC) measures stability in Robustness.
- 1,804 samples from 29 benchmarks support evaluation of 26 MLLMs, whose results show difficulty jointly considering image inputs and text queries.
- The study identifies Compositional Bias as a systemic issue pervasive even in advanced reasoning-heavy models.
2 Related Work
Prior judge-bias research largely focuses on text-only evaluation, leaving multimodal interactions insufficiently studied. MM-JudgeBias addresses this gap by examining compositional biases across general multimodal judging settings.
- Existing LLM-as-a-Judge studies identify biases from superficial cues, prompt structure, self-similarity, favoritism, language, and domain conditions.
- Most prior bias analyses remain text-only, overlooking cases where visual and textual cues jointly determine correctness.
- Recent MLLM judging studies assess multimodal capability, but biases specific to MLLM-as-a-Judge remain underexplored.
- Existing multimodal studies address selected biases, including egocentric, position, length, or visual-transformation biases, rather than general compositional bias.
- MM-JudgeBias introduces integrality, congruity, and robustness perspectives with two metrics for systematic multimodal bias measurement.
3 MM-JudgeBias: MLLM-as-a-Judge Bias Evaluation Benchmark
MM-JudgeBias evaluates whether MLLM judges holistically integrate query, image, and response, distinguishing sensitivity to semantic disruptions from stability under irrelevant changes. It defines a nine-type taxonomy, paired perturbation-based evaluation, and complementary reliability metrics for multimodal judgment.
- 3.1 Task Formulation: MM-JudgeBias treats a judgment instance as a triplet (Q, I, R), where an MLLM judge maps the query, image, and response to a scalar quality score.Scores use a 10-point convention, with fθ: (Q, I, R) → y.
- 3.1 Task Formulation: The benchmark pairs each unbiased triplet with a controlled perturbed triplet to test whether judgments reflect holistic rather than partial consideration of the three components.The paired scores are y = fθ(Q, I, R) and ŷ = fθ(Q′, I′, R′).
- 3.2 Evaluation Metrics: Bias-Deviation (BD) measures whether semantic distortions lower scores appropriately, while Bias-Conformity (BC) measures score stability under perturbations that should not change response quality.BD is used for Integrality and Congruity; BC is used for Robustness, where an ideal score is near 1.0.
- 3.2 Evaluation Metrics: The benchmark aggregates BD across disruptive categories and BC across preserving categories to distinguish discerning judges from models that deviate indiscriminately.Its construction yields 1,804 high-difficulty evaluation pairs sampled from 29 representative benchmarks.
- 3.3 Bias Taxonomy: MM-JudgeBias organizes nine bias types into Integrality, Congruity, and Robustness, covering modality necessity, cross-modal contradictions, and stability under semantic-preserving changes.Examples include modality exclusion, query/image-response mismatches, redundant descriptions, irrelevant images, and visual or texture changes.
4 Empirical Results and Analysis
The evaluation finds that MLLM judges remain vulnerable to compositional and conventional biases, with reliability varying across models and bias types. Practical checks show that abstention and score calibration have limited effects, while modality-enforcing prompts help some models.
- 26 MLLMs are evaluated across closed-source, open-source, and critic categories to assess susceptibility to compositional biases.
- Overall reliability generally aligns with model capability, while internal reasoning usually improves reliability and critic fine-tuning does not largely resolve bias vulnerabilities.
- Models are especially susceptible to the three integrality biases and image-misalignment, while many rely on image or text alone rather than integrating both modalities.
- Integrality is harder than congruity for most models, whereas robustness is generally high but weaker for open-source and critic models under visual or textual noise.
- Model scale does not consistently improve compositional integrity; LLaVA-Critic-72B has lower overall reliability than its 7B counterpart because robustness gains are offset by other bias weaknesses.
- Position sensitivity persists, verbosity bias is stronger, and self-enhancement bias leads judges to favor their own outputs.
- Abstention changes overall scores only marginally, while score-guided prompting also produces modest changes, supporting the protocol’s practical reliability.
- Modality constraints and modality reasoning improve strong general-purpose judges, but intervention effectiveness differs across models.
5 Conclusion
The paper formalizes Compositional Bias in multimodal judging and introduces MM-JudgeBias to measure it systematically. Experiments show that this bias is pervasive and that general reasoning ability or model scale does not necessarily ensure reliable judgment.
- MM-JudgeBias organizes compositional bias into Integrality, Congruity, and Robustness, covering nine bias types and using BD and BC metrics.
- Experiments on 26 state-of-the-art MLLMs find pervasive compositional bias, including failures to integrate query, image, and response holistically.
- Stronger reasoning ability or larger model scale does not necessarily produce more reliable multimodal judgment, indicating that judgment reliability is distinct from general task performance.
Limitations
The study focuses on multimodality-specific biases and evaluates vision-language interactions using pointwise scoring, leaving broader bias dimensions and judgment protocols for future work.
- Scope: The benchmark intentionally narrows its scope to multimodality-specific distortions arising from visual-textual interplay, rather than broader human-centric biases.The authors describe BC and BD as extensible to more nuanced, domain-specific biases.
- Evaluation setting: The current evaluation is tailored to vision-language interactions within a pointwise scoring paradigm.The authors propose extending evaluation to video, audio, interleaved multimodal sequences, pairwise comparisons, and batch-ranking scenarios.
A Data Statistics
MM-JudgeBias contains 1,804 samples organized across tasks, domains, bias strategies, and difficulty levels, with controlled construction intended to isolate input-level perturbation effects.
- Bias and difficulty: The dataset covers nine bias-induction strategies and three cognitive difficulty levels: Easy, Moderate, and Hard.Table 6 describes the distribution by bias type and difficulty.
- Dataset composition: 1,804 samples are distributed across four functional task types and 12 visual domains curated from 29 source benchmarks.This hierarchical composition is intended to provide broad task and visual-context coverage.
- Construction rationale: The benchmark avoids directly reusing existing queries because trivial tasks may not elicit measurable biases from advanced or thinking models.The construction emphasizes diversity and cognitive challenge for bias detection.
- Length statistics: Hard samples use longer queries and responses, reflecting greater logical density as cognitive difficulty increases.The reported length trend links increasing difficulty with longer sequences.
- Length statistics: Biased queries are generally longer than unbiased queries because of linguistic triggers and contextual perturbations.The ground-truth response is held constant to isolate input-level perturbations and reduce length-related confounding.
B Data Taxonomy Details
The benchmark organizes diverse samples by task function, visual domain, and difficulty to test compositional bias across varied multimodal contexts and reasoning demands.
- Taxonomy structure: MM-JudgeBias uses four functional task types and 12 visual domains curated from 29 benchmarks to diversify task queries and visual contexts.The organization is intended to reduce dependence on a narrow task or data distribution.
- Difficulty levels: Multimodal queries are generated at Easy, Moderate, and Hard levels by varying reasoning steps and linguistic complexity.A level is randomly selected per generation call to balance reasoning depth.
- Difficulty levels: Text-only queries use the same three-level difficulty structure to scale from simple comprehension to multi-step logical deduction.The text-only taxonomy maps 11 domain types to task-specific prompts.
- Functional task types: Perception/Understanding evaluates recognition of visual entities and spatial or structural relationships across visual granularities.It ranges from simple object recognition to complex positional logic.
- Functional task types: Information Extraction tests parsing of structured visual data, including text, charts, tables, and digital interfaces.The category examines reliability beyond simple OCR, including relational logic and data format effects.
- Functional task types: Knowledge measures integration of internal factual knowledge with external visual cues across commonsense and domain-specific settings.It examines whether judges over-rely on linguistic priors over visual evidence.
- Functional task types: Reasoning tests multi-step logical synthesis in mathematical, symbolic, and professional-exam domains.The tasks are designed to discourage shallow heuristics and pattern matching.
C Data Synthesis Details
The dataset synthesis pipeline generates image-dependent queries, filters and verifies them for quality, and creates controlled perturbations and judgment prompts for bias evaluation.
- Query generation: Generated queries are grounded in source images from 29 benchmarks and use domain-specific contextual prompts for accuracy.The contextual prompts are tailored to the image sources.
- Query generation: A prompt template generates exactly three distinct queries per image while enforcing that solving requires both image and text.The template combines difficulty, domain, knowledge, and functional-subtask specifications.
- Query generation: Difficulty is randomly sampled among Easy, Moderate, and Hard to balance reasoning depth across generated queries.The levels are defined by reasoning steps and linguistic complexity.
- Query generation: For factual or commonsense queries, a knowledge guideline references information in original source queries to maintain consistency with the image.This constraint is intended to prevent factual hallucinations during synthesis.
- Selection and verification: A secondary model ranks three candidates and selects the Best Query that maximizes multimodal dependency before human verification.Gemini-2.5-Pro performs batch-wise ranking, while annotators validate alignment and quality.
- Selection and verification: Human review requires that selected queries agree with the image, require image reference, match the domain, and remain answerable.Annotators assign Pass only when all four criteria are satisfied.
- Text-only baseline: A text-only subset uses self-contained queries without visual references and provides a baseline for detecting preferences for textual fluency.Its generation uses tiered difficulty and broad task-specific domain coverage.
- Bias induction: Perturbations modify visuals, inject text into images, append image descriptions, or pair text-only queries with irrelevant images.These operations target visual, textual, descriptive, and unnecessary-image biases.
G Judgement Output Examples
The examples expose compositional failures across integrality, congruity, and robustness: judges may accept responses without required visual grounding, overlook image mismatch, or follow irrelevant textual cues. The section also outlines protocols for testing inherited position, verbosity, and self-enhancement biases.
- Figure 7 presents representative compositional-bias failures, with one example for each high-level bias category.
- Integrality: A missing image can leave a judge favorably evaluating mathematical reasoning without verifying that referenced numbers are grounded in the visual input.
- Congruity: Replacing a financial report with an unrelated elephant image can still lead the judge to assess internal response consistency despite image-dependent evidence being unavailable.
- Robustness: An added appearance description can cause the judge to accept a narrow bag interpretation by relying on textual cues rather than visual evidence.
- Inherited judge biases: Position bias is tested by reversing two high-quality responses across 200 samples, while verbosity bias adds redundant sentences and self-enhancement compares self-generated with other-generated responses.
I Details of Abstention-Aware Evaluation
The evaluation adds explicit abstention and score guidance, then tests modality-enforcing prompts on the same biased samples. Abstention can identify some unjudgeable cases, while modality-focused prompting improves some categories but trades off robustness.
- Abstention-aware evaluation: The abstention-aware protocol permits N/A when a response cannot be properly evaluated and excludes those samples from numerical metric computation.
- Abstention-aware evaluation: The protocol reports abstention ratios separately so scores describe only samples judges still consider evaluable.
- Abstention-aware evaluation: GPT-5 mini and Qwen3-VL abstain on nearly 50% of integrality cases, but reliability remains poor when they do not abstain.
- Prompt interventions: Score-guided prompting adds explicit qualitative descriptions for each score from 1 to 10 before comparing results with the original prompt.
- Prompt interventions: Modality constraints and modality reasoning improve reliability on integrality and congruity but reduce robustness, revealing a cross-category trade-off.
L Analysis of the Unnecessary-Image
The unnecessary-image ablation separates visual-path activation from irrelevant-image semantics and supports both as contributors to the observed bias. Human verification further supports the filtering of unrelated image–text pairs.
- Ablation design: The ablation compares text-only QA, text plus an unnecessary image, and text plus a blank image to disentangle semantic interference from visual-path activation.
- Ablation results: Across three representative judges, the original unnecessary-image condition has lower BC than the blank-image reference, indicating greater vulnerability than mere image presence.
- Ablation results: The comparison between unnecessary and blank images indicates that irrelevant visual content contributes a semantic component beyond modality routing.
- Ablation interpretation: The study retains the original condition for practical robustness assessment because blank images can themselves bias judgment and are not unbiased triplets.
- Human verification: Human annotation agrees completely with model-based filtering, identifying 0 related and 197 unrelated image–text pairs.
- Human verification: The unrelated pairs are categorized by domain mismatch, semantic misalignment, and non-contributory image content.
M Novelty and Positioning Relative to Prior Work
The paper positions MM-JudgeBias as a multimodal, judge-focused bias benchmark rather than a text-only, task-quality, or modality-bias study. Its examples and supporting tables illustrate the broader scope of this positioning.
- Prior work: Prior judge-bias research mainly studies text-only position, verbosity, and self-enhancement biases, leaving multimodal-specific reliability issues unaddressed.
- Prior work: MLLM modality-bias studies examine generation-time reliance on one modality, whereas this work evaluates judgment of congruity among query, image, and response.
- Positioning: Existing multimodal judge evaluations prioritize judgment quality or human agreement, while this work treats innate bias tendencies as a prerequisite for reliable evaluation.
- Positioning: MM-JudgeBias covers multiple domains and tasks through the complementary categories Integrality, Congruity, and Robustness.
- Evaluation scope: The abstention and prompt-intervention tables extend the comparison beyond benchmark construction to alternative evaluation protocols and prompting strategies.
- Illustrative cases: The benchmark’s examples include text-dominance, image-misalignment, and detail-description cases that show how judges can neglect or overuse modalities.