Source-linked AI summary
MM-JudgeBias: A Benchmark for Evaluating Compositional Biases in MLLM-as-a-Judge
Sua Lee, Sanghee Park, Jinbae Im
TL;DR
Reliability and bias in multimodal judges remain underexplored, despite judgments depending on visual and textual evidence. MM-JudgeBias measures compositional biases and finds them pervasive across current MLLM judges.
Problem
Reliability and fundamental biases of MLLM judges remain underexplored, while prior bias studies largely focus on text-only scenarios despite multimodal judgments depending jointly on visual and textual cues.
Source: §1 · p. 2, §2.1 · p. 3Method
Results
Across 26 MLLMs, compositional bias is pervasive: many struggle to integrate query, image, and response holistically, and often fail to penalize missing or mismatched evidence or remain stable under irrelevant perturbations.
Source: §5 · p. 9Takeaways & Limitations
Judgment reliability is distinct from general task performance, since stronger reasoning ability or larger model scale does not necessarily yield more reliable multimodal judgments.
Source: §5 · p. 9The study focuses on multimodality-specific biases and does not exhaustively cover cultural, social, or geographic human-centric biases.
Source: Limitations · p. 10
Abstract
from arXiv · showhide
Multimodal Large Language Models (MLLMs) have been increasingly used as automatic evaluators-a paradigm known as MLLM-as-a-Judge. However, their reliability and vulnerabilities to biases remain underexplored. We find that many MLLM judges fail to reliably integrate key visual or textual cues, yielding unreliable evaluations when evidence is missing or mismatched, and exhibiting instability under semantically irrelevant perturbations. To address this, we systematically define Compositional Bias in MLLM-as-a-Judge systems and introduce MM-JudgeBias, a benchmark for evaluating it. MM-JudgeBias introduces controlled perturbations across Query, Image, and Response, and evaluates model behavior via two complementary metrics: Bias-Deviation (BD) for sensitivity and Bias-Conformity (BC) for stability. Our dataset of over 1,800 curated and refined multimodal samples, drawn from 29 source benchmarks, enables a fine-grained diagnosis of nine bias types across diverse tasks and domains. Experiments on 26 state-of-the-art MLLMs reveal systematic modality neglect and asymmetric evaluation tendencies, underscoring the need for more reliable judges.
1 Introduction
MM-JudgeBias addresses the underexplored reliability of MLLM judges by defining compositional bias as failures to integrate query, image, and response evidence. It evaluates this bias across nine types and reports that it persists in state-of-the-art models.
Motivation
- Unlike task-solving accuracy, judge reliability depends on verifying alignment among textual queries, visual evidence, and candidate responses.The paper frames this verification integrity as an underexplored concern in MLLM-based judging.Source: §1 · p. 2
Problem definition
- Compositional bias describes judges failing to integrate query, image, and response, instead relying on partial, misaligned, or spurious compositions.Such failures can lead judges to endorse erroneous or ungrounded responses, undermining evaluation reliability.Source: §1 · p. 2
Benchmark and metrics
- MM-JudgeBias defines nine bias types across integrality, congruity, and robustness, and measures them with Bias-Deviation (BD) and Bias-Conformity (BC).BD measures sensitivity to disruptions in integrality and congruity; BC measures stability under semantic-preserving variations.Source: §1 · p. 2
Benchmark data
- The benchmark contains 1,804 refined samples from 29 source benchmarks, spanning four task types and 12 domains.The samples cover diverse query tasks and difficulty levels.Source: §1 · p. 2
Empirical scope and findings
- Across 26 MLLMs, compositional bias is systemic, including in advanced reasoning-heavy models that struggle to jointly consider image inputs and text queries.The evaluation includes closed-source, open-source, and critic models.Source: §1 · p. 2
2 Related Work
Prior work documents reliability biases in LLM judges but largely focuses on text-only evaluation; multimodal judging biases remain underexplored. This work positions compositional bias as a broader concern for multimodal judgment and proposes systematic metrics to assess it.
2.1 Bias and Reliability in LLM-as-a-Judge
- LLM judges exhibit biases from superficial cues, while related analyses identify preference leakage, model-specific favoritism, and persistent cross-lingual and domain effects.Ye et al. (2025) identified 12 bias types; these findings motivate detecting and quantifying bias to maintain automated evaluation credibility.Source: §2.1 · pp. 2–3
- Most prior bias studies address text-only linguistic robustness, leaving multimodal evaluation—where visual and textual cues jointly determine correctness—underexamined.The gap motivates investigating modality dependence and neglect in multimodal judging.Source: §2.1 · p. 3
2.2 Bias and Reliability in MLLM-as-a-Judge
- Existing MLLM judge-bias work examines general biases or visual transformations in text-to-image evaluation, while this work introduces compositional biases for general multimodal judgment.It frames these biases through integrality, congruity, and robustness, and proposes two metrics to measure them systematically.Source: §2.2 · p. 3
3 MM-JudgeBias: MLLM-as-a-Judge Bias Evaluation Benchmark
MM-JudgeBias evaluates whether an MLLM judge integrates the Query, Image, and Response, responding to meaning-changing perturbations while remaining stable under irrelevant ones. It defines paired bias tests, complementary metrics, a nine-type taxonomy, and a curated dataset for this evaluation.
3.1 Task Formulation
- Each judgment scores a Query–Image–Response triplet on a 10-point scale, and paired perturbed triplets test whether the judge integrates all three components.The expected response is a lower score after semantic disruption and a stable score after semantic-preserving perturbation.Source: §3.1 · p. 4
3.2 Evaluation Metrics
- BD measures appropriate score reductions after semantic distortions, while BC measures score invariance under semantic-preserving changes; together they distinguish discerning judges from volatile ones.BD excludes reference scores of 1, where further reduction is impossible; BC near 1.0 indicates stability under irrelevant variations.Source: §3.2 · p. 5
3.3 Bias Taxonomy
- The taxonomy defines nine bias types across Integrality, Congruity, and Robustness, testing modality necessity, cross-modal alignment, and stability under irrelevant or superficial changes.Integrality and Congruity use BD to test penalties for missing modalities or mismatches; Robustness uses BC to assess stability under redundant details, unnecessary images, visual transformations, or texture insertion.Source: Table 1 · p. 4, §3.3 · p. 5
3.4 Data Construction
- The dataset contains 1,804 high-difficulty evaluation pairs sampled from 29 benchmarks spanning four cognitive tasks and 12 domains.The pipeline synthesizes and human-filters multimodally dependent queries, generates reference triplets, and applies nine corresponding perturbation strategies to create biased counterparts.Source: §3.4 · pp. 5–7
4 Empirical Results and Analysis
Across 26 MLLM judges, reasoning and critic training can improve reliability, but substantial compositional and conventional judge biases persist. Practical checks show that the benchmark’s findings are robust, while modality-focused prompting can mitigate compositional bias with model-dependent effects.
4.1 Experimental Setup
- The evaluation covers 26 closed-source, open-source, and specialized critic MLLMs, testing supported reasoning models in both standard and reasoning modes.The comparison assesses whether internal deliberation improves cross-modal judgment.Source: §4.1 · p. 7
4.2 Main Results
- Reasoning generally improves overall reliability, with Qwen3-VL-8B the exception; Prometheus-Vision and LLaVA-Critic outperform expectations for their base models, but critic training does not eliminate bias.Overall reliability generally aligns with model capability, while underlying base-model capability remains important.Source: §4.2 · p. 7
- Judges are especially weak on integrality and image-misalignment biases; response-dominance remains difficult even without image or query, and most models rely on image or text alone.Integrality is harder than congruity, suggesting judges detect semantic contradictions more readily than missing required information; open-source and critic models are more vulnerable to visual and textual noise.Source: §4.2 · p. 7
- Model scale does not consistently improve compositional integrity, and LLaVA-Critic-72B has lower overall reliability than its 7B counterpart despite robustness gains.Weak inter-run/inter-sample variance correlation indicates execution stochasticity and sample differentiation are distinct; high variance suggests judges struggle under biased conditions rather than using constant scores.Source: §4.2 · pp. 7–8
4.3 Analysis on Existing Biases
- MLLM judges remain sensitive to response order and are even more vulnerable to verbosity, while judges also tend to favor their own generated responses.These results show that conventional LLM-as-a-Judge biases remain relevant in MLLM-as-a-Judge settings.Source: §4.3 · p. 8
4.4 Analyses on Practical Validity
- Abstention and score-guided checks preserve similar trends with only marginal overall-score changes, supporting the protocol’s practical reliability beyond invalid cases and score calibration differences.The abstention ratio remains limited across the three representative judges; score-guided prompting changes overall scores only modestly.Source: §4.4 · pp. 8–9
- Modality Constraints and structured Modality Reasoning improve strong general-purpose judges, with generally larger gains from structured reasoning, though effectiveness varies by model.The strategies ask judges to verify all query, image, and response inputs, with Modality Reasoning adding rubric decomposition, modality-wise inspection, and final synthesis.Source: §4.4 · p. 9
5 Conclusion
The paper formalizes compositional bias in MLLM-as-a-Judge and introduces MM-JudgeBias to evaluate it under controlled perturbations. Experiments find pervasive bias across current judges, with multimodal judgment reliability distinct from general task performance.
5 Conclusion
- MM-JudgeBias organizes compositional bias into Integrality, Congruity, and Robustness, instantiated as nine fine-grained bias types.The benchmark evaluates multimodal judges under controlled perturbations.Source: §5 · p. 9
- Bias-Deviation (BD) measures sensitivity to semantically disruptive perturbations, while Bias-Conformity (BC) measures stability under semantically neutral variations.The paper identifies compositional bias as failure to ground evaluation in the full multimodal context, instead relying on partial, misaligned, or irrelevant evidence.Source: §5 · p. 9
- Compositional bias is pervasive across 26 MLLM judges, which often fail to integrate query, image, and response, penalize missing or mismatched evidence, or stay stable under irrelevant perturbations.The experiments include closed-source, open-source, and critic-specialized models.Source: §5 · p. 9
- Greater reasoning ability or model scale does not necessarily yield more reliable multimodal judgment, which is distinct from general task performance.Source: §5 · p. 9
Limitations
The study’s limitations are its focus on multimodality-specific biases and vision-language pointwise scoring, though its metrics are presented as extensible to broader domains, modalities, and judgment protocols.
Limitations
- The benchmark focuses on visual-textual biases such as integrality and congruity, not the broader cultural, social, and geographic spectrum of human-centric biases.The BC and BD metrics are designed to be extensible to more nuanced, domain-specific biases.Source: Limitations · p. 10
- The evaluation is tailored to vision-language interactions and pointwise scoring, while future work could extend it to other modalities and pairwise or batch-ranking protocols.Proposed extensions include video, audio, and interleaved multimodal sequences to test whether observed reliability patterns hold across sensory inputs.Source: Limitations · p. 10
A Data Statistics
MM-JudgeBias contains 1,804 samples spanning diverse tasks and domains, designed to test judgments that depend on image, query, and response together. Its difficulty levels and controlled sequence lengths support evaluation of compositional bias.
A Data Statistics
- 1,804 high-quality samples span four functional task types and 12 visual domains, curated from 29 benchmarks.Source: A Data Statistics · p. 15, Table 5 · p. 15, Table 7 · p. 16
- Nine bias-induction strategies span Easy, Moderate, and Hard levels, and require judges to integrate image, query, and response.The dataset avoids directly reusing existing benchmark queries, which the authors argue often lack the complexity needed to elicit measurable biases.Source: A Data Statistics · p. 15, Table 6 · p. 15
- Harder samples have longer queries and responses; biased queries are generally longer, while biased and unbiased response lengths remain identical.Keeping the ground-truth response constant isolates input-level perturbations from length-related confounds.Source: Figure 5 · p. 15, A Data Statistics · p. 16
B Data Taxonomy Details
MM-JudgeBias groups samples into four functional categories drawn from diverse benchmarks and visual domains. This taxonomy is intended to assess compositional bias across varied tasks, modalities, and difficulty levels.
B Data Taxonomy Details
- Perception/Understanding covers visual entity recognition and spatial or structural understanding, from simple objects to positional relations.It evaluates varying levels of visual granularity.Source: B Data Taxonomy Details · p. 17
- Information Extraction tests parsing of visual text, charts, tables, and interfaces, including format dependence and relational logic beyond OCR.Source: B Data Taxonomy Details · p. 17
- Knowledge assesses whether judges integrate internal factual knowledge with visual evidence across commonsense and domain-specific contexts, rather than over-relying on linguistic priors.Source: B Data Taxonomy Details · p. 17
- Reasoning probes multi-step logical synthesis in mathematical, symbolic, and professional-exam domains to test compositional understanding beyond shallow heuristics.Source: B Data Taxonomy Details · p. 17
- The taxonomy spans 12 visual domains and 29 benchmarks to support holistic assessment across modalities and task difficulties.Its categories are curated to preserve variety in task queries and visual contexts, reducing dependence on a narrow task setting or data distribution.Source: B Data Taxonomy Details · pp. 16–17
C Data Synthesis Details … F Biased Data Examples
The paper details a pipeline for synthesizing and reviewing multimodal and text-only queries, constructing bias perturbations, and prompting MLLM judges to evaluate samples. It also provides examples of biased and unbiased samples across the benchmark’s nine bias types.
C.1 Multimodal Query-generation Prompt
- The multimodal prompt generates three image-grounded queries per image, requiring both visual and textual evidence.It varies difficulty randomly, uses prompts tailored to 29 image sources, and applies reference-knowledge guidance in factual/commonsense tasks and randomized subtasks for diversity.Source: §C.1 · pp. 17–18
C.2 Query-selection Prompt
- Gemini-2.5-Pro ranks three candidates for multimodal dependency, after which human annotators verify the selected query against four criteria.The 12 English-proficient annotators label a query Pass only if it is image-consistent, image-dependent, domain-appropriate, and answerable; selected queries are then curated to match the intended task and domain distribution.Source: §C.2 · pp. 18–19, §C.3 · p. 19
C.4 Text-only generation
- Text-only queries provide a unimodal baseline for testing whether judges favor textual fluency even when images are absent or unnecessary.The generation prompt requires self-contained queries without visual references or external knowledge, with tiered difficulty and 11 domain types spanning task-specific prompts.Source: §C.4 · p. 19
D Bias Perturbation Details
- Bias perturbations alter image appearance, insert query-related text into images, append image descriptions to queries, or pair text-only samples with verified irrelevant images.Visual transformations combine one geometric and seven to nine aesthetic operations while preserving image semantics; irrelevant-image pairs are retained only after verification that the image is unnecessary.Source: D Bias Perturbation Details · pp. 19–20
E Judgment Prompts
- Judges use score-wise prompts for primary evaluation and pairwise prompts to analyze position and verbosity biases.Score-wise evaluation requests an integer score from 1 to 10 with detailed feedback; pairwise evaluation asks judges to choose between responses A and B, enabling checks for preference based on order or length rather than quality.Source: E Judgment Prompts · pp. 20–21
F Biased Data Examples
- Figure 6 presents pairwise examples of unbiased and biased samples across the nine bias types.The examples provide a qualitative view of the benchmark perturbations.Source: F Biased Data Examples · p. 21
G Judgement Output Examples · H Details of Existing Bias Analysis
Illustrative cases expose failures to verify modality integration, respect query-image congruity, and resist textual perturbations; the section then specifies protocols for evaluating position, verbosity, and self-enhancement bias.
G Judgement Output Examples
- Figure 7’s integrality case shows a judge rewarding sound calculation despite a missing image, without verifying that required numbers are visually grounded.The unbiased judgment explicitly assessed visual relevance, which disappeared in the biased judgment.Source: G Judgement Output Examples · pp. 21–22
- In the congruity case, the judge recognizes an unrelated elephant image but still evaluates internal consistency instead of penalizing its mismatch with the image-dependent task.The unbiased judgment grounds its decision in the financial report’s visual data.Source: G Judgement Output Examples · p. 22
- In the robustness case, added appearance details lead the judge to accept a narrow bag interpretation based on textual cues rather than the visual evidence.Its feedback shifts from referring to the pictured bag to justifying the decision from the query description.Source: G Judgement Output Examples · p. 22
H Details of Existing Bias Analysis
- Position-bias testing uses 200 samples with two high-quality responses evaluated twice after swapping their positions, measuring whether the judge maintains its preference.Consistent preferences indicate greater robustness to response order.Source: H Details of Existing Bias Analysis · p. 22
- Verbosity-bias testing adds redundant or filler sentences to each response and presents original and verbose versions in both positions across the same 200 samples.Robustness is the proportion of instances where the judge prefers the original concise response.Source: H Details of Existing Bias Analysis · p. 22
- Self-enhancement bias is assessed by auditing main-experiment logs where the judge is also a response generator and comparing scores for its own versus other models’ responses.This comparison quantifies the judge’s ego-centric preference when it is part of the candidate pool.Source: H Details of Existing Bias Analysis · p. 23
I Details of Abstention-Aware Evaluation · J Details of Score-guided Prompting · K Details of Modality-enforcing Prompting
The prompt studies test abstention, explicit score guidance, and modality enforcement as ways to assess or mitigate bias. Abstention captures some perturbations as unjudgeable but does not resolve poor reliability when models still score them, while modality-enforcing prompts improve overall reliability with a robustness trade-off.
I Details of Abstention-Aware Evaluation
- The abstention-aware prompt adds N/A for unclear, nonsensical, or impossible-to-assess inputs, and excludes N/A samples from metrics while reporting their share separately.This lets numerical scores reflect samples still considered evaluable, while the abstention ratio captures how often perturbations are treated as unjudgeable.Source: I Details of Abstention-Aware Evaluation · p. 23
- GPT-5 mini and Qwen3-VL abstain on nearly 50% of integrality samples; response-dominance abstention is very high and image-dominance abstention exceeds 50%.Reliability scores remain poor when the models do not abstain; LLaVA-Critic's low abstention ratio supports the need for dedicated measures for biased situations.Source: I Details of Abstention-Aware Evaluation · p. 23
J Details of Score-guided Prompting
- Score-guided prompting augments the original prompt with qualitative descriptions for each score from 1 to 10 to make scale interpretation more explicit and consistent.The prompt template is shown in Figure 15.Source: J Details of Score-guided Prompting · p. 23
- The score-guided setting is evaluated on the benchmark's same biased samples and compared with the original prompt across bias types, category-wise averages, and overall score.Table 16 reports the full results, but the supplied passage gives no specific outcome values or direction of change.Source: J Details of Score-guided Prompting · p. 23
K Details of Modality-enforcing Prompting
- Prompting variants are motivated by the observation that reasoning-capable judges tend to outperform base counterparts and aim to enforce use of all input modalities.Modality Constraints asks judges to verify instruction, image, and response; Modality Reasoning adds rubric decomposition, modality-wise inspection, and final synthesis.Source: K Details of Modality-enforcing… · p. 23
- Modality constraints and modality reasoning improve reliability on integrality and congruity, raising overall reliability but reducing robustness.The results indicate a trade-off across bias types and the need for a more carefully designed solution.Source: K Details of Modality-enforcing… · pp. 23–24
L Analysis of the Unnecessary-Image · M Novelty and Positioning Relative to Prior Work
The Unnecessary-Image analysis finds that irrelevant image semantics add to visual-path effects, while human checks support the reliability of the image–text filtering. The paper positions MM-JudgeBias as a multimodal benchmark focused on judge bias across diverse tasks and on whether judges integrate query, image, and response.
L Analysis of the Unnecessary-Image
- The ablation compares text-only QA, text-only QA plus an unnecessary image, and text-only QA plus a blank image to separate routing from semantic effects.[A] is the original test, [B] isolates visual-path triggering, and [C] compares blank and unnecessary images to focus more directly on semantic effects.Source: L Analysis of the Unnecessary-Image · p. 24
- Because a blank image can itself bias judgment, the authors retain [A] to assess both effects together as practical overall robustness.Although [C] may more directly reflect irrelevant semantic content, the blank-image condition is not an unbiased triplet.Source: L Analysis of the Unnecessary-Image · p. 24
- Human annotation agrees completely with the filtering model, identifying 0 related and 197 unrelated image–text pairs.The authors conclude the observed bias is unlikely to result from noisy or contaminated image–text matching.Source: L Analysis of the Unnecessary-Image · p. 24
- The 197 unrelated pairs comprise 195 domain mismatches, 1 semantic misalignment, and 1 non-contributory image.The semantic-misalignment case had potential domain overlap but unrelated specific semantics; the non-contributory case provided no useful task information.Source: L Analysis of the Unnecessary-Image · p. 24
- Across three representative judges, the original Unnecessary-Image setting [A] yields lower BC scores than blank-image setting [B], indicating semantic interference beyond visual-path activation.The [C] score is negatively correlated with the [A]–[B] gap, so stronger semantic effects tend to accompany a larger gap.Source: L Analysis of the Unnecessary-Image · p. 24, Table 13 · p. 25
M Novelty and Positioning Relative to Prior Work
- Unlike text-only judge-bias benchmarks and generation-focused modality-bias studies, MM-JudgeBias evaluates multimodal judgment and whether judges verify query–image–response congruity.The response is a third component absent from generation-oriented modality-bias studies, allowing evaluation of whether judges use all three components rather than a shortcut.Source: M Novelty and Positioning Relative to… · p. 25
- Where prior MLLM-as-a-Judge work emphasizes judgment quality, this benchmark makes judges’ innate bias tendencies a primary object of evaluation.The paper presents this focus as a prerequisite for reliable multimodal evaluation.Source: M Novelty and Positioning Relative to… · p. 25
- MM-JudgeBias addresses limited prior work on MLLM-judge bias with a multi-domain, multi-task benchmark organized around Integrality, Congruity, and Robustness.The paper contrasts this scope with work limited to inherited biases in restricted domains or multimodal robustness biases in other evaluator settings.Source: M Novelty and Positioning Relative to… · p. 25