Source-linked AI summary
TrustJudge: Inconsistencies of LLM-as-a-Judge and How to Alleviate Them
Yidong Wang, Yunze Song, Tingyuan Zhu, Xuanwang Zhang, Zhuohao Yu, Hao Chen, Chiyu Song, Qiufeng Wang, Cunxiang Wang, Zhen Wu, Xinyu Dai, Yue Zhang, Wei Ye, Shikun Zhang
TL;DR
LLM-as-a-judge systems suffer from score-comparison conflicts and non-transitive pairwise judgments caused by information loss and ambiguous ties. TrustJudge uses distribution-sensitive scoring and likelihood-aware aggregation to address these problems. It reduces both inconsistency types while maintaining or improving evaluation accuracy across models and scales.
Problem
LLM-as-a-judge frameworks exhibit score-comparison and pairwise-transitivity inconsistencies, associated with information loss in discrete scores and ambiguous tie judgments.
Method
TrustJudge preserves judgment entropy through distribution-sensitive probabilistic scoring and resolves pairwise inconsistencies through likelihood-aware aggregation.
Results
TrustJudge reduces Score-Comparison inconsistency by 8.43% and Pairwise Transitivity inconsistency by 10.82% with Llama-3.1-70B-Instruct while maintaining or improving evaluation accuracy.
Takeaways & Limitations
TrustJudge provides theoretical and practical mechanisms for more reliable and credible automated LLM evaluation across architectures and scales.
Takeaways & Limitations
TrustJudge’s evaluation quality depends on the instruction-following competence of the underlying judge model, with smaller models more likely to produce invalid scores or comparisons.
Abstract
from arXiv · showhide
The adoption of Large Language Models (LLMs) as automated evaluators (LLM-as-a-judge) has revealed critical inconsistencies in current evaluation frameworks. We identify two fundamental types of inconsistencies: (1) Score-Comparison Inconsistency, where lower-rated responses outperform higher-scored ones in pairwise comparisons, and (2) Pairwise Transitivity Inconsistency, manifested through circular preference chains (A>B>C>A) and equivalence contradictions (A=B=C\neq A). We argue that these issues come from information loss in discrete rating systems and ambiguous tie judgments during pairwise evaluation. We propose TrustJudge, a probabilistic framework that addresses these limitations through two key innovations: 1) distribution-sensitive scoring that computes continuous expectations from discrete rating probabilities, preserving information entropy for more precise scoring, and 2) likelihood-aware aggregation that resolves transitivity violations using bidirectional preference probabilities or perplexity. We also formalize the theoretical limitations of current LLM-as-a-judge frameworks and demonstrate how TrustJudge's components overcome them. When evaluated with Llama-3.1-70B-Instruct as judge using our dataset, TrustJudge reduces Score-Comparison inconsistency by 8.43% (from 23.32% to 14.89%) and Pairwise Transitivity inconsistency by 10.82% (from 15.22% to 4.40%), while maintaining higher evaluation accuracy. Our work provides the first systematic analysis of evaluation framework inconsistencies in LLM-as-a-judge paradigms, offering both theoretical insights and practical solutions for reliable automated assessment. The framework demonstrates consistent improvements across various model architectures and scales, enabling more trustworthy LLM evaluation without requiring additional training or human annotations. The codes can be found at https://github.com/TrustJudge/TrustJudge.
1 INTRODUCTION
LLM-as-a-judge frameworks use single-score and pairwise evaluations but exhibit score-comparison and pairwise-transitivity inconsistencies. TrustJudge addresses these problems by preserving judgment entropy and resolving ambiguous comparisons, reducing both inconsistency types while maintaining evaluation accuracy.
- 1 INTRODUCTION: LLM-as-a-judge frameworks combine integer single-score assessments with pairwise preference comparisons.Pairwise evaluations swap response order to reduce position bias.
- 1 INTRODUCTION: The identified inconsistencies include lower-scored responses winning pairwise comparisons and cyclic or contradictory pairwise preferences.These patterns violate consistency between evaluation protocols and rational preference principles.
- 1 INTRODUCTION: TrustJudge preserves judgment entropy through probabilistic scoring and resolves transitivity violations through likelihood-aware aggregation.Its proposed mechanisms include finer scoring granularity, probabilistic scoring, perplexity, and token-level confidence.
- 1 INTRODUCTION: TrustJudge reduces Score-Comparison inconsistency by 8.43% and Pairwise Transitivity inconsistency by 10.82% with Llama-3.1-70B-Instruct as judge.The reported rates decrease from 23.32% to 14.89% and from 15.22% to 4.40%, respectively.
- 1 INTRODUCTION: The framework is presented as a practical solution for more reliable automated assessment across diverse models and tasks.The conclusion links reduced inconsistencies with maintained evaluation accuracy and more trustworthy deployment.
2 METHODOLOGY
TrustJudge formalizes two inconsistency types and applies separate probabilistic mechanisms to single-score and pairwise evaluation. It preserves score-distribution information through expected continuous scores and aggregates bidirectional or perplexity-based evidence to resolve ambiguous comparisons.
- 2.1 INCONSISTENCY DEFINITIONS AND METRICS: Score-Comparison inconsistency is defined by disagreement between score ordering and pairwise outcome, including non-ties between equal scores.Pairwise transitivity inconsistency includes circular preferences and violations of indifference transitivity.
- 2.2 TRUSTJUDGE: TrustJudge is a probabilistic framework that preserves judgment entropy while addressing score-comparison conflicts and transitivity violations.Its single-score and pairwise procedures maintain consistency between the two evaluation protocols.
- 2.2 TRUSTJUDGE: Distribution-sensitive scoring expands the score scale, normalizes score probabilities, computes an expected score, and rescales it to the original range.The approach produces continuous scores while preserving fine-grained distinctions between response qualities.
- 2.2 TRUSTJUDGE: The PPL-based pairwise option compares perplexities for both response orderings and selects the ordering with lower perplexity.This method breaks ambiguous ties using the judge model’s likelihood over each ordering.
- 2.2 TRUSTJUDGE: The bidirectional probability option sums preference probabilities from both orderings and selects the outcome with maximum aggregate probability.Aggregating both directions reduces position bias while retaining scalable pairwise comparisons.
- 2.2 TRUSTJUDGE: A tolerance δ can declare a tie when score, perplexity, or probability discrepancies do not exceed the chosen threshold.This parameter lets users tune ranking granularity without retraining the model.
3 THEORETICAL ANALYSIS
The theoretical analysis shows that discrete scoring can collapse distinct judgment distributions into the same score, whereas distribution-sensitive scoring preserves their distinctions. It also shows that perplexity-based confidence produces a lower-entropy signal for ambiguous pairwise decisions.
- 3 THEORETICAL ANALYSIS: Discrete scoring can assign identical scores to distinct probability distributions with different conditional entropies.The arg max mapping is non-injective and discards distribution shape and uncertainty.
- 3 THEORETICAL ANALYSIS: Distribution-sensitive scores remain distinct for those same probability distributions, demonstrating information preservation.The expected-score construction distinguishes belief states that discrete scoring merges.
- 3 THEORETICAL ANALYSIS: PPL-based confidence reduces ambiguity because non-uniform rationale perplexities yield entropy below the maximum outcome entropy.The confidence distribution is proportional to exp(−γ · PPL(J_k)) for γ > 0.
4 EXPERIMENTS
Experiments evaluate TrustJudge across diverse datasets, judge models, scales, architectures, and applications. TrustJudge consistently reduces evaluation inconsistencies while maintaining or improving accuracy, with benefits from fine-grained scoring and likelihood-aware aggregation.
- Main Results: 10.82%-36.93% absolute reductions in NTR_k=5 violations were achieved across settings, including a decrease from 54.69% to 17.76% for Llama-3.2-3B, without additional training or fine-tuning.TrustJudge also delivers 4.78%-8.43% absolute Conflict Ratio improvements over baselines and surpasses G-Eval by approximately 1%-2%.
- Accuracy: 1.19%-6.85% exact-match improvements over baseline were observed across model sizes, with the largest gain occurring for Llama-3.2-3B.Pairwise win rates against baseline and G-Eval approaches range from 45.41%-65.11%.
- Ablation Study: 5-point baseline scoring has the highest single-score inconsistency, while softmax normalization reduces inconsistency by 0.32%-6.24% and 100-point granularity reduces it by up to 5.19%.These findings support the contribution of distribution-sensitive scoring to preserving scoring fidelity.
- Pairwise Comparison: The likelihood-aware aggregation strategy reduces pairwise inconsistency to 1.94% for Llama-3.1-70B and 2.83% for GPT-4o, while PPL-based comparison improves the Llama-3.1-8B baseline by 16.47% absolute.PPL-based comparison operates directly on sequence probabilities without explicit win/tie/lose position identification.
- Scoring Granularity: Conflict Ratios consistently decline as scoring granularity increases from 5 to 100 points, and TrustJudge remains below baseline at every tested granularity.The benefit is especially pronounced for Llama-3.1-70B and GPT-4o.
- Architectures and Sizes: TrustJudge reduces inconsistency across model architectures and narrows the gap between smaller and larger models, although Gemma generally outperforms comparable Qwen models and 9B Gemma outperforms 27B Gemma.The framework also mitigates non-transitivity across variants, enabling midsized models to surpass significantly larger baseline models under controlled settings.
- Applications: TrustJudge remains effective for reasoning models despite their higher inconsistency rates, and its 100-point scoring improves DPO win rates by 16.21% for Llama-3-8B and 1.94%-10.69% for Qwen2.5-7B over 5-point baselines.The DPO results use AlpacaEval2 across 805 questions and support utility for both evaluation and preference optimization.
5 CONCLUSION
TrustJudge addresses fundamental inconsistencies in LLM-as-a-judge evaluation with probabilistic scoring and aggregation, reducing inconsistency while maintaining or improving evaluation accuracy.
- TrustJudge uses distribution-sensitive probabilistic scoring to preserve judgment entropy and likelihood-aware aggregation to mitigate inconsistent evaluations.
- TrustJudge reduces Score-Comparison and Pairwise Transitivity inconsistencies across LLM architectures and scales without compromising evaluation accuracy.The framework also improves exact match rates and win rates relative to established baselines.
- LLM-as-a-judge research has developed scalable discrete scoring and pairwise methods, but coarse annotations leave contradictions between scoring types unresolved.
- Probabilistic evaluation methods use fine-grained rating distributions to improve alignment with human preferences and address limitations of discrete judgments.
- Pairwise evaluation prompts require judges to compare responses and output a strict preference or tie verdict in a prescribed format.
B LIMITATIONS
TrustJudge evaluations depend on the instruction-following capabilities of the judge model, with smaller models more likely to fail to produce valid scores or comparisons.
- TrustJudge reliability is directly tied to the competence of the underlying evaluation model.
- Smaller language models may have weaker instruction comprehension and execution, causing failures to yield valid scores or comparisons.
- The evaluation examples include a single-score 5-point prompt and prompts for multi-dimensional assessment.
- The experiments cover strong and weak open-source and closed-source models, including models across multiple parameter scales.
E DPO TRAINING SETTINGS
DPO training uses a one-epoch cosine-scheduled setup with a 5.0×10−7 learning rate, β=0.1, global batch size 32, and sequence-length limits.
- DPO training runs for one epoch with a learning rate of 5.0×10−7 and temperature parameter β set to 0.1.
- The setup uses a global batch size of 32, four samples per device across eight GPUs, and a cosine schedule with 10% warmup.
- Maximum sequence length is 2048 tokens, while maximum prompt length is limited to 512 tokens.
F EXTENSION TO MULTI-DIMENSIONAL EVALUATION
TrustJudge generalizes to multi-dimensional evaluation across factuality, coherence, and helpfulness, reducing both inconsistency metrics across models and sub-dimensions.
- Setup: Multi-dimensional evaluation covers factuality, coherence, and helpfulness using 120 randomly sampled Arena Hard questions and dimension-specific prompts.
- Method: Per-dimension computation calculates CR and NTR_k independently before reporting averages across the three sub-dimensions.
- Implementation: Pairwise factuality prompts compare two candidate responses and require a final verdict identifying the more factually accurate response or a tie.
- Results and analysis: CR decreases by 5.13%–11.03%, while NTR3 and NTR4 decrease by 11.23%–24.99% across models in multi-dimensional evaluation.These reductions occur on every evaluated model using dimension-specific prompts and per-dimension computation.
- Results and analysis: For Llama-3.1-70B, NTR4 falls from 44.65% to 16.21%, NTR3 from 18.29% to 5.48%, and CR from 52.20% to 41.47%.
- Method: The scalar channel uses distribution-sensitive scoring, while the pairwise channel uses likelihood-aware aggregation with calibrated tie handling.
G GENERALIZATION ACROSS DATASET CATEGORIES
Across eight MT-Bench categories and three judges, TrustJudge’s likelihood-aware aggregation sharply reduces pairwise transitivity inconsistencies, while distribution-sensitive scoring is more category-dependent. The theoretical analysis explains this scoring behavior as preservation of information discarded by discrete modes.
- Results: 18.74% to 4.40%: likelihood-aware aggregation reduces the averaged non-transitivity ratio across 24 category–judge cells.The PPL-based method reaches 5.64%, and nearly every category and judge has a single-digit non-transitivity ratio.
- Results: 32.85% to 4.46%: likelihood-aware aggregation substantially cuts non-transitivity in the difficult Qwen–Math setting.Llama–STEM reaches 0.00%, illustrating the broad category coverage of the reduction.
- Results: 23.32% to 20.63%: distribution-sensitive scoring lowers averaged Score-Comparison inconsistency, but its advantage varies by category.It is best or tied best in Coding, Reasoning, and Writing, whereas G-Eval leads in several other categories and the raw baseline slightly leads in Math.
- Theoretical analysis: Discrete mode scoring maps distinct judge belief distributions to the same score, discarding distribution shape and entropy.The theorem constructs distinct distributions with different entropies but identical discrete scores, while expected-value scoring distinguishes them.
H.2 THEORETICAL ANALYSIS OF LIKELIHOOD-AWARE AGGREGATION
Likelihood-aware aggregation addresses ambiguous pairwise judgments by using rationale likelihoods to create more discriminative confidence signals and by combining both input orderings. Its PPL-based signal reduces entropy, while bidirectional aggregation guarantees symmetry and ordering stability.
- PPL-based aggregation: PPL-based confidence transforms ambiguous, high-entropy outcome beliefs into lower-entropy signals for final decisions.When rationale perplexities differ, an appropriate γ makes the confidence distribution non-uniform; as γ increases, it concentrates on the lowest-perplexity outcome.
- PPL-based aggregation: Entropy reaches its maximum for a uniform outcome distribution, so any non-uniform PPL-derived confidence distribution is strictly more discriminative.The argument uses the strict maximum of Shannon entropy at the uniform distribution.
- Bidirectional aggregation: Bidirectional aggregation is symmetric: swapping responses inverts the comparison outcome, unlike a single-pass estimator affected by positional bias.The aggregated score explicitly combines preference probabilities from both orderings.
- Bidirectional aggregation: The bidirectional estimator is stable with respect to input ordering because swapped-input scores correspond exactly under outcome inversion.This construction preserves equivalent scores for opposite outcomes after response order is exchanged.