PaperlayerView exact revision on arXiv

Source-linked AI summary

Learning to Judge: LLMs Designing and Applying Evaluation Rubrics

Clemencia Siro, Pourya Aliannejadi, Mohammad Aliannejadi

arXiv:2602.08672v1cs.CLcs.LG

TL;DR

LLM evaluators may not share human-defined notions of quality, motivating a test of whether they can design and apply their own rubrics. GER-Eval separates rubric generation from application and evaluates coherence, consistency, and human alignment across tasks and models. Models are internally consistent with their own rubrics, but reliability weakens for factual tasks and agreement fragments across architectures.

  • Problem

    It remains unclear whether LLMs’ self-defined rubrics produce consistent scores and meaningfully align with human evaluation criteria.

  • Method

    GER-Eval decouples rubric generation from application, having LLMs generate criteria and then use them to score candidate outputs.

  • Results

    LLMs generate coherent, interpretable, task-aware rubrics and apply them consistently within models, but show weaker factual-task reliability and cross-model agreement; GPT-4o outperforms Llama on human alignment.

  • Takeaways & Limitations

    Evaluation behaves as a learned linguistic capability: models understand their own evaluative language better than human or peer rubrics, revealing internal coherence but limited shared understanding.

  • Takeaways & Limitations

    The study covers only a subset of English-language closed-source and open-weight models and does not independently verify factual accuracy beyond human reference scores.

Abstract

from arXiv · show

Large language models (LLMs) are increasingly used as evaluators for natural language generation, applying human-defined rubrics to assess system outputs. However, human rubrics are often static and misaligned with how models internally represent language quality. We introduce GER-Eval (Generating Evaluation Rubrics for Evaluation) to investigate whether LLMs can design and apply their own evaluation rubrics. We evaluate the semantic coherence and scoring reliability of LLM-defined criteria and their alignment with human criteria. LLMs reliably generate interpretable and task-aware evaluation dimensions and apply them consistently within models, but their scoring reliability degrades in factual and knowledge-intensive settings. Closed-source models such as GPT-4o achieve higher agreement and cross-model generalization than open-weight models such as Llama. Our findings position evaluation as a learned linguistic capability of LLMs, consistent within models but fragmented across them, and call for new methods that jointly model human and LLM evaluative language to improve reliability and interpretability.

1 Introduction

LLMs are increasingly used to evaluate generated text, but it remains unclear whether their rubric dimensions and judgments match human evaluative criteria. GER-Eval tests whether models can generate and apply their own rubrics and finds coherent within-model evaluation alongside important cross-model and task-dependent limitations.

  • LLM-based evaluation extends beyond surface metrics by using natural-language rubrics to approximate human judgments across NLG tasks.
  • The reliability, consistency, and linguistic biases of model judgments remain unresolved, especially regarding whether models use human-intended dimensions or internal representational cues.
  • Prior LLM-as-judge studies report sensitivity to prompt phrasing, label order, response length, position, self-preference, and rerun instability.
  • Existing approaches have explored model-generated reward signals, fine-grained criteria, interactive evaluation functions, and task-specific criteria, but GER-Eval tests their underlying reliability assumption.
  • GER-Eval separates rubric generation from application: models generate names, definitions, and scales, then use those rubrics to score outputs with zero-shot or few-shot prompts.
  • Across four datasets, models generate coherent, interpretable, task-relevant rubrics and apply them consistently within models, but cross-model agreement and human alignment are weaker.
  • GPT-4o shows higher internal consistency and stronger human alignment than open-weight Llama models, while factual and knowledge-intensive settings reduce reliability across models.

2 Related Work

NLP evaluation has traditionally used human-authored benchmarks and task-specific rubrics, while newer LLM-based evaluators offer greater scalability. GER-Eval addresses the untested assumption that self-defined LLM rubrics reliably produce consistent, human-aligned scores.

  • Human-annotated benchmarks and predefined rubrics provide reliable gold standards but are expensive to scale and often inflexible across domains and tasks.
  • LLM-based benchmarks prompt models to rate or rank responses as automated evaluators.
  • Prior work assumes self-defined LLM rubrics improve score reliability and consistency relative to human-defined rubrics, but this assumption has not been systematically verified.
  • GER-Eval investigates whether LLMs can generate and apply self-defined rubrics consistently while meaningfully aligning with human evaluation criteria.

3 GER-Eval: A Framework for LLM-Generated Evaluation Rubrics

GER-Eval is a two-stage framework that separates rubric generation from rubric application, enabling analysis of how models design and use evaluation criteria. The experiments vary tasks, prompts, rubric sources, models, and datasets to test consistency and human alignment.

  • 3 GER-Eval: A Framework for LLM-Generated Evaluation Rubrics: GER-Eval decouples rubric generation from application to study how models conceptualize rubrics and apply their own criteria.
  • 3.1 Notation: Each generated criterion contains a name, description, scale, and scoring instruction, while candidate outputs form the evaluation set.
  • 3.2 Rubric Generation: Rubric generation varies Task-only, Task + Contexts, and Task + Contrastive Examples prompts to test stability and domain-sensitive specificity.
  • 3.3 Rubric Application: During application, models score each candidate using the task, criterion, instruction, output, and optional demonstrations, producing reasoning and a score.
  • 3.3 Rubric Application: Scores may be numerical or categorical, and Rubric-only and Rubric + Demonstrations conditions test application with and without few-shot examples.
  • 3.3 Rubric Application: The framework is directly comparable to human-defined rubrics and supports analysis across tasks and models.
  • 3.4 Experimental Setup: Experiments span conversational, summarization, and instruction-following tasks across four benchmarks with human scores on multiple quality attributes.
  • 3.4.5 Models: The study compares closed- and open-source models and uses task-specific rubric counts and prompting-condition analyses to examine model behavior.

4 Results

LLMs generate structured, largely human-aligned rubrics, but rubric application is more stable within models than across models or against human judgments. Reliability and transfer are strongest for conversational tasks and decline when evaluation requires factual, technical, or domain-specific reasoning.

  • Rubric Generation: Models generate five to eight rubrics per dataset, with few-shot prompting generally increasing rubric breadth.GPT-4o and GPT-4o-mini produce around 90% unique rubrics.
  • Rubric Generation: Generated criteria exceed 80% alignment with human rubrics across most datasets, reaching 100% on HelpSteer2 for several models under few-shot prompting.SumPubMed often falls below 60%, reflecting the difficulty of reproducing specialized biomedical terminology and coverage.
  • Rubric Generation: Prompt design and model priors shape rubric quality: Mixtral reaches 92% alignment with task-only prompting, while Llama reaches 94% with contrastive examples.Contextual and contrastive examples produce richer, more varied rubrics, while GPT-4o remains comparatively stable across setups.
  • Rubric Generation: Demonstrations increase task-grounded criteria, especially for conversational and instruction-following tasks, whereas summarization prompts yield more generic rubrics.GPT-4o and Mixtral generate more task-specific rubrics than Llama or Qwen, indicating that prompt design and model priors influence alignment.
  • Rubric Application: Agreement between zero-shot and few-shot scores typically reaches 70–90%, with correlations above 0.8 for most configurations using LLM-generated rubrics.GPT-4o and Mixtral are most robust, while agreement decreases on SummEval and especially SumPubMed.
  • Rubric Application: With human rubrics, correlations with human scores reach ρ ≈ 0.8–0.9 on HelpSteer2 and USR, but fall to around 0.5 on SummEval and below 0.3 on SumPubMed.Variance is largest in the biomedical domain, indicating sensitivity to task complexity and domain shift.
  • Rubric Transfer: Overall, models reproduce their own scoring patterns more consistently than they match human evaluations, especially when tasks demand domain reasoning or factual consistency.Conversational rubrics generalize more robustly because they rely on broadly shared linguistic features, unlike specialized-domain rubrics.
  • Rubric Transfer: Rubric transfer is strong on USR, with ICC ≈0.7–0.8, α ≈0.74–0.81, and ρ ≈0.7–0.9, but weak on SumPubMed, where ICC < 0.2 and ρ < 0.3.Accuracy, Coverage, and Terminology Use show near-zero or negative correlations in SumPubMed.

5 Discussion

LLMs can generate and apply coherent rubrics, but reliability depends on task domain and model family. Their strongest consistency is within models and their own rubrics, while factual evaluation and cross-model transfer remain limited.

  • LLMs reliably generate coherent evaluation rubrics, but reliability and validity depend on task domain and model family.
  • Task dependence: Conversational and instruction-following datasets show the highest stability and agreement with human judgments.Reliability decreases on factual or domain-specific tasks such as SUMPUBMED, where factual knowledge and terminological precision matter.
  • Task dependence: In factual settings, models produce internally consistent scores while prioritizing fluency and surface coherence over factual correctness.This reflects a reliability–validity trade-off: procedural stability does not ensure semantic adequacy.
  • Rubric transfer: Models interpret their own rubrics most effectively, human rubrics moderately well, and other models’ rubrics least consistently.Human rubrics align better on fluency and coherence than on factual or knowledge-oriented dimensions.
  • Rubric transfer: Each model develops a distinct evaluation dialect that is internally coherent but externally misaligned.Rubric-transfer inconsistency persists even when models use similar criteria, implying divergent evaluative semantics across architectures.
  • Implications: Evaluation may be a learned linguistic behavior arising from internal representations rather than fixed external standards.The discussion proposes collaborative frameworks in which human and model rubrics co-evolve with factual grounding.

6 Conclusion

The paper examines whether LLMs can generate and apply their own rubrics reliably across conversational and domain-specific tasks. It finds coherent, consistent linguistic evaluation but weaker factual reliability, with GPT-4o outperforming Llama in human alignment.

  • LLMs generate coherent rubrics and apply them consistently in open-domain evaluation but struggle with factual and knowledge-intensive tasks.
  • Closed-source models such as GPT-4o align more closely with human judgments than open-weight models like Llama.
  • Models interpret their own evaluation language more effectively than human or peer rubrics, revealing internal coherence but limited shared understanding.

Limitations

The study’s conclusions are bounded by its model, language, modality, and evaluation-setting coverage. It also does not independently verify factual accuracy or model evaluator change over time.

  • The study evaluates a subset of closed-source and open-weight LLMs on English-language datasets.Results may not generalize to smaller, multilingual, or domain-specialized models.
  • The analysis covers textual outputs and rubric-based evaluations, without examining multimodal inputs or interactive feedback.
  • The study does not independently verify factual accuracy beyond human reference scores and treats LLMs as static evaluators.Temporal drift and adaptation over time are not modeled.

Ethical Considerations

The paper frames LLM evaluation as a transparency and oversight issue rather than a replacement for human judgment. It highlights risks from automation bias, overreliance, and model-specific bias.

  • Using LLMs as evaluators may introduce automation bias, overreliance on model judgments, and model-specific biases.
  • The study promotes transparency about where LLM evaluation is reliable and where it remains limited.
  • Hybrid evaluation pipelines should keep human judgment central in factual, safety-critical, and socially sensitive domains.

A Additional methodology Details

GER-Eval generates rubric criteria, creates instructions for each criterion, applies them to outputs, and returns structured scores with reasoning.

  • GER-Eval generates a set of rubric criteria M from a task and prompting condition.
  • For each rubric m_i, the process generates scoring instructions I(m_i).
  • Each rubric instruction is applied to outputs Y to produce reasoning r_i,j and ratings s_i,j.
  • The final result represents each rubric, its instruction, and output-level reasoning and scores.

A.2 Datasets

The appendix evaluates GER-Eval across four text-generation domains, multiple model families, prompting conditions, rubric statistics, and structured generation and scoring prompts.

  • Datasets: The datasets cover news summarization, instruction-following, open-domain dialogue, and biomedical summarization.Together, they test generalizability across diverse text-generation tasks.
  • Models: The model set includes GPT-4o, GPT-4o-mini, Mixtral 8×22B, Llama 3.3-70B-Instruct, and Qwen 2.5-72B-Instruct.This enables comparisons across proprietary and open-weight model families.
  • Experimental settings: Scoring uses temperature 0 with greedy decoding, while rubric generation uses temperature 0.7 and nucleus sampling with p = 0.9.Generation and scoring therefore use different decoding settings to encourage rubric diversity while keeping scoring deterministic.
  • Experimental settings: The study samples 50 data points per dataset and uses task-specific positive and negative responses in the contrastive condition.Scoring produces categorical judgments when labels exist and otherwise maps ratings to a continuous 1–5 scale.
  • Rubric statistics: Breadth counts generated criteria, Unique removes semantically redundant criteria using a cosine-similarity threshold τ = 0.82, and Alignment estimates overlap with human rubrics.Alignment uses GPT-4o to tag generated criteria against human rubric items; criteria with no matching tag are unseen.
  • Prompts: The appendix documents separate prompts for rubric generation, instruction creation, and scoring, including task information, criteria, steps, and candidate outputs.An end-to-end HelpSteer2 example illustrates rubric generation and application.

C Supplemental Results

Supplemental analyses report task-sensitive rubric names, human rubric references, inter-rater reliability, score distributions, LLM–human correlations, and scoring strictness across datasets.

  • Rubric analyses: The appendix reports model- and dataset-specific task-sensitive rubric names alongside corresponding human-defined rubrics.It also summarizes global rubric frequencies and overlap with human rubrics.
  • Reliability analyses: Inter-rater reliability is reported across LLMs under zero-shot and few-shot conditions for each evaluation rubric.Table 13 includes ICC(2), Krippendorff’s α, and Fleiss’s κ across all four datasets using human rubrics.
  • Score distributions: Figure 6 visualizes Coherence score distributions across datasets under zero-shot prompting using violin plots with medians and quartiles.
  • Correlation analyses: Figures 7 and 8 compare Spearman correlation ρ between LLM and human scores across four datasets under zero-shot and few-shot conditions.
  • Scoring behavior: Figure 9 analyzes LLM scoring behavior by contrasting strictness and leniency across datasets under zero-shot prompting.
Loading 2602.08672v1…