PaperlayerView exact revision on arXiv

Source-linked AI summary

GenRubric: Self-Evolving Rubric Generation for Scalable LLM Evaluation

Yifan Chen, Haitao Li, Qingyao Ai, Fengbin Zhu, Tat-Seng Chua, Min Zhang, Yiqun Liu

arXiv:2608.29856v1cs.CL

TL;DR

Open-ended LLM evaluation needs explicit, query-specific criteria, but expert-written rubrics are costly and existing automatic methods depend on inference-time refinement or external supervision. GenRubric self-evolves rubric generators from unlabeled queries using rubric-induced cross-consistency, and its generated-rubric evaluations agree more closely with expert rubrics across model sizes and held-out domains.

  • Problem

    Implicitly instantiated criteria leave open-ended evaluation requirements underspecified and difficult to audit, while expert-written rubrics require substantial expertise and annotation effort.

  • Method

    GenRubric uses reinforcement learning on unlabeled queries, rewarding rubric-induced responses that generalize across independently sampled rubrics with group-level and criterion-level quality signals.

  • Results

    GenRubric achieves stronger agreement with expert rubrics across 4B, 8B, and 14B models, with gains extending to held-out domains and consistent performance across four domains.

  • Takeaways & Limitations

    Rubric-induced self-consistency provides an effective signal for scalable query-specific rubric generation without additional human annotations during self-evolution.

  • Takeaways & Limitations

    The reinforcement-learning objective uses queries alone and discards responses, references, rubrics, labels, and scores from the training datasets.

Abstract

from arXiv · show

Large language models are increasingly used as scalable evaluators for open-ended tasks. However, many LLM judges derive query-specific criteria during scoring, leaving the evaluation requirements insufficiently specified and their coverage difficult to audit. Query-specific rubrics make these requirements explicit, but expert-written rubrics are costly to construct, while existing automatic methods typically rely on inference-time refinement or external supervision. We introduce GenRubric, a self-evolving framework that improves rubric generation from unlabeled queries without requiring additional human annotations during self-evolution. Our approach is based on rubric-induced self-consistency: independently sampled rubrics for the same query provide partial views of its latent evaluation requirements, and a comprehensive rubric should induce a response that generalizes across these complementary evaluation views. We implement this principle through reinforcement learning, combining a cross-rubric comprehensiveness signal with group-level and criterion-level rewards for rubric quality. We train GenRubric models at 4B, 8B, and 14B scales across multiple domains. Experiments on human-annotated rubric benchmarks show that self-evolution improves the agreement between evaluations induced by generated rubrics and those induced by expert-written rubrics. The improvements further generalize to held-out domains, demonstrating the potential of self-evolving rubric generation for scalable and query-specific LLM evaluation. Code and models are publicly available at https://github.com/foggpoy/GenRubric.

Introduction

Open-ended tasks require query-specific evaluation criteria, but those criteria are often implicit, costly to author, or learned from limited supervision. GenRubric addresses this gap by self-evolving rubric generation from unlabeled queries using cross-rubric behavioral consistency.

  • Open-ended responses can satisfy multiple valid criteria, including factual correctness, coverage, risk awareness, and practical utility.
  • Implicitly constructing evaluation criteria during scoring can blur requirements, underspecify evaluation boundaries, and make coverage and consistency difficult to audit.
  • Expert-written rubrics make query-specific requirements explicit and checkable, but constructing one for every query requires substantial expertise and annotation effort.
  • Existing rubric generators commonly learn from annotated response preferences, so their criteria can reflect available examples without fully representing a query’s evaluation requirements.
  • GenRubric treats independently sampled rubrics as complementary partial views and evaluates comprehensiveness by whether one rubric induces responses that generalize across the others.
  • GenRubric combines cross-rubric, group-level, and criterion-level reinforcement-learning rewards and improves agreement with expert-rubric-induced evaluations, including on held-out domains.

Related Work

Prior rubric generators either refine rubrics with fixed models at inference time or train them from response-based supervision. GenRubric instead evaluates rubrics through the cross-rubric generalization of their induced responses.

  • Training-Free Rubric Generation: Training-free methods construct or refine rubrics at inference time through dynamic elicitation, coarse-to-fine synthesis, recursive refinement, or retrieval augmentation.
  • Trainable Rubric Generators: Most trainable rubric generators use pairwise or binary response preferences, sometimes jointly with rubric-conditioned judges.
  • Trainable Rubric Generators: Response-based supervision is constrained by available response and preference distributions, while reliable preference collection remains costly in new domains.
  • Trainable Rubric Generators: GenRubric differs by assessing rubrics through the cross-rubric generalization of the responses they induce.

Method

GenRubric learns query-specific rubric groups through a closed loop that generates rubric-conditioned responses and cross-evaluates them with independently sampled rubrics. Reinforcement-learning rewards target comprehensiveness, response discrimination, criterion utility, and nonredundancy, with inference requiring only direct rubric generation.

  • Rubric generation: GenRubric learns a structured query-specific rubric generator πθ(R | x), where each item pairs a verifiable criterion with an importance weight.Positive weights specify desirable properties, while negative weights specify observable errors to penalize.
  • Rubric-induced cross-evaluation: During self-evolution, candidate rubric groups induce responses, which a fixed LLM evaluates across every valid rubric group sampled for the same query.Malformed outputs receive zero reward and are excluded from cross-evaluation.
  • Rubric-induced cross-evaluation: Cross-rubric comprehensiveness rewards responses that satisfy complementary requirements captured by independently sampled rubric groups.Scores are normalized by each evaluating rubric group's maximum attainable positive score before aggregation.
  • Group-level and criterion-level rewards: Group-level discrimination rewards rubric groups that distinguish responses of different quality, while criterion-level rewards favor discriminative and nonredundant rubric items.Criteria activated by every or no induced response provide little distinction, and identical activation fingerprints indicate functional redundancy.
  • Criterion-level credit assignment: Token-level Group Relative Policy Optimization assigns credit across rollouts and within rubric groups, giving stronger signals to higher-quality groups and useful individual criteria.Complete rubric-item spans receive item-level rewards, while structural tokens receive the sequence-level reward.
  • Inference: At inference time, GenRubric directly maps a new query to its rubric group without response generation or cross-evaluation.The response-generation and cross-evaluation procedures are used only during self-evolution.

Experiments

Experiments evaluate GenRubric across model sizes, training stages, reward components, domains, and response-quality subsets. GenRubric consistently improves agreement with expert rubrics, including held-out domains and already strong responses.

  • Main results: GenRubric outperforms specialized rubric generators and general-purpose LLMs across 4B, 8B, and 14B models.Performance generally improves with model size, while GenRubric-4B already surpasses general-purpose models on Top-1 consistency and pairwise accuracy.
  • Main results: GenRubric-4B reaches 0.603 Spearman correlation on ResearchRubrics, versus 0.322 for a DeepResearch-specific rubric generator.This comparison shows cross-domain training can remain effective on a specialized held-out task.
  • Rubric analysis: Stronger systems generate longer complete rubric groups, and rubric coverage is positively associated with evaluation agreement, although length alone is insufficient.GenRubric retains moderate criterion lengths while using additional tokens mainly to cover more evaluation requirements.
  • Training stages: SFT improves over base models, and RL further improves all four metrics at every tested model size.SFT provides structured generation, format-following, and basic task-solving capabilities; RL optimizes evaluation utility through rubric-induced self-consistency.
  • Training stages: Direct RL improves the 8B and 14B base models on the main rank-correlation metrics without cold-start supervision.The gains are less uniform for 14B, and direct base-to-RL training was evaluated only at 8B and 14B because of computational constraints.
  • Reward components: Optimizing only the cross-rubric comprehensiveness reward raises Spearman correlation from 0.324 to 0.438, while the full reward achieves the best result on every metric.Auxiliary rewards provide comparatively smaller and nonmonotonic intermediate changes but collectively offer complementary refinement.
  • Cross-domain generalization: All GenRubric variants outperform their corresponding base models on every metric in held-out non-finance ProfBench domains.GenRubric-14B leads rank correlations and pairwise accuracy, while GenRubric-8B leads Top-1 consistency.
  • High-quality responses: GenRubric achieves the best result on all four metrics when evaluation is restricted to each query’s three highest-scoring expert-rubric responses.All variants substantially improve over corresponding base models, indicating fine-grained differentiation among already strong responses.

Conclusion

GenRubric learns query-specific rubric generation through self-evolution without additional human annotations during that stage. Its cross-rubric behavioral consistency signal, combined with group- and criterion-level rewards, improves agreement with expert rubrics across model sizes, unseen domains, and high-quality responses.

  • Conclusion: GenRubric uses cross-rubric generalization of induced responses together with group-level and criterion-level learning signals.The framework learns query-specific rubric generation without additional human annotations during self-evolution.
  • Conclusion: Experiments at 4B, 8B, and 14B show stronger agreement with expert rubrics than general-purpose LLMs and existing rubric generators.The improvements also hold on unseen domains and high-quality responses.
  • Conclusion: The results support rubric-induced self-consistency as an effective signal for scalable rubric generation.

Reinforcement Learning Training Data

GenRubric’s reinforcement learning uses queries from six public datasets spanning medical, financial, legal, and deep-research domains. Training discards released responses and annotations, separates evaluation queries from the training pool, and therefore relies on unlabeled queries.

  • Query pool: The reinforcement-learning query pool combines six public datasets covering medical, financial, legal, and deep-research tasks.The sources are HealthBench, PRBench, Report-Bench, DeepResearch Bench, LEXam, and FinanceBench.
  • Unlabeled training: Training retains only each dataset’s query field and discards responses, reference answers, rubrics, labels, and scores.Consequently, reinforcement learning receives no supervision from the datasets’ released annotations.
  • Data separation: HealthBench and PRBench evaluation instances are excluded from the reinforcement-learning pool.This removes 300 HealthBench queries and 300 PRBench queries, including 150 financial and 150 legal queries.

Evaluation Metrics

The evaluation compares rankings induced by generated and expert-written rubrics across four complementary metrics, with query-level values aggregated at the dataset level. The metrics assess global ranking agreement, pairwise ordering, top-response identification, and fine-grained relation recovery.

  • Metric Overview: Four metrics compare response rankings induced by generated rubrics with those induced by corresponding expert-written rubrics.Both rubrics score the same n = 8 model responses for each evaluation query.
  • Ranking Agreement: Spearman’s ρ measures global monotonic agreement between generated- and expert-rubric rankings without requiring identical score ranges.A value of 1 indicates identical rankings, −1 completely reversed rankings, and values near 0 weak monotonic agreement.
  • Ranking Agreement: Kendall’s τb measures pairwise ordering agreement while correcting for ties in either score vector.It compares concordant and discordant response pairs and accounts for ties under the generated and expert rubrics.
  • Top-Response Selection: Top-1 Consistency measures whether generated-rubric top-scoring responses belong to the expert rubric’s top-scoring set without introducing false winners.The metric permits a subset of multiple expert-tied winners but returns 0 if any generated top-scoring response is not expert-top-scoring.
  • Pairwise Discrimination: Pairwise Accuracy measures exact recovery of the expert rubric’s higher, lower, or tied relation for every response pair.Each query contains eight responses, and ties predicted incorrectly in either direction count as errors.
  • Aggregation: Metric values are computed per query and rubric-generation run, averaged across runs, then aggregated by dataset or domain while excluding undefined values.Constant score vectors can produce excluded non-finite correlations.

Domain-Level Results

GenRubric performs strongly across medical, financial, legal, and deep-research queries, with gains distributed across domains rather than concentrated in one subset. Its results also show broadly positive scaling and a pronounced advantage on deep-research tasks.

  • Evaluation Set: 700 evaluation queries comprise 300 medical, 150 financial, 150 legal, and 100 deep-research queries.The main paper reports results averaged over the complete evaluation set, with domain-specific results in Tables 7–10.
  • Overall Performance: Every GenRubric variant outperforms all existing rubric-generator baselines on all four metrics in every domain.GenRubric achieves the best result in 14 of 16 domain–metric combinations and ranks second in the remaining two.
  • Overall Performance: GenRubric’s complete-evaluation gains remain consistent across medical, financial, legal, and deep-research tasks rather than being driven by one domain.The domain-level results support the same overall performance pattern across all four domains.
  • Legal Subset: On the legal subset, Claude-Sonnet-4.6 leads the rank-correlation metrics, while GenRubric-8B leads Top-1 consistency and pairwise accuracy.GenRubric-8B trails Claude-Sonnet-4.6 by 0.0206 on ρ and 0.0203 on τb, but outperforms other general-purpose LLMs on both.
  • Scaling: Moving from 4B to 8B improves all four metrics in every domain, while 14B adds gains on several metrics, especially in financial and deep-research subsets.The scaling trend is broadly positive but not strictly monotonic for every metric.
  • Deep-Research Subset: GenRubric-4B improves deep-research ρ from 0.3218 to 0.6032 and pairwise accuracy from 0.3646 to 0.7111 versus the specialized 30B generator.This comparison indicates the deep-research gains are not attributable solely to model scale.

Criterion-Level Evaluation Prompt

Table 11 presents the prompt used by the rubric judge to determine whether a response satisfies a criterion.

  • Prompt Purpose: Table 11 presents the rubric judge’s prompt for criterion-level evaluation.The prompt is used to determine whether a response satisfies a criterion.
  • Judgment Task: The rubric judge uses the prompt to determine whether each response satisfies the evaluated criterion.
  • Operational Role: The prompt operationalizes the rubric judge’s criterion-satisfaction decision for responses.

Robustness to the Rubric Judge

GenRubric’s improvements persist when the rubric judge is replaced with GLM-5.2, supporting robustness to judge choice. The relative ordering among GenRubric variants changes modestly, but every self-evolved model still beats its corresponding base model.

  • Experimental Setup: The sensitivity experiment replaces Qwen3.5-27B with GLM-5.2 and repeats evaluation for three Qwen3 base models and their GenRubric counterparts.
  • Robustness Result: At every parameter scale, GenRubric outperforms its corresponding base model on all four metrics under the alternative judge.
  • Variant Ordering: Under GLM-5.2, GenRubric-14B leads ρ, τb, and Top-1 consistency, while GenRubric-8B leads pairwise accuracy.These changes in variant ordering do not affect the separation between self-evolved and base models.
Loading 2608.29856v1…