PaperlayerView exact revision on arXiv

Source-linked AI summary

RubricHub: A Comprehensive and Highly Discriminative Rubric Dataset via Automated Coarse-to-Fine Generation

Sunzhu Li, Jiale Zhao, Miteto Wei, Huimin Ren, Yang Zhou, Jingwen Yang, Shunyu Liu, Kaike Zhang, Wei Chen

arXiv:2601.08430v2cs.AI

TL;DR

Open-ended generation lacks ground truth, while existing rubric methods are limited by manual effort, narrow domain coverage, and coarse criteria. The paper introduces an automated Coarse-to-Fine Rubric Generation framework and the approximately 110k-example RubricHub dataset, then uses RuFT and RuRL for post-training. Qwen3-14B reaches 69.3 on HealthBench, surpassing GPT-5 at 67.2, while the authors identify limited coverage of complex mathematics, competitive coding, and long-horizon agentic tasks as a scope boundary.

  • Problem

    Existing rubric methods face expensive manual construction, narrow domain coverage, and low discriminability, limiting scalable supervision for open-ended generation.

  • Method

    The paper combines principle-guided and response-grounded synthesis, multi-model aggregation, and difficulty evolution to build RubricHub and support RuFT and RuRL post-training.

  • Results

    69.3 versus 67.2 on HealthBench: post-trained Qwen3-14B surpasses GPT-5 and leads its official Non-thinking Instruct counterpart by 22.6 points.

  • Takeaways & Limitations

    RubricHub provides fine-grained, highly discriminative criteria for multi-domain evaluation and rubric-driven post-training.

  • Takeaways & Limitations

    RubricHub lacks systematic coverage of complex mathematics, competitive coding, and long-horizon agentic tasks.

Abstract

from arXiv · show

Reinforcement Learning with Verifiable Rewards (RLVR) has driven substantial progress in reasoning-intensive domains like mathematics. However, optimizing open-ended generation remains challenging due to the lack of ground truth. While rubric-based evaluation offers a structured proxy for verification, existing methods suffer from scalability bottlenecks and coarse criteria, resulting in a supervision ceiling effect. To address this, we propose an automated Coarse-to-Fine Rubric Generation framework. By synergizing principle-guided synthesis, multi-model aggregation, and difficulty evolution, our approach produces comprehensive and highly discriminative criteria capable of capturing the subtle nuances. Based on this framework, we introduce RubricHub, a large-scale ($\sim$110k) and multi-domain dataset. We validate its utility through a two-stage post-training pipeline comprising Rubric-based Rejection Sampling Fine-Tuning (RuFT) and Reinforcement Learning (RuRL). Experimental results demonstrate that RubricHub unlocks significant performance gains: our post-trained Qwen3-14B achieves state-of-the-art (SOTA) results on HealthBench (69.3), surpassing proprietary frontier models such as GPT-5. Our code is available at \href{https://github.com/teqkilla/RubricHub}{ this URL}.

1 Introduction

Open-ended generation lacks ground-truth answers, making quality judgments subjective and unstable; RubricHub addresses this with automated, discriminative rubric generation and rubric-driven post-training.

  • Motivation: Rubric-based evaluation decomposes open-ended quality into explicit, checkable criteria, providing more interpretable and stable training signals.This structured proxy helps narrow the gap between verifiable reasoning and open-ended generation.
  • Motivation: Existing rubrics face scalability, domain-breadth, and discriminability bottlenecks that create ceiling effects in supervision signals.Manual creation is expensive, many datasets are domain-specific, and coarse criteria miss subtle quality differences.
  • RubricHub: The generation framework combines principle-guided and response-grounded synthesis, heterogeneous-model aggregation, and difficulty evolution.These components target alignment with query intent, comprehensive coverage, reduced single-source bias, and stronger discrimination among high-quality responses.
  • Validation: 69.3 versus 67.2 on HealthBench: post-trained Qwen3-14B surpasses GPT-5 and leads its official Non-thinking Instruct counterpart by 22.6 points.The result comes from the two-stage RuFT and RuRL post-training pipeline.
  • RubricHub: RubricHub is a large-scale dataset of approximately 110k multi-domain rubrics providing fine-grained, highly discriminative supervision.The dataset is constructed from an automated Coarse-to-Fine Rubric Generation framework.

2 Preliminaries

Rubrics provide structured scoring guides for evaluating output quality through weighted criteria. Their criteria may be objectively verifiable or semantically assessed by language-model evaluators, producing structured reward signals.

  • Rubric: A rubric defines evaluation criteria and performance levels for assessing output quality in education and model evaluation.For each query q, the paper defines a fine-grained rubric R_q as a set of N_q weighted criteria.
  • Rubric: Verifiable criteria encode objective constraints such as format or word count and are assessed with rule-based systems.Semantic criteria instead capture qualitative attributes such as reasoning depth or tone.
  • Rubric: Semantic criteria capture qualitative attributes and require LLM-based evaluators.Each criterion includes semantic requirements and grader parameters.
  • Rubric: Criterion weights determine relative importance and provide the basis for structured reward signals r(q, o).The paper distinguishes rule-based evaluation from LLM-based semantic evaluation within this scoring framework.

3 Method

The method automatically generates fine-grained RubricHub criteria through response grounding, principle guidance, multi-model aggregation, and difficulty evolution, then applies them in RuFT and RuRL post-training. RubricHub spans multiple domains and is designed to provide dense, discriminative supervision.

  • 3 Method: Rubric generation proceeds through principle-guided and response-grounded synthesis, multi-model aggregation, and difficulty evolution.The pipeline anchors criteria to responses, combines heterogeneous perspectives, and refines criteria toward exceptional-response nuances.
  • 3 Method: Multi-model aggregation consolidates heterogeneous candidate rubrics into a compact base rubric that explicitly eliminates single-source bias.The aggregation stage addresses model-specific blind spots and subjective preferences by consolidating redundant items and resolving conflicts.
  • 3 Method: Difficulty evolution augments fundamental correctness criteria with discriminative nuances that distinguish excellent from exceptional responses.The final rubric merges base and additive criteria, upgrading generic checks into rigorous standards such as edge-case handling with O(n) complexity.
  • 3 Method: RubricHub contains approximately 110k question–rubric pairs across five domains, with Medical and Science each representing 27.1% of the dataset.Instruction Following contributes 20.9% and Writing 15.9%; Writing and Medical queries average over 30 fine-grained criteria.
  • 3 Method: RubricHub exhibits a non-saturated evaluation regime that separates model scales, while Qwen3-235B averages approximately 0.6.The reported score distribution indicates that evolved criteria remain challenging even for top-tier models.
  • 3 Method: RuFT filters generated responses by rubric scores for supervised fine-tuning, while RuRL converts rubric criteria into structured reward signals.RuFT selects the highest-scoring candidate above threshold τ, whereas RuRL assigns binary criterion scores and forms a weighted dense reward.

PNq

The supplied passage states that policy optimization uses DAPO under a rubric-based reward, but provides no further detail for this section.

  • Policy optimization uses DAPO under a rubric-based reward.

4 Experiment

Experiments evaluate RubricHub across five domains, compare post-training schemes and baselines, and analyze rubric quality, grader reliability, training dynamics, and rejection-sampling choices. The results consistently favor the complete RuFT→RuRL pipeline and fine-grained rubric design.

  • Main Results: The evaluation spans Science, Instruction-Following, Writing, Medical, and general-chat benchmarks, using proprietary, rubric-based, and official Qwen baselines.The proposed models are trained from Qwen3-4B and Qwen3-14B base models.
  • Main Results: Base < RuFT < RuRL < RuFT→RuRL across domains, while Qwen3-14B rises from 5.2 to 74.4 on ArenaHard V2.The authors characterize RuFT as a supervised cold start that enables RuRL to further optimize performance.
  • Main Results: HealthBench reaches 69.3 for Qwen3-14B, exceeding GPT-5 at 67.2 and outperforming Baichuan-M2-32B in four of five domains.The model also scores 92.6 on IFEval and 74.4 on ArenaHard V2, surpassing GPT-4.1 and DeepSeek V3.1 on those benchmarks.
  • Rubric Quality: RubricHub-generated rubrics improve HealthBench from 47.7 to 62.1 and ResearchQA from 76.7 to 82.5 relative to original RaR rubrics.Using the full dataset and then the complete RuFT→RuRL pipeline produces further gains and the best results among the compared settings.
  • Evaluation Reliability: Human–LLM agreement improves from F1 0.81 and κ 0.58 at 7B to F1 0.90 and κ 0.74 at 30B, then largely saturates through 235B.The authors report κ ranging from 0.74 to 0.80 among 30B, 120B, and 235B models.
  • Training Dynamics: HealthBench training metrics rise steadily and synchronously across Accuracy, Completeness, and Communication Quality rather than concentrating on one rubric dimension.The dynamics are presented as evidence of balanced capability improvement during training.
  • Ablation Study: Increasing rejection-sampling candidates from 1 to 12 raises the average maximum Training Set score from 63.45 to 79.51 and HealthBench from 43.61 to 48.81.The study links larger candidate pools and rubric filtering to improved final output quality.

5 Related Works

Related work uses rubric-based evaluation to make open-ended assessment more structured and reliable, while automatic rubric construction has emerged to reduce reliance on expert-authored criteria. Existing approaches mainly synthesize, mine, or induce rubrics from different data sources.

  • Rubric-Based Evaluation: Rubric-based evaluation decomposes response quality into interpretable criteria, improving structure and consistency over coarse-grained LLM judging.The cited literature includes expert-authored rubrics across multiple domains.
  • Automatic Rubric Construction: Automatic rubric construction has been explored to provide scalable rubric-style supervision beyond expert-designed criteria.The paper groups existing methods into three broad categories.
  • Automatic Rubric Construction: Existing methods include LLM-synthesized rubrics, rubrics mined from human-authored documents, and rubrics induced from preference data.These categories respectively generate criteria, structure evaluation dimensions from resources, or infer dimensions from pairwise comparisons.

6 Conclusion

The paper introduces RubricHub, a large-scale multi-domain dataset built with automated coarse-to-fine rubric generation, and uses it to drive rubric-based fine-tuning and reinforcement learning. Qwen3-14B achieves substantial gains, including performance above GPT-5 on HealthBench.

  • Conclusion: RubricHub contains approximately 110k multi-domain examples with fine-grained criteria designed to distinguish subtle differences among high-performing responses.Its generation framework combines principle-guided and response-grounded synthesis, multi-model aggregation, and difficulty evolution.
  • Conclusion: RubricHub drives Rejection Sampling Fine-Tuning and Reinforcement Learning, enabling scalable automated supervision for model alignment.The conclusion presents fine-grained rubrics as a solution for open-ended tasks lacking ground truth.
  • Conclusion: Qwen3-14B achieves significant gains and surpasses proprietary models such as GPT-5 on benchmarks including HealthBench.The passage presents this result as evidence for the efficacy of fine-grained rubrics.

7 Limitations

The paper’s limitations concern domain coverage, grader reliability and cost, and computational efficiency. RubricHub does not yet systematically cover several verifiable or long-horizon agentic tasks, while rubric-driven training remains resource-intensive.

  • Domain Scope: RubricHub primarily targets non-verifiable domains and lacks systematic coverage of complex mathematics, competitive coding, and long-horizon multi-step planning.GPQA-Diamond provides some scientific reasoning coverage, but the stated gaps remain.
  • Grader Reliability and Capacity: Negative Pitfall criteria add noise that degrades RL performance, while compact graders can fall below the reliability threshold even with positive criteria.The authors identify a need for specialized, high-precision compact grader architectures.
  • Grader Reliability and Capacity: The approach relies on costly large-scale graders because reliable evaluation depends on sufficient model capacity.This reliance is presented as a grader-capacity limitation rather than only a modeling choice.
  • Efficiency: Rubric-driven training, especially RuRL, incurs substantial computational overhead and inference latency despite partial mitigation through parallel grader deployment.The paper calls for further architectural optimization, including hybrid serial-parallel scoring, for efficient large-scale iterations.

B Detailed Training Settings

The training settings combine rubric-based response selection with rubric-driven reinforcement learning across two Qwen3 base models. The supplied prompts also specify structured JSON scoring and fine-grained, verifiable penalty criteria.

  • Models: Post-training uses Qwen3-14B and Qwen3-4B as base models.
  • RuFT: RuFT builds 30K instances by generating six candidates, retaining the highest-scoring response above threshold τ = 0.6, and using the result to initialize RuRL.GPT-5.1 generates candidates for randomly sampled prompts.
  • RuRL: RuRL trains on approximately 110K RubricHub instances separately by domain for up to five epochs with DAPO and eight rollouts per prompt.The configuration uses batch size 64, mini-batch 32, and a 1×10−6 AdamW learning rate.
  • Rubric evaluation: The evaluator returns JSON fields for an explanation and a boolean criteria_met value, considering all rubric criteria and marking negative criteria true only when undesirable behavior occurs.Examples clarify that concise responses do not meet an “overly verbose” criterion.
  • Rubric design: Rubric-generation rules require atomic, non-overlapping, scenario-specific, precise, directly checkable penalty criteria covering explicit and implicit requirements.The design also uses the reference answer to capture quality, safety, formatting, and completeness expectations.

Data Sample: Medical

The medical sample concerns postoperative dependent redness after ankle surgery and cast removal. Its criteria emphasize a cautious vascular explanation, recovery context, warning signs, clinician follow-up, and accessible organization.

  • Patient scenario: The sample asks why a recently operated right foot turns red when placed on the floor despite no pain.The patient had ankle surgery on 28 May and recently had the cast removed.
  • Action guidance: The rubric requires advice to inform the surgeon, seek earlier contact for warning signs, and relate physical therapy and movement to improved circulation.It also permits safe measures such as alternating elevation and gently lowering the leg.
  • Communication: The desired answer uses nontechnical language, directly addresses the core question, and organizes causes, normal findings, red flags, and next steps clearly.The criteria also require an educational disclaimer and consultation with the surgeon or physical therapist.
  • Medical explanation: The rubric expects gravity-dependent blood pooling and vasomotor instability after immobilization to explain the redness.Reduced calf-muscle activity during casting is described as limiting the muscle pump that returns blood up the leg.
  • Safety assessment: The response should distinguish common positional redness during recovery from complications and list warning signs such as severe pain, swelling, fever, or color and sensation changes.The criteria preserve reassurance while requiring attention to DVT or infection indicators.

Data Sample: Science

The science samples encode mathematical reasoning through explicit properties, equations, arithmetic, and answer formatting. They also test interpretation of ordered outcomes in a card-multiplication problem.

  • Incircle problem: The incircle problem gives triangle sides AB = 10 cm, BC = 9 cm, and AC = 7 cm and asks for BZ on side AB.
  • Incircle method: The expected solution uses equal tangent segments from each vertex and expresses the three side lengths as sums of tangent variables.Notation such as t_A, t_B, and t_C should remain consistent.
  • Incircle result: 6 cm is the required final value for BZ, with correct units and intermediate algebraic justification.The rubric also requires a logical sequence from setup through conclusion.
  • Card problem: The card problem asks for possible first-turn outcomes when two cards are flipped from a pile of 26 cards.The supplied options include 676, 52, 650, and 26.
  • Card counting: 650 is obtained by treating the first and second cards as an ordered selection: 26 × 25.The criteria distinguish this from the unordered combination count C(26,2) = 325.
Loading 2601.08430v2…