Source-linked AI summary
APTER: Adaptive Post-Training with Expert-Grounded Rubrics
Xukai Wang, Liangqi Li, Zhiyue Xu, Jingang Zhou, Xiaoyu Shi, Jiansheng Cai, Bo Zhang, Zhe Li, Xu-Yao Zhang
TL;DR
Specialized reasoning requires domain-appropriate constraints, evidence, and complete reasoning, but existing post-training signals do not explicitly encode which criteria to check or how failures should guide improvement. APTER uses expert-grounded query-level rubrics and adaptive criterion-level repair, improving mathematics and medical macro-averages across three model generations by up to 15.86 and 8.04 points, respectively.
Problem
Existing post-training signals insufficiently encode expert-defined criteria for evaluating and improving specialized reasoning beyond final-answer correctness or holistic preference.
Method
APTER instantiates reusable expert criteria into query-level rubrics and uses their verdicts for optimization, diagnosis, and targeted repair.
Results
Mathematics and medical macro-averages improve across all three model generations, reaching gains of 15.86 and 8.04 points, respectively.
Takeaways & Limitations
APTER’s benefits are especially evident when responses must satisfy multiple query-specific professional criteria.
Abstract
from arXiv · showhide
As large language models enter professional domains, they must satisfy domain constraints, include critical evidence, and provide complete reasoning rather than merely produce fluent responses. Existing post-training methods often rely on holistic preferences or outcome-level verification, while recent rubric-based methods usually generate rubrics independently for each query. In specialized domains, such unconstrained rubrics may omit critical requirements and vary across samples, hindering the diagnosis and targeted repair of persistent capability deficiencies. We propose APTER (Adaptive Post-Training with Expert-Grounded Rubrics), a framework that integrates structured domain knowledge into fine-grained evaluation, optimization, and diagnosis for specialized complex reasoning. First, expert-grounded rubric construction starts from an expert criteria framework built by domain experts, where each criterion represents a stable professional capability. For each query, APTER selects relevant criteria and instantiates them into query-level rubrics linked to their source criteria, turning reusable expert criteria into executable query-level supervision without reference answers. Second, adaptive post-training uses rubric verdicts as both optimization and criterion-level diagnostic signals. Aggregating low-scoring verdicts by criterion ID reveals persistent deficiencies and triggers targeted supervised fine-tuning updates during reinforcement learning. Experiments on mathematical reasoning and medical question answering show consistent gains across both domains. Across three model generations, APTER improves the mathematics and medical averages over the corresponding base models by up to 15.86 and 8.04 points, respectively. Code and rubric datasets are available at https://github.com/AntDT-APTER/APTER.
1. Introduction
APTER addresses the limits of coarse, outcome-level, and independently generated rubric supervision for specialized reasoning by grounding query-level evaluation in reusable expert criteria. It uses rubric verdicts for optimization, diagnosis, and targeted repair without requiring reference answers.
- Motivation: Specialized tasks require domain constraints, critical evidence, and complete reasoning beyond fluent responses or plausible conclusions.Medical benchmark analyses indicate that evaluation should use lifecycle-oriented, safety-aware, and clinically faithful criteria specified by domain experts.
- Motivation: RLHF and RLVR provide effective but insufficiently expert-grounded supervision because they do not explicitly represent required domain criteria.RLHF uses pairwise preferences, whereas RLVR uses outcome verification for tasks with verifiable answers.
- Rubric construction: APTER routes expert-defined criteria to each query and instantiates executable rubrics without requiring reference answers.This construction addresses the risk that independently generated rubrics omit or misprioritize domain-critical requirements.
- APTER framework: APTER grounds query-level rubrics in reusable, expert-constructed professional criteria for fine-grained evaluation, optimization, and capability diagnosis.The criteria include requirements such as evidence coverage, constraint handling, and reasoning criteria.
- Adaptive post-training: Rubric verdicts serve as optimization and criterion-level diagnostic signals, enabling Ada-IFT to identify persistent deficiencies and trigger targeted repair.APTER implements this adaptive post-training framework for mathematical reasoning and medical question answering with expert criteria frameworks and rubric datasets.
2. Related Work
Prior work develops scalable rubric generation, rubric-based rewards, and RL–SFT interleaving, but query-specific rubrics may vary in granularity and domain depth. APTER links reusable expert criteria to query-level rubrics and uses criterion-linked verdicts for rewards, diagnosis, and targeted repair.
- Rubric construction and evaluation criteria: Recent rubric-based methods study scalable rubric generation, refinement, retrieval, and adaptive design, while HealthBench emphasizes expert-aligned criteria for professional evaluation.These approaches commonly generate rubrics independently for individual queries or responses, allowing flexibility but causing variation in granularity and domain depth.
- Rubric construction and evaluation criteria: APTER instantiates relevant expert criteria into query-level rubrics with stable source links, providing reusable supervision without reference answers.The pipeline uses routing, multi-role generation and refinement, and expert-in-the-loop calibration to construct the rubrics.
- Rubric-based rewards for reasoning RL: Rubric-based reasoning rewards extend from outcome verification and process supervision to rubric scores used in reward design, anchors, and advantage decomposition.APTER likewise uses rubric-level rewards, while stable expert-criterion links make verdicts reusable for capability diagnosis.
- Interleaving RL with supervised fine-tuning: ReST, rejection sampling, ReLIFT, and iterative self-improvement interleave RL with SFT to repair reasoning failures that exploration alone may not resolve.APTER instead triggers repair from recurring failures aggregated under stable criteria, using verdicts for Rubric RL, Rubric-based SFT, and Ada-IFT.
3. Methodology
APTER combines expert-grounded rubric construction with adaptive post-training: expert criteria become query-level rubrics with retained provenance, and their verdicts support reinforcement learning, supervised fine-tuning, and criterion-level diagnosis. Its default Ada-IFT configuration uses recurring criterion failures to trigger targeted repair.
- Adaptive post-training: APTER supports Rubric RL, Rubric-based SFT, and Ada-IFT by using judge-produced rubric scores as rewards, supervised-data selectors, or diagnostic signals.Rubric-based SFT retains high-scoring candidate responses, whereas Ada-IFT extends Rubric RL with criterion-level diagnosis and supervised repair.
- Expert-grounded rubric construction: APTER builds reusable expert criteria into query-level rubrics that retain source criterion IDs and are calibrated through expert feedback.The criteria framework organizes professional capabilities hierarchically and remains fixed within a training run, enabling stable aggregation across queries.
- Expert-grounded rubric construction: For each query, a router selects relevant criteria, while analysts, a consolidator, an auditor, and a refiner produce weighted, executable rubrics without reference answers.The multistage process checks clarity, assessability, relevance, redundancy, wording, scoring scales, and weights while preserving criterion provenance.
- Adaptive post-training: Rubric rewards provide finer-grained, more interpretable supervision than holistic rewards or outcome-level verification by scoring specific reasoning and domain-compliance criteria.Retained criterion provenance makes verdicts reusable for diagnosis in Ada-IFT.
- Adaptive post-training: Ada-IFT triggers targeted SFT when recurring failures on stable expert criteria indicate persistent capability deficiencies, rather than relying only on question difficulty or a fixed curriculum.The method aggregates low-scoring verdicts by criterion, pauses RL when a criterion reaches the trigger threshold, and retrieves criterion-indexed repair samples.
4. Experiment
APTER improves mathematics and medical performance across three Qwen generations, with especially large gains on open-ended clinical evaluation. Ablations attribute these gains to rubric-based rewards, expert-grounded query-specific rubrics, Ada-IFT repair, and rubric refinement.
- Main results: 14.99, 15.86, and 6.13 points: mathematics macro-average gains across Qwen2.5-7B-Instruct, Qwen3-8B, and Qwen3.5-9B, respectively.APTER improves every reported mathematics benchmark across all three model generations.
- Main results: 6.63, 5.19, and 8.04 points: medical macro-average gains across the three Qwen generations, while HealthBench gains reach 17.96, 15.42, and 16.45 points.Improvements on exam-style MedQA and MedMCQA are generally smaller than on HealthBench.
- Post-training method ablation: 44.60 and 53.48: Ada-IFT raises AIME 24 and HealthBench from Rubric RL’s 41.63 and 50.54, respectively.Rubric RL already improves RiC-SFT from 28.37 to 41.63 on AIME 24 and from 47.10 to 50.54 on HealthBench.
- Reward type and rubric granularity: 37.75→41.63: rubric rewards outperform binary outcome verification on AIME 24, and query-level rubrics outperform five generic criteria on every benchmark.The query-level versus generic-criteria comparison includes HealthBench, which rises from 47.84 to 50.54.
- Rubric provenance: 32.98 →41.63 and 47.27 → 50.54: expert grounding improves AIME 24 and HealthBench over independently constructed question-level rubrics.The expert-grounded variant routes rubrics from a shared expert-defined criterion framework.
- Rubric Refiner effectiveness: 4.00, 4.96, 2.87, and 2.36 points: refined rubrics improve MATH500, AIME 24, OlympiadBench, and HealthBench-500, respectively.These matched runs indicate improved reward quality rather than merely a changed rubric count.
5. Conclusion · Appendix
APTER is presented as a framework for specialized complex reasoning that integrates structured domain knowledge into fine-grained evaluation, optimization, and capability diagnosis. Its rubric construction organizes reusable expert criteria into query-level rubrics with retained provenance, without per-query expert authoring or reference answers.
- 5. Conclusion: APTER integrates structured domain knowledge into fine-grained evaluation, optimization, and capability diagnosis for specialized complex reasoning.
- 5. Conclusion: The framework includes an expert-grounded rubric construction component.
- 5. Conclusion: This component organizes domain expertise into a reusable criteria framework.
- 5. Conclusion: It instantiates relevant criteria as query-level rubrics with retained criterion provenance.
- 5. Conclusion: The construction process does not require per-query expert authoring.
- 5. Conclusion: The construction process does not require reference answers.
A. Method and Reproducibility Details · A.1. Computational Overhead
Ada-IFT uses the same judge verdicts as vanilla Rubric RL while adding a diagnostic channel for criterion-level failure analysis. Its routine diagnostic overhead is lightweight, with additional frontier-model computation triggered only when persistent deficiencies require repair.
- A.1. Computational Overhead: Ada-IFT shares the same judge verdicts as vanilla Rubric RL.
- A.1. Computational Overhead: The reward channel aggregates rubric-level scores into scalar rewards.
- A.1. Computational Overhead: The diagnostic channel uses rubric scores to compute criterion-level failure rates and counters.
- A.1. Computational Overhead: During ordinary RL steps, the diagnostic channel adds only lightweight aggregation and indexing.
- A.1. Computational Overhead: A persistent criterion-level deficiency triggers an additional repair process.
- A.1. Computational Overhead: Repair additionally invokes a frontier model to generate candidate repair responses.
A.2. Targeted SFT Sample Fields
APTER stores verified improved responses with their queries and source criteria, then places them in a criterion-indexed SFT buffer for targeted updates before returning to reinforcement learning.
- Targeted SFT Sample Fields: The supervised tuple is (q, y★, c), augmented with provenance metadata for auditing and formed only from verified improved responses.Inputs include the query, current policy response, low-scoring query-level rubric, source criterion, and diagnostic information.
- Targeted SFT Sample Fields: Accepted samples enter the criterion-indexed SFT buffer and drive a targeted update before training returns to RL.
A.3. Medical Training Data Composition · A.4. Expert Criteria Framework
APTER’s medical training data uses II-Medical queries with newly constructed expert-grounded rubrics, while its expert criteria frameworks encode persistent domain capabilities for query-level evaluation. The medical framework contains 10 capabilities and 50 criteria, routed and instantiated per query with persistent source IDs.
- A.3. Medical Training Data Composition: APTER uses only II-Medical queries from RubricHub and constructs new expert-grounded rubrics for the medical training subset.
- A.3. Medical Training Data Composition: Table 3 reports the exact query counts used after the medical training files were constructed.
- A.4. Expert Criteria Framework: Domain experts construct the frameworks by enumerating recurring capabilities and organizing them as domain → sub-domain → capability → leaf criterion.
- A.4. Expert Criteria Framework: Each leaf criterion stores a persistent criterion ID and a scope definition, making the framework reusable across questions.
- A.4. Expert Criteria Framework: The mathematics framework contains seven sub-domains, 24 capabilities, and 103 leaf criteria, with repeated capability names specialized by mathematical context.
- A.4. Expert Criteria Framework: 10 capabilities and 50 criteria comprise the medical framework, whose relevant criteria are instantiated as observable binary rubrics while retaining source criterion IDs.
A.5. Training Configuration … C.1.1. Case Study Design
The supplementary sections specify APTER’s reproducible training, rubric-refinement, calibration, prompt, and expert-annotation procedures. Together, they show how criteria are operationalized, audited, repaired, judged, and inspected across four mathematics failure cases.
- A.5. Training Configuration: APTER separates mathematics and medical training configurations because context length, sampling, KL regularization, and batching differ.Table 6 provides the complete Qwen3-8B recipe for controlled ablations and the Ada-IFT case study.
- A.6. Rubric Refiner Statistics: 7.0% of rubric sets remain unchanged after refinement, while average criteria per mathematics question decrease from 4.82 to 4.09.The Auditor–Refiner screens for redundancy, surface bias, and boundary ambiguity on a 5k-question subset.
- A.7. Expert Annotation Decision Flow: Experts route invalid or unscorable evaluations into shared standard calibration, while valid standards proceed to score confirmation or overriding.Calibration actions can revise rubrics or criteria, remove irrelevant criteria, and support later judge, rubric-generation, framework, or weight adjustments.
- B. Prompt Templates: The prompt templates make APTER reproducible across routing, multi-role rubric generation, judging, candidate generation, targeted repair, and rubric-in-CoT SFT.The mathematics examples are representative; runtime placeholders are filled with query, criterion, rubric, and response fields.
- B.2. Multi-role Rubric Generation Prompts: Rubric generation emphasizes problem-critical, concrete, binary, mutually independent criteria and rejects generic, nondiscriminative dimensions.The multi-role process uses Analyst, Consolidator, Auditor, and Refiner roles; refinement targets atomicity and defenses against surface-level bias.
- B.3. Rubric Judging Prompt: The judging prompt returns one Boolean verdict per rubric item and marks an item false whenever any required part is missing or incorrect.Mathematics judging additionally ignores superficial formatting differences and returns only the ordered Boolean array.
- C.1.1. Case Study Design: The case study contains four mathematics items representing distinct rubric-evaluation failure modes and records verdicts, weighted aggregates, expert scores, and resolution actions.Its purpose is to illustrate expert calibration for model-answer, judge-model, rubric-direction, and criterion-level defects, making the process auditable.
C.1.2. Case 1: Model Answer Defect · C.1.3. Case 2: Judge Scoring Defect · C.1.4. Case 3: Rubric Direction Defect
The three calibration cases distinguish defects in the model answer, judge scoring, and rubric direction. APTER responds by confirming genuine answer errors, overriding rigid judge scores, or rewriting flawed rubrics.
- C.1.2. Case 1: Model Answer Defect: Case 1: The model correctly discounts USD 80 to USD 60 but incorrectly adds USD 8 tax, reporting USD 68 instead of USD 64.80.The tax should be 0.08 * 60 = 4.80 and applies to the discounted price.
- C.1.2. Case 1: Model Answer Defect: Case 1: Because judge and expert scores are both 0, calibration confirms a genuine model-answer defect rather than a judge or rubric flaw.This is identified as the only pure agreement case and demonstrates model-answer validation.
- C.1.3. Case 2: Judge Scoring Defect: Case 2: Expert calibration overrides the judge score while keeping the reasonable rubric unchanged.The response is mathematically correct, and dividing by 2 is sufficiently clear.
- C.1.4. Case 3: Rubric Direction Defect: Case 3: The model supports the claim that n^2 even implies n even only with examples, so the judge assigns 1 while the expert assigns 0.Examples cannot establish a universal statement for every integer.
- C.1.4. Case 3: Rubric Direction Defect: Case 3: The generated rubric is flawed because it accepts examples and a correct conclusion without requiring a logically valid proof method.The rubric should require contrapositive reasoning or parity cases, including the fact that every odd integer has an odd square.
- C.1.4. Case 3: Rubric Direction Defect: Case 3: Calibration marks the rubric flawed and redirects the query to rubric revision, requiring a universal proof obligation and logical conclusion for all integers.Examples may support intuition but cannot serve as sufficient evidence.
C.1.5. Case 4: Criterion Defect
Case 4 shows criterion-level repair: the rubric’s overall direction is sound, but R3 imposes a defective fixed three-step requirement. The repair revises R3 to require sufficient verifiable intermediate reasoning while preserving the other criteria.
- Case 4: Criterion Defect: The model correctly scales 3 cups of flour to 9 and computes 2 * 3 = 6 cups of sugar.R1 and R2 both receive Judge 1 and Expert 1.
- Case 4: Criterion Defect: R3 is defective because it requires at least three calculation steps for full credit, although the expert rates it 1.The judge rates R3 0 while the expert rates it 1.
- Case 4: Criterion Defect: The calibration action replaces R3’s fixed step count with a sufficiency-based requirement for enough reasoning to verify proportional scaling.The revised criterion does not fix the number of calculation steps.
- Case 4: Criterion Defect: The case keeps the rest of the rubric intact because only one criterion’s operational standard requires revision.This distinguishes criterion-level repair from changing the rubric’s overall direction.
C.1.6. Discussion
APTER supports expert intervention at four levels—model answer, judge model, rubric, and criterion—rather than limiting annotation to final correctness labels. This multi-level calibration records disagreement root causes as structured supervision for improving rubric generation, judge calibration, and expert criteria refinement.
- C.1.6. Discussion: Expert annotation in APTER is not limited to assigning a final correctness label.The framework enables intervention at multiple levels.
- C.1.6. Discussion: Experts can intervene at the model-answer, judge-model, rubric, and criterion levels.They can verify genuine response errors, correct automated scoring errors, rewrite misdirected query-level rubrics, or revise flawed criteria.
- C.1.6. Discussion: Multi-level calibration prevents disagreements from collapsing into a single expert-versus-judge label.Instead, it identifies the root cause of each disagreement as structured supervision.
- C.1.6. Discussion: The structured supervision supports future rubric generation, judge calibration, and expert criteria framework refinement.This mechanism is presented as important for reliable rubric-based supervision.
C.2. Effectiveness of Process-based Rubrics in RL
APTER’s process-based rubrics distinguish final-answer correctness from reasoning quality, assigning zero to an uncertified lucky answer, exposing repairable capability failures, and granting full credit only to complete derivations. Three Qwen3-8B rollouts therefore illustrate how criterion-level verdicts support outcome-sensitive reward and diagnosis.
- C.2. Effectiveness of Process-based Rubrics in RL: The three rollout cases contrast a correct answer with unsound reasoning, a failed trajectory exposing a repair signal, and a fully correct solution.Each case includes query-level rubrics, per-criterion verdicts, weighted reward, and the original response.
- C.2. Effectiveness of Process-based Rubrics in RL: Together, the cases show that process-based rewards reject uncertified lucky answers, identify concrete deficiencies for repair, and reward complete derivations.The reward therefore does not merely track the final boxed value.
- C.2.1. Case 1: Reward Hacking — Correct Answer, Unsound Method: Case 1 receives rubric score 0.000 despite the final answer being correct.The response obtains 8 through asymptotic and integral approximations without the controlled error bounds required by the rubrics.
- C.2.1. Case 1: Reward Hacking — Correct Answer, Unsound Method: All four Case 1 process criteria fail because the method never certifies that S_128 - S_32 lies within an interval guaranteeing the nearest integer.The weighted rubric total is 0.000, whereas an outcome-only verifier could assign full credit to the lucky answer.
- C.2.2. Case 2: Reasoning Collapse Triggering Ada-IFT Repair: Case 2 receives rubric score 0.000 and final answer wrong after abandoning pair-generation and de-duplication for an unsupported known result.Although it correctly counts the three inscribed squares, the response fails all three required criteria.
- C.2.2. Case 2: Reasoning Collapse Triggering Ada-IFT Repair: Case 2 exposes a concrete capability failure in completeness of case analysis that can trigger targeted Ada-IFT repair.The required solution partitions squares by included vertices and derives N2 + 18 = 198, yielding 183.
- C.2.3. Case 3: Full Credit — A Clean, Rigorous Solution: Case 3 earns weighted reward 1.000 because all five criteria pass after a logically complete derivation using Vieta’s formulas.The solution preserves every sign and derives k = 7 without a logical gap.