PaperlayerView exact revision on arXiv

Source-linked AI summary

Rufus-Air: An Open LLM Post-Training Recipe

Chia-Yuan Chang, Renyuan Cheng, Rui Feng, Xiaotian Han, Yuan He, Hongye Jin, Linwei Li, Shiyang Li, Fenglin Liu, Xin Liu, Priyanka Nigam, Haoyang Wen, Zhenghao Xu, Zhuocheng Xu, Bing Yin, Qingyu Yin, Chao Zhang, Rongzhi Zhang, Zhihan Zhang, Zixuan Zhang, Zixuan Zhang, Tuo Zhao

arXiv:2609.29421v1cs.CLcs.AIcs.LG

TL;DR

Post-training recipes are often reported too thinly for teams to reproduce them. Rufus-Air documents an open, eight-stage recipe on GLM-4.5-Air-Base; it leads the official GLM-4.5-Air release on every reported benchmark except Arena-Hard v2 Creative Writing and is competitive with similarly sized open models.

  • Problem

    Post-training recipes are often reported thinly, with practical details needed for reproducibility among the least disclosed.

    Source: §1 · p. 3

  • Method

    Rufus-Air documents a full post-training recipe using open-source components, public data, and no newly commissioned human annotation.

    Source: §7 · p. 31

  • Results

    Rufus-Air leads the official GLM-4.5-Air release on every reported benchmark except Arena-Hard v2 Creative Writing and is competitive with similarly sized open models.

    Source: §1 · p. 3

  • Takeaways & Limitations

    The recipe is documented so teams with tens of nodes can follow it stage by stage or reuse an individual stage or infrastructure component.

    Source: §7 · p. 31

    Rufus-Air trails both GLM-4.5-Air and Nemotron-3-Super on Creative Writing, and the authors have not tried a style-tuned reward model.

    Source: §5.2 · p. 29

Abstract

from arXiv · show

Rufus-Air is an open and reproducible post-training recipe on GLM-4.5-Air-Base (106B-A12B), organized as a serial pipeline of eight stages: SFT, Reasoning RL, Coding RL, Instruction-Following RL, General Agent, Coding Agent, Search Agent, and RLHF. We document the data, reward design, infrastructure, stage order, and stagewise results needed to reproduce the recipe. Stages progress from basic to advanced capabilities and from hard, verifiable rewards to softer judge-based signals. Training builds on open-source components and public data, much of it used as released, without new human annotation or an in-house distillation teacher. Our main findings are that (i) diverse, high-quality SFT establishes a strong capability floor; (ii) difficulty filtering keeps RL prompts within a productive learning range; (iii) reward reliability provides a practical principle for ordering stages; and (iv) infrastructure and engineering choices are part of the recipe, not just an implementation detail. Rufus-Air improves over the official GLM-4.5-Air post-trained release and is competitive with similarly sized open models.

1 Introduction

Rufus-Air addresses the thin reporting of post-training recipes with an open, reusable account of its full recipe. It reports that the model leads the official GLM-4.5-Air release on all reported benchmarks except Arena-Hard v2 Creative Writing and is competitive with similarly sized open models.

1 Introduction

  • Post-training recipes are often reported thinly, leaving teams without details needed to reproduce them.The authors contrast this with a few exceptions, including Tülu 3 [49].Source: §1 · p. 3
  • Rufus-Air is presented as a reusable recipe built on GLM-4.5-Air-Base [33], using open-source components and a compute footprint small enough for teams outside frontier labs.Some conclusions draw on training experience rather than full ablations, and the report marks them as such.Source: §1 · p. 3
  • The recipe treats SFT as capability-building, filters RL prompts to a productive difficulty band, orders stages by reward-hacking exposure, and documents enabling infrastructure choices.The infrastructure examples include token-in/token-out rollouts, consistent chat templates, a reliable sandbox, and large batches with Rollout Routing Replay [59].Source: §1 · p. 3
  • Rufus-Air leads the official GLM-4.5-Air release on every reported benchmark except Arena-Hard v2 Creative Writing and is competitive with similarly sized open models.The authors identify the documented, reusable recipe—not benchmark performance—as the contribution they care about.Source: §1 · p. 3

2 Recipe Overview

Rufus-Air is a serial eight-stage post-training pipeline built on GLM-4.5-Air-Base [33]. Its order moves from basic to advanced capabilities and, broadly, from more verifiable rewards toward softer signals; the finished model leads on most reported instruction-following, alignment, and agentic comparisons, with stated exceptions.

2 Recipe Overview

  • The recipe starts from public GLM-4.5-Air-Base [33], a 106B-total, 12B-active-parameter MoE checkpoint, chosen for its public weights, compatible stack, and practical RL scale.The base model is used unchanged; Rufus-Air names the final checkpoint, while intermediate checkpoints are named for their producing stage.Source: §2 · pp. 3–4

2.1 Pipeline Stages and Order

  • The recipe runs eight serial stages, each trained from the prior checkpoint, from SFT through RLHF; the stages target general capability, specialized tool use, and open-ended quality.SFT, Reasoning RL, Coding RL, and Instruction-Following RL build general capability; General Agent, Coding Agent, and Search Agent add tool use.Source: §2.1 · p. 4, Figure 1 · p. 4
  • Stages progress from basic to advanced capabilities and generally from hard, verifiable rewards toward softer signals, limiting exposure to gameable rewards.The order tracks reward-hacking exposure rather than reward format alone: IF RL uses a rubric-based judge but runs before agent stages because instruction following is close to existing policy behavior.Source: §2.1 · p. 4

2.2 Recipe Outcome

  • Rufus-Air leads on instruction-following and most alignment and agentic rows, with exceptions on Tau2-Airline, Tau2-Retail, and Arena-Hard v2 Creative Writing; it ties Nemotron-3-Super on Terminal-Bench 2.1.On competition mathematics and knowledge, where Reasoning RL does the work, Rufus-Air sits close to the other models.Source: §2.2 · pp. 5–6, Table 1 · p. 5

3 Post-Training Stages

This section covers the eight-stage post-training pipeline, from SFT and general-capability RL through specialized agent training and RLHF, with rewards progressing from hard, verifiable signals toward softer judge-based signals. It details SFT, Reasoning RL, Coding RL, and Instruction-Following RL, including data choices, training results, and instruction-following rubric effects on formatting, complex instructions, reasoning ability, and response length.

3.1 Supervised Fine-Tuning

  • SFT checkpoint 3799 provided a strong but uneven starting policy, and held-out scores flattened within the first epoch despite later reductions in training loss.The 9.01M-sample mix used public dataset releases without a Rufus-Air response-regeneration pass or new human annotation. Checkpoint 3799 became the RL initialization and established message, reasoning, and tool-call formats for downstream verifiers and judges.Source: §3.1 · pp. 7, 9, Table 4 · p. 9

3.2 Reasoning RL

  • +5.3 points in GPQA: Reasoning RL raised the score from 68.2 to 73.5, while both AIME years fell by 2.6–2.8 points.The stage relied on correctness filtering to retain prompts with a demonstrated valid solution and learnability filtering to exclude prompts that were too easy or currently unsolved. Online selection kept groups with average reward in (0, 0.8], advancing the difficulty frontier as the policy improved.Source: §3.2 · pp. 10–11, Table 6 · p. 12

3.3 Coding RL

  • +7.3 points in LiveCodeBench v6 pass@1: Coding RL improved over the Reasoning RL checkpoint, with pass@1 peaking at 75.9 after extending the response budget.Most gains accrued during the 64K-token phase; extending the budget to 128K reduced truncation from 11–28% to below 0.1%, after which reward and pass@1 continued to rise. Pass@8 improved by 2.3 points.Source: §3.3 · pp. 13–14, Table 8 · p. 14, Figure 4 · p. 14

3.4 Instruction-Following RL

  • Instruction-Following RL targets both multi-constraint single-turn requests and multi-turn conversations requiring consistent adherence to layered instructions.Single-turn prompts combine 2–6 Python-verifiable constraints; adversarially generated conversations test persistence, revisions, distractors, and system-prompt hierarchy, using programmatic checks or an LLM judge for rubrics.Source: §3.4 · pp. 14–15
  • IFEval rose +4.0 points to 94.5, IFBench +14.0 points to 77.8, MultiChallenge +24.7 points to 55.8, and AdvancedIF +17.4 points to 62.9 after instruction-following RL.Table 9 measures the stage against the Coding RL checkpoint it trains from; the reported gains target hard formatting and multi-turn or multi-constraint instruction following.Source: Table 9 · p. 16, §3.4 · p. 16
  • The stage filtered out likely unanswerable samples and prompts with over 80% pass rate, then jointly trained two datasets on the Coding RL checkpoint.The final dataset contains 13K conversations averaging 6.1 turns and 2.7 rubrics; training used GRPO for 90 steps.Source: §3.4 · p. 16
  • The stage assigns positive reward only when every required rubric is satisfied, using code checks for mechanical constraints and an LLM judge for subjective ones.Optional desirable properties are excluded because they can add reward noise and encourage length bias.Source: §3.4 · p. 16
  • Instruction-following RL did not regress reasoning ability, while training after Reasoning RL yielded larger instruction-following gains than training directly from SFT, especially on AdvancedIF.GPQA improved +3.2 and AIME 25 changed −1.2; the authors attribute the curriculum effect to reasoning as a prerequisite for complex instructions.Source: §3.4 · p. 16
  • Necessary-condition rubrics shortened final responses, but adding optional-content rubrics produced long, redundant outputs as the policy tried to satisfy more possible rubrics.The optional-rubric setup used on the order of ten rubrics per instance.Source: §3.4 · p. 17

3.5 General Agent

The agent stages train from general tool orchestration toward coding and web search, then RLHF shapes open-ended response quality; their reported results include gains on targeted benchmarks and a shipped-checkpoint comparison.

3.5 General Agent

  • General Agent training raises MCP-Atlas pass@1 by +7.80 points and Tau2-Retail by +9.80 points over the IF RL checkpoint.The stage filters out tasks that neither a strong teacher nor the initial policy ever solves, then trains on tasks the current policy does not already solve reliably using terminal binary verification.Source: §3.5 · pp. 17–18

3.6 Coding Agent

  • SWE-bench Verified pass@1 moves from 65.60 to 67.80, and Terminal-Bench 2.1 from 38.76 to 40.17 after Coding Agent training.The stage used about 4K learnable tasks and was not trained to convergence; the SWE-bench gain does not survive to the shipped checkpoint, while Terminal-Bench 2.1 later reaches 42.7.Source: §3.6 · pp. 19–20

3.7 Search Agent

  • Search Agent adds +3.0 on BrowseComp, +5.4 on Seal-0, and +3.4 on HLE-Verified over the Coding Agent checkpoint.The training set keeps questions solved in 1–3 of 8 initial-policy rollouts; the LLM judge gives partial credit, and evaluation uses a different standardized harness and a search backend unseen in training.Source: §3.7 · pp. 20–23, Table 12 · p. 22

3.8 RLHF

  • Arena-Hard v2 Hard Prompt rises from 83.06 to 89.05 (+5.99), and Creative Writing from 38.56 to 52.97 (+14.41) after RLHF.Training uses filtered HH-RLHF prompts and an open reward model with a linear length penalty; monitored benchmarks move by at most 1.3 points over the stage.Source: §3.8 · pp. 24–25

4 Training Infrastructure

The training stack combines shared open-source components with in-house training and agent-environment layers. Its infrastructure handles long SFT sequences, RL stability, multi-turn tool interaction, and faithful token transfer.

4 Training Infrastructure

  • Rufus-Slime runs SFT and every RL stage on Megatron-LM and SGLang, while Rufus-Gym adds multi-turn rollout for agent stages.Both in-house components are built on open-source software.Source: §4 · p. 25

4.1 SFT and General RL

  • SFT packs sequences up to 128K tokens, uses an effective batch size of 4096, and takes about 13 days for three epochs on 64 nodes.Loss is computed only on assistant turns; system prompts, user turns, and tool observations remain in context but are masked.Source: §4.1 · p. 25
  • Rollout Routing Replay caches MoE expert-routing decisions from rollout and replays them during training, removing that component of the rollout–training log-probability gap.Other stability choices include large rollout batches, mixed-precision rollout with BF16 training, and truncation-ratio monitoring.Source: §4.1 · pp. 25–26

4.2 Agentic RL and Its Environment

  • The agent environment returns a token trajectory, reward, and typed termination signal, while the agent loop alternates inference and tool execution until a stop condition fires.Sampled token IDs and metadata pass unchanged to the trainer; coding trajectories use a dedicated Firecracker microVM sandbox for each active trajectory.Source: §4.2 · pp. 26–27

5 Evaluation

Rufus-Air is compared with similar-scale open-weight models across instruction-following, reasoning, coding, and agentic benchmarks. It generally leads the same-base models and is strongest against Nemotron-3-Super on instruction-following and agentic tasks, while results across stages require careful interpretation.

5.1 Benchmarks

  • The evaluation spans automatically checkable and open-ended instruction-following tests, competition mathematics, expert science questions, and competitive programming.The listed benchmarks include IFEval, IFBench, Multi-challenge, Arena-Hard v2, AIME, GPQA-Diamond, LiveCodeBench v6, and FrontierScience.Source: §5.1 · p. 28
  • Agent evaluations cover simulated customer service and multi-step tool tasks, persistent web research, terminal coding, and verified software issues.The named tests include Tau2-Bench, MCP-Atlas, BrowseComp, Seal-0, HLE-Verified, Terminal-Bench 2.1, and SWE-bench Verified.Source: §5.1 · p. 28
  • Tau2-Bench and MCP-Atlas use reimplemented harnesses, while LLM-judged evaluations use one judge per benchmark without multi-judge rescoring.MCP-Atlas uses the official tool container and benchmark grading prompt; tool-error failures are retried rather than counted as failures.Source: §5.1 · p. 29

5.2 Main Results

  • Rufus-Air leads the two same-base models on every reported row except Creative Writing and beats Nemotron-3-Super on many instruction-following and agentic rows.Against Nemotron-3-Super, it is within two points on several mathematics, science, and agent rows, and trails on Creative Writing and Tau2-Airline.Source: §5.2 · p. 29
  • Creative Writing is the only reported row on which Rufus-Air trails both GLM-4.5-Air and Nemotron-3-Super.RLHF raised the Creative Writing score by +14.4, but the authors had not tried a style-tuned reward model.Source: §5.2 · p. 29
  • Stage-to-stage changes should be read against the immediately preceding checkpoint, and some benchmark values are not directly comparable across protocols.Terminal-Bench 2.1, SWE-bench Verified, and Seal-0 stage-to-final differences span intervening stages and are not attributed to any one stage.Source: §5.2 · pp. 29–30
  • Multi-challenge moves most under IF RL (+24.7), Creative Writing under RLHF (+14.4), and Tau2-Retail under General Agent (+9.8).The Coding Agent run, which ended at its compute budget, moves least on its two reported target results (+1.4 and +2.2).Source: §5.2 · p. 30

6 Discussion

The discussion presents SFT as the capability floor, filtering as a way to focus RL on learnable prompts, and reward reliability as a practical basis for stage order. It also emphasizes infrastructure and cost, while limiting what the sequential recipe establishes.

On SFT

  • Before any RL, SFT checkpoint 3799 leads the public GLM-4.5-Air release on IFEval (+5.3), IFBench (+24.2), and both AIME years (+6.6 and +3.5), but trails on GPQA (−5.7).The authors treat this checkpoint as the floor from which RL begins.Source: §6 · p. 30

On learnability filtering

  • Outcome-reward RL stages filter prompts the policy already solves and, in most stages, prompts it never solves; online filtering or dynamic sampling further excludes unproductive groups.Coding RL keeps prompts never solved in warm-up and does not re-filter during training.Source: §6 · p. 30

On stage order

  • The pipeline puts less gameable reasoning and coding rewards early and the preference reward of RLHF last, with IF RL early because its target is close to current policy behavior.The authors say later stages kept GPQA and AIME 26 within a point of earlier checkpoints while adding targeted skills, while noting inconsistent intermediate GPQA readings.Source: §6 · p. 30

On infrastructure

  • The training stack uses Rufus-Slime, Megatron-LM, and SGLang, while Rufus-Gym supports on-policy multi-turn agent rollouts, persistent coding sandboxes, and distributed environment actors.These engineering choices address trajectory consistency and contention between tool execution and reward computation.Source: §6 · p. 31

On the cost of agentic RL

  • Agentic RL adds substantial non-GPU costs, so the authors advise budgeting those stages separately rather than using a per-stage average.Coding Agent sandbox service costs about $10K per month; one BrowseComp pass is estimated at a few hundred dollars at list prices.Source: §6 · p. 31

On what the recipe does not establish

  • The sequential stage order is one that worked, not one shown to be optimal, and Coding Agent was trained to the available compute rather than convergence.No benchmark suite was tracked across every stage, so movements between untracked stages cannot be attributed to any one stage.Source: §6 · p. 31

7 Conclusion

Rufus-Air documents an open post-training recipe on public GLM-4.5-Air-Base and reports results above the public GLM-4.5-Air release on nearly all reported benchmarks. The report is intended as a stage-by-stage resource that other teams can follow or adapt.

7 Conclusion

  • Rufus-Air documents an eight-stage post-training pipeline on public GLM-4.5-Air-Base, using open-source components, public data, and no newly commissioned human annotation.The data include public SFT datasets, much of them used as released, and RL prompt sets built from public sources and synthetic generation.Source: §7 · p. 31
  • The resulting model leads the public GLM-4.5-Air release on every reported benchmark except Arena-Hard v2 Creative Writing and is competitive with similarly sized open models.The comparison is reported in Table 1 and §5.2.Source: §7 · p. 31
  • The recipe is documented for teams with tens of nodes to follow stage by stage or reuse an individual stage or infrastructure component.Its open-source stack and public data allow others to check and build on the recipe.Source: §7 · p. 31

A Contributions

Contributions are grouped in pipeline order and then by cross-cutting work, with stage leads named first. The listed work spans stage design and data, training infrastructure and agent environments, evaluation, project direction, and report editing.

Pipeline stages

  • Stage leads are listed first for SFT, Reasoning RL, Coding RL, Instruction-Following RL, General Agent, Coding Agent, Search Agent, and RLHF.The contribution list organizes stage work in recipe order.Source: A Contributions · p. 41
  • Search Agent and RLHF contributions include stage leadership and work on design, experiments, data curation, and evaluation-harness implementation.The Search Agent entry names a stage lead, design and evaluation-harness contributors; the RLHF entry names a lead and experiment and data-collection contributors.Source: A Contributions · p. 41

Cross-cutting work

  • Infrastructure work includes Rufus-Slime, Megatron-LM training modules, tracking, and Rufus-Gym’s agentic RL extension and harness.The contribution list names leads and contributors for both training infrastructure and the agent environment.Source: A Contributions · p. 41
  • Evaluation contributions included integrating agentic evaluation into Rufus-Gym and integrating non-agentic evaluation.Each stage’s leads evaluated their own stage.Source: A Contributions · p. 42
  • Project direction covered pipeline design, stage order, experiment and ablation choices under a fixed compute budget, and cross-stage resource coordination.The report editor wrote cross-cutting material on conclusions, stage order, and the protocol for reporting public baseline numbers.Source: A Contributions · p. 42

B Training Configurations

The appendix pins the training and rollout stack and directs readers to a per-stage configuration table covering all eight pipeline stages.

Infrastructure stack

Per-stage settings

  • Table 16 summarizes configurations for all eight stages, including General Agent and Coding Agent settings not stated in the main text.The supplied table caption identifies it as the per-stage training configuration summary.Source: B Training Configurations · p. 42, Table 16 · p. 43

C Data Tables and Example Transformations

The data tables document the public SFT mixture, its dataset-page terms, and the provenance of released responses, distinguishing upstream data creation from Rufus-Air's own use.

Dataset tables

  • Table 17 records public SFT datasets, their download locations, dataset-page terms, and the upstream provenance of released responses.Its caption clarifies that provenance describes released datasets, not Rufus-Air-side response regeneration.Source: Table 17 · p. 43, C Data Tables and Example… · p. 43

Dataset terms

  • AM-Thinking-v1-Distilled is marked for research use only, with commercial and harmful uses prohibited.The dataset page states this additional use restriction alongside the model authors’ Apache-2.0 release.Source: C Data Tables and Example… · p. 44

Released-data provenance

  • Upstream-generator entries expose dependencies in public releases; Rufus-Air does not regenerate those responses or record an additional Rufus-Air generator checkpoint or output license.The upstream-generator column does not supersede dataset-page terms.Source: C Data Tables and Example… · p. 44

D How to Read the Public Baseline Numbers

Publicly reported baseline numbers are assembled from multiple sources rather than measured under one common setup. Their provenance and evaluation differences qualify how Table 1 comparisons should be read.

Reading public comparisons

Cell provenance

  • Cell sourcing prioritizes a model’s own card or report, then other reports or benchmark sources when cells are missing, with the benchmark owner’s leaderboard taking precedence.Numbers are included only when benchmark versions match; unversioned numbers are omitted.Source: D How to Read the Public Baseline… · p. 44

Harness differences

Harness and version caveats

  • Agentic benchmarks use each source’s own harness, while judge and baseline details are sometimes unstated and benchmark versions or judges can differ.The passage also notes that the same checkpoint can receive different scores across sources.Source: D How to Read the Public Baseline… · pp. 44–45

E Search Agent Evaluation and Trajectory Analysis

The Search Agent evaluation standardizes search tools and grading, then analyzes context management and tool-use trajectories. The analysis links BrowseComp outcomes to scratchpad use and search depth, while other benchmarks show different call patterns.

Evaluation harness

  • The evaluation uses one search harness across models, with Google Serper, Jina Reader, a sandboxed Python interpreter, and Claude Sonnet 4.5 grading final answers.Live pages and searches are not cached or shared across models, so scores are comparable only within one evaluation window.Source: E Search Agent Evaluation and… · p. 46

Context management

  • 10.7 points: disabling external memory with a scratchpad drops BrowseComp accuracy on the same checkpoint.Among the single choices varied in this stage, this is the largest measured effect.Source: E Search Agent Evaluation and… · p. 46

Tool-call patterns

  • Table 19 reports tool use per trajectory; GLM-4.5-Air’s BrowseComp search:scrape ratio is 6.4:1, while Search Agent’s final ratios are 2.3:1 on BrowseComp, 1.3:1 on HLE-Verified, and 0.9:1 on Seal-0.Coding Agent scrapes two to three times as many pages on every benchmark as the public release.Source: Table 19 · p. 46, E Search Agent Evaluation and… · p. 47

Trajectory depth

  • On BrowseComp, Search Agent reaches 75.3 iterations, but accuracy falls from 88% within 25 iterations to 64% at 25–50 and below 30% beyond 50.On Seal-0 and HLE-Verified, Search Agent is shallower than its starting checkpoint; about 10% of BrowseComp trajectories end from token-budget exhaustion or context overflow, essentially none correct.Source: E Search Agent Evaluation and… · p. 47

F Tau2-Bench with Different User Simulators

Table 20 compares Tau2-Bench pass@1 across customer simulators, showing that scores can shift substantially while some rankings remain stable. Its Rufus-Air column is an earlier checkpoint than the final model in Table 1.

F Tau2-Bench with Different User Simulators

  • Rankings are more stable than scores: the full ordering is unchanged in telecom, and Rufus-Air ranks first in retail under all three simulators, by just 0.2 points under Sonnet 5.The Rufus-Air column is an earlier checkpoint; the final checkpoint was evaluated under Sonnet 5 only.Source: F Tau2-Bench with Different User… · p. 47, Table 20 · p. 47
  • The airline leader varies by simulator: Nemotron-3-Super leads under Sonnet 4 and Sonnet 5, while Rufus-Air leads under Sonnet 4.5.Match simulator settings when comparing results across papers.Source: F Tau2-Bench with Different User… · p. 47
  • 23.7 points: changing the user simulator shifts scores by up to this amount, as GLM-4.5-Air retail scores 57.0 under Sonnet 4 versus 80.7 under Sonnet 5.Sonnet 5 yields the highest scores in all twelve model–domain pairs.Source: Table 20 · p. 47, F Tau2-Bench with Different User… · p. 47

Related papers

Loading 2609.29421v1…