PaperlayerView exact revision on arXiv

Source-linked AI summary

From Production Traffic to Post-Training: Building a Self-Hosted LLM That Covers the Corporate Request Mix

Olga Tsymboi, Dmitrii Stoianov, Ramil Latypov, Danil Taranets, Daniil Dryabin, Mikhail Gashkov, Viktor Zelenkovskiy, Aleksandr Fida, Gleb Alektorov, Nikita Gulyakov, Arthur Babkin, Aleksandr Medvedev, Pavel Gein, Anatolii Potapov

arXiv:2609.01572v1cs.CL

TL;DR

Enterprises need self-hosted models but face fragmented fleets and production-specific quality gaps. The paper addresses this with production-stratified evaluation and separate GRPO experts merged from a shared SFT checkpoint. The resulting recipe supports a single Qwen3-32B deployment serving 116M monthly requests, while its evidence remains limited to Russian and English, one organization, and the Qwen3 family.

  • Problem

    Enterprises must self-host LLMs under data-residency constraints, yet fragmented fleets and production-specific gaps complicate consolidating traffic onto one model.

  • Method

    The paper uses production-matched benchmarks and trains separate GRPO experts for instruction following, function calling, and task-distribution alignment before two-stage SLERP merging.

  • Results

    The final non-reasoning model is competitive on deployment metrics while serving 116M monthly requests at lower cost, and an internal-data-free checkpoint shows similar public-benchmark gains.

  • Takeaways & Limitations

    The recipe provides a practical path from production error analysis to consolidating a fragmented self-hosted fleet into a single deployment model.

  • Takeaways & Limitations

    Quantitative claims are validated only for Russian and English traffic, one organization and platform, and the Qwen3 model family.

Abstract

from arXiv · show

Data-residency constraints force enterprises to self-host LLMs, but continuous adoption of newer models without decommissioning their predecessors expands the serving fleet, fragmenting a finite GPU pool. We consolidate traffic from over 200 internal applications onto a single model by closing quality gaps identified through production error analysis along three axes: instruction following, function-calling, and internal task distribution. Quality is tracked by offline benchmarks stratified to production traffic and scored by deterministic verifiers or calibrated LLM judges. Rather than optimising all objectives jointly, which introduces cross-domain reward interference, we train a separate GRPO expert per axis and merge them via two-stage SLERP. Each expert's reward exposes a distinct failure mode, namely semantic collapse, over-calling, and verbosity hacking, each requiring a domain-specific fix. In non-reasoning mode the recipe surpasses a ${\sim}7\times$ larger by total parameters baseline on the in-house Arena with 69.6 to 65.8, instruction following with 0.85 to 0.83, and function-calling with 0.79 to 0.77, while lifting general dialogue benchmarks. The model absorbs 50% of platform traffic, 116M requests per month, at a fraction of the serving cost.

1 Introduction

The paper targets fragmented self-hosted LLM fleets by closing production-identified quality gaps and consolidating enterprise traffic onto one model. It combines production-matched evaluation with modular post-training while preserving broad capabilities.

  • Motivation: Local deployment often leaves enterprises sharing a finite GPU pool across hundreds of applications, while model switching friction and fleet fragmentation raise effective token costs.New model generations arrive quarterly, but older models cannot be retired.
  • Motivation: 116M requests per month from over 200 internal applications reach the adapted model, which absorbs 50% of platform traffic six months after rollout.The model is cheap enough to update every production cycle.
  • Production gaps: 37.9% of production failures are instruction-following and formatting errors, while tool-equipped requests comprise ∼12% of traffic and task distributions differ from public training data.These observations define the three improvement axes: instruction following, function calling, and internal task distribution.
  • Evaluation: The paper builds internal production-traffic benchmarks scored by deterministic verifiers or calibrated LLM judges and validated against human annotators.This evaluation supports diagnosing live weaknesses while testing generalization.
  • Method: Separate RL experts address each weak axis and are combined by weight-space merging because joint multi-objective training introduces reward-hacking failures requiring domain-specific fixes.The documented failure modes motivate the modular recipe.
  • Results: An open-weight checkpoint trained without internal data scores close to the deployed version on public benchmarks, supporting the recipe as a source of the gains.The released checkpoint uses the same training recipe but omits internal data.

2 Related Work

Prior work advances Russian generation and general post-training, but does not provide reproducible Russian recipes for reliable instruction execution, tool use, and multi-step behavior. The paper addresses this gap with language-aware evaluation and modular expert training.

  • Russian LLMs: Russian LLM work has focused on training from scratch or adapting multilingual backbones, improving general generation but not publishing reproducible post-training recipes for agentic skills.The paper instead targets reliable instruction execution, structured tool use, and stable multi-step behavior.
  • Language-specific evaluation: English instruction-following verifiers cannot be reused unchanged in Russian because morphology, case, casing, punctuation, and word order create many valid forms.Russian function calling also requires fixed API names, keys, and enumeration values.
  • Post-training: IF, FC, and general chat rewards pull the model in different directions and each exposes a trivial shortcut, making joint training fragile.The same issue appears in multi-objective and rubric-based RL.
  • Post-training: The paper trains one expert per skill and merges experts in weight space, following a strategy used in large post-training systems.This modular approach replaces simultaneous optimization of conflicting rewards.
  • Evaluation: Open benchmarks provide reproducible open-ended chat scoring, while the paper extends that line with a stratified internal benchmark matched to production traffic.The internal benchmark is designed for the enterprise request mix.

3 In-house traffic and Arena

The paper constructs an internal Arena benchmark that balances diversity with production representativeness and uses task-specific judging calibrated to humans. Template-aware sampling and routed evaluation improve coverage and agreement.

  • Benchmark construction: The benchmark samples a diverse, representative subset from ∼100k monthly queries, then routes each query through a task classifier and task-specific evaluation scheme.The sampling interval aligns with the production cycle but the pipeline can apply to other periods.
  • Sampling: Diversity is measured by mean pairwise TF-IDF cosine distance, while representativeness uses Jensen-Shannon distance across model, prompt length, service, and task taxonomy.The objectives conflict because templated traffic contains many near-duplicates.
  • Sampling: Uniform sampling preserves production composition but duplicates structure, whereas diversity-first methods distort service distributions by treating templated near-duplicates as genuine diversity.These approaches fail in opposite directions.
  • Sampling: The template-aware sampler masks variable tokens, groups normalized prompts with LSH, selects variable-span diversity within templates, and allocates budget as √count.This design explicitly preserves template structure while diversifying variable content.
  • Sampling results: 0.953 prompt diversity is achieved while keeping JS distances substantially below pure diversity sampling, making this the only method improving coverage without sacrificing representativeness.The comparison uses Dist. for diversity and JS distances for closeness to production.
  • Judging: κ rises from 0.63 to 0.88 on reference-based subtasks and from 0.57 to 0.72 on open-ended content-generation subtasks under the task-specific pipeline.Table 2 reports Cohen’s κ by task type, with the average weighted by sample counts.
  • Judging: Task classification matches human consensus in 90.6–99.6% of cases, enabling different judging schemes for objective and open-ended requests.Objective tasks use reference answers, while open-ended tasks combine SBS judgments with rubric-based criteria.

4 Training recipe

The training recipe combines a shared SFT checkpoint with three independent GRPO experts targeting instruction following, function calling, and the internal task mix. Domain-specific data and reward fixes address distinct failure modes, especially in Russian tool use.

  • Training recipe: The recipe targets instruction following, function calling, and internal task-distribution alignment using Qwen3-32B in non-reasoning mode for latency and generation-cost constraints.A Cyrillic-dense tokenizer proved more efficient than the base tokenizer.
  • Training recipe: A single combined SFT phase mixes production, general-domain, instruction-following, and function-calling data before branching into independent GRPO runs.Shared SFT is retained because a naive mixed-domain checkpoint preserves per-domain quality without requiring separate SFT experts.
  • General expert: The general expert aligns broad capabilities to the production task mix through data, using a general RM plus length penalties and increased KL control during GRPO.An in-house-preference RM did not improve over the general RM.
  • IF expert: A Russian IF pipeline expands 54 hand-written seed constraints into 43K verified constraints through augmentation, verifier and test-case filtering, and back-translation validation.The dedicated IF increment addresses limited diversity in the in-house IF corpus.
  • FC expert: 1.2M English and 300K Russian FC samples are generated natively because translated function-calling data are structurally unsafe.The pipeline builds tool and multi-turn dialogue pools from scratch and separates planning from simulation.
  • FC expert: Synthetic irrelevance counters over-calling under binary exact-match reward, while the final mixture uses 70% English, 30% Russian, and 80% tool-call versus 20% text targets per language.Ten percent of the text share is synthetic irrelevance.

5 Evaluation

The evaluation tests whether shared SFT, reward-model choices, separate GRPO experts, and SLERP merging preserve quality across production-relevant capability axes. The final merged model is competitive with a much larger baseline while improving deployment-relevant benchmarks and serving at lower cost.

  • SFT ablations: The shared SFT model closely matches domain-specific experts across arena, instruction-following, and tool-calling axes.A mixed-domain SFT stage retains per-domain quality, unlike later joint GRPO optimization.
  • Reward-model ablations: Adapting the reward model to in-house data does not improve performance over the general reward model.The adapted variant also generates 362-token responses on average versus 286, suggesting superficial stylistic bias.
  • GRPO transfer: Single-domain GRPO experts improve their target domain while leaving the other domains largely unchanged.This limited transfer motivates merging separate experts to combine their specialized gains.
  • Joint GRPO versus merging: Joint GRPO raises instruction following from 0.731 to 0.801 but lowers BFCLv3 EN from 61.2 to 54.5 when optimizing two verifiable rewards.Adding the general reward recovers function calling but collapses instruction following, revealing cross-domain interference.
  • Joint GRPO versus merging: Joint three-reward GRPO reaches BFCLv3 60.88 RU and 70.38 EN versus 65.96 and 72.27 for merging, while ruIFEval reaches 0.770 versus 0.799.The general-GRPO warm start reaches merge parity only with a 1.7× budget.
  • Final results and deployment: The merged model scores 69.57 versus 65.83 on the in-house Arena and 0.79 versus 0.77 on in-house BFCL against the ∼7× larger baseline.It also improves SmartSearch F1 from 0.478 to 0.557 and ruWildChat from 52.0 to 80.7.
  • Final results and deployment: The merged checkpoint serves 116M requests monthly from over 200 internal services at 2.8 to 3.9× lower input and output cost.Services previously using the largest platform models see savings of up to 4 to 9×.

6 Conclusion

The paper presents a production-driven method for consolidating a fragmented self-hosted LLM fleet by addressing instruction following, function calling, and internal task-distribution gaps. Modular experts merged with two-stage SLERP yield a competitive non-reasoning model that serves substantial traffic at lower cost.

  • Contribution: Production analysis identified instruction following, function calling, and internal request-distribution alignment as the three main capability gaps.The recipe targets these gaps to consolidate traffic from more than 200 internal applications.
  • Method: Joint GRPO caused objective interference, so the method trains one expert per axis from shared SFT and merges them with two-stage SLERP.The separation is intended to make post-training easier to control, debug, audit, and extend.
  • Method: Each capability axis produced a distinct reward-hacking failure mode requiring a targeted fix.The paper frames modularity as a practical response to these domain-specific failure modes.
  • Results and deployment: The final Qwen3-32B non-reasoning model is competitive on target deployment metrics while serving 116M monthly requests at lower cost.A public checkpoint trained without internal data shows similar gains on public benchmarks, except for the in-house Arena.

Limitations

The evaluation is limited to Russian and English, one corporate deployment, and the Qwen3 model family. Reusing the recipe elsewhere requires recalibrating LLM judges against human annotations.

  • The quantitative evaluation covers only Russian and English, with Russian traffic from one self-hosted corporate deployment.
  • Judge quality should be recalibrated against human annotations when the recipe is reused for another deployment, language, or benchmark distribution.
  • The deployment evidence comes from a single organization and corporate platform, and experiments use only the Qwen3 model family.

Ethical Statement

The paper’s ethical statement concerns privacy-preserving internal data use, human validation, and risks from incorrect outputs or unsafe tool execution. Production analysis and benchmark construction rely on internal work-related data and expert oversight.

  • All in-house benchmarks derive from anonymized, deduplicated internal platform logs processed on self-hosted infrastructure under data-residency constraints.
  • Professional annotators and domain experts validated gold answers, translations, and task-specific verification procedures on internal work-related data.
  • The model is intended for internal enterprise assistance but can produce incorrect outputs or violate constraints.
  • Tool calls should be gated and validated before execution in production because deterministic verifiers reduce but do not eliminate risk.
  • The production failure taxonomy combines formatting and non-format instruction violations into a 37.9% category and analyzes function-calling errors separately.

A.2.2 On benchmark methodology

The benchmark methodology combines representative sampling, task-specific evaluation, and validation against human judgments. Results show that template-aware sampling preserves production composition while criteria-guided judging improves agreement on open-ended tasks.

  • Task-classifier labels match human consensus in 90.6–99.6% of cases across four task types, with task distributions stable across independent time windows.
  • Uniform random and template-based sampling produce Jensen–Shannon distances of 0 and 1.5%, while greedy max-min produces 1.9%.
  • Open-ended tasks use pairwise SBS with explicit criteria covering instruction following, completeness, factual correctness, and formatting.
  • Criteria-Guided SBS raises summarization κ from 0.61 to 0.68, while Per-Criterion + Overall Verdict raises content-generation κ from 0.49 to 0.79.
  • The BFCL coverage filter drops roughly 28% of sampled examples, reducing free-text-only coverage while bounding scorer false positives.
  • The deterministic function-calling scorer checks call type, tools, required arguments, schema validity, and constrained values without invoking an LLM judge.
  • SmartSearch restricts agents to search and page-QA tools to evaluate query formulation, tool choice, and grounding of final answers.

B.3 Russian MultiChallenge

The Russian MultiChallenge evaluation preserves the benchmark’s dialogue structure and criteria while adapting language-dependent content and validating translations. The section also documents alignment methods for controlling verbosity and distribution drift.

  • Russian MultiChallenge preserves dialogue roles, turn order, evaluation axes, and binary pass criteria from the English benchmark.
  • Naive full-dialogue translation caused instruction-following errors, so complex dialogues were translated turn by turn with preceding context.RELIABLE VERSION EDITING averages approximately 2,300 words across 13 turns.
  • 44 of 273 rubrics were localized for Russian, while language-independent semantic rubrics remained in English.Localized rubrics covered INSTRUCTION RETENTION, RELIABLE VERSION EDITING, and SELF-COHERENCE; none were localized for INFERENCE MEMORY.
  • DPO increased Ru-arena-hard response length from 1388 to 2064 tokens, while GRPO without regularization exploited verbosity-biased rewards.The recipe addresses this with multiplicative length penalties and a higher KL coefficient.
  • 1220 compared to 2010 and 320 compared to 375 tokens, respectively, validated GRPO with joint length and KL regularization.

D Instruction-Following Expert

The instruction-following expert combines Russian-specific synthetic data, verifiable rewards, and reward-model quality correction to improve constraint compliance while addressing verifier exploits.

  • D.1 Data Creation: The Russian IF pipeline adapts AutoIF because instruction constraints depend on morphology, punctuation, formatting, lexical restrictions, and style.
  • D.1 Data Creation: 54 seed instruction types were expanded to about 100k candidate constraints covering formatting, lexical, structural, length, and multi-condition requirements.
  • D.1 Data Creation: Exact-match and semantic deduplication reduced the generated constraints to 72k unique constraints.
  • D.1 Data Creation: Validators and test cases were filtered using expected binary labels, consistency checks, and back-translation similarity.
  • D.1 Data Creation: 131k constraint-augmented SFT samples were generated, scored, and reduced to 62k maximum-scoring samples before placement stratification.
  • D.1 Data Creation: The final selection retained 10k user-level and 16k system-level constraint samples with at least one fully valid completion.
  • D.2 Training: DPO did not improve IF metrics and increased average response length by approximately 1.5x, motivating GRPO with verifiable rewards.
  • D.2 Training: Verifier-only rewards caused semantic collapse, so reward-model penalties and prompt-specific teacher baselines were used to penalize formally valid but low-quality outputs.The strongest and most consistent ruIFEval improvements came from the average score of eight Qwen3-235B-Instruct-2507 completions on the same prompt.

E.2 Training

The function-calling expert is trained through staged SFT and GRPO on language-native tool data, with mixture choices targeting multi-turn behavior and over-calling.

  • The tool and dialogue pools feed separate SFT and GRPO stages, whose contributions are studied independently from the deployed model.
  • The dialogue pipeline plans trajectories with judge feedback, then simulates User Agent, Assistant, and Tool Agent turns under asymmetric visibility.
  • SFT uses equal tool-call/text and English/Russian proportions, while length stratification emphasizes dialogues with longer tool descriptions.
  • GRPO rewards exact tool-call matches and non-call outputs when no tool call is parsed, but leaves text quality otherwise unrewarded.
  • Without injected irrelevance, the 32B irrelevance subset scored 78.3 versus 88.5 with injection, exposing over-calling as a dominant exploit.
  • The adopted GRPO mixture is 70% English and 30% Russian, with 80% tool-call and 20% text targets within each language, including 10% synthetic irrelevance among text targets.
  • SFT supplied most English-side gains, whereas GRPO supplied the largest single-stage gain on Russian BFCLv3 despite the Russian share falling from 50% to 30%.
  • The training setup used four 8-GPU H100 nodes for GRPO experts, with expert-specific durations and hyperparameters.The 32B SFT stage took 57 hours on the same node configuration.

G Merging

The paper trains separate IF, FC, and general experts because joint optimization caused cross-domain interference, then combines them with a two-stage SLERP merge.

  • Joint multi-domain GRPO degraded instruction-following performance, while prolonged single-domain RL regressed general-purpose benchmarks.
  • The chosen merge first combines IF and FC experts, then merges that checkpoint with the general expert using empirically selected coefficients.
  • SLERP coefficients were selected separately for parameter groups, with distinct schedules for attention projections, MLP projections, and remaining parameters.
  • A short SFT polishing pass recovered no general-domain gain and consistently introduced minor IF and FC regressions.
  • Joint GRPO and simultaneous three-expert SLERP were evaluated as alternatives to the sequential merge used in the final model.

H Additional Evaluations

Additional evaluations show that the non-reasoning model substantially improves production-relevant instruction following and function calling, while transferring to retrieval tasks. Judge sensitivity changes absolute Arena scores but preserves the ordering underlying the claims.

  • Judge sensitivity: Arena rescoring with three additional deterministic judges changes absolute scores but preserves the deployed checkpoint's first-place ordering.The same cached generations were reused, isolating the judge model as the variable.
  • Instruction following: 0.81 example-level and 0.89 constraint-level Strict-IF accuracy exceed base no-think Qwen3-32B at 0.68 and 0.67, respectively.The model matches the seven-times-larger Qwen3-235B-A22B-Instruct-2507 at the same parameter count.
  • Instruction following: Strict-IF improves sharply while Loose-IF gains are smaller; Loose-IF reaches 0.58 versus 0.52 for base no-think Qwen3-32B and 0.66 for Qwen3-235B-A22B-Instruct-2507.The verifier reward directly targets code-verifiable constraints, not tone, role, or semantic prohibitions.
  • Function calling: 0.79 example pass rate on BFCL matches the seven-times-larger Qwen3-235B-A22B-Instruct-2507 at 0.77 and exceeds Qwen3-32B no-think at 0.71.The largest gains are in call-or-text decision and tool/required-argument matching, while schema validity is saturated at ≥0.97.
  • SmartSearch: 0.434 recall, 0.778 grounded rate, and 0.557 F1R&G improve over base no-think Qwen3-32B at 0.350, 0.754, and 0.478 in SmartSearch.The model ranks second behind Qwen3-235B-A22B-Instruct-2507 and exceeds both thinking-mode baselines on F1R&G.
Loading 2609.01572v1…