PaperlayerView exact revision on arXiv

Source-linked AI summary

SLAI T-Rex: Full-Parameter Post-training of the DeepSeek-V4 Family on Ascend SuperPOD

Dongfang Li, Xiaodong Luo, Ruoyu Sun, Xuhui Chen, Linyuan Qiu, Jian Meng, Zhengxuan Lu, Yiting Wang, Yucheng Xie, Tao Guo, Tianxiang Fang, Jing Li, Sihang Chen, Shihao Hong, Chang Liu, Weihua Dai, Zirong Zeng, Ziwei Zhu, Zhuohan Wang, Zhengjun Yue, Igor Vasilyev, Min Liu, Weijian Sun, Xin Chen, Yingmeng Gao, Jinhua Zhou, Taolue Chen, Chenwei Wu, Dong Zhang, Wenlong Jin, Jinmin Xiang, Barkova Maria, Ushakov Anton, Xianfei Jin, Tian Ding, Zhihang Lin, Qian Chen, Linxin Yang, Mingzhe Yang, Bingwei Zhang, Hongzhang Yang, Fangxue Zhang, Shijun Qin, Jie Yu, Cuihua Hu, Tolstykh Vasiliy, Nosov Ivan, Abdullin Amir, Zhicheng Zhou, Xin Zhang, Zhixiong Ning, Xutong Zhao, Junjie Huang, Jiajun Liu, Weiyan Kong, Zheng Zhang, Wenhan Luo, Lin Hu, Yangbo Guo, Li Zeng, Shihao Zhang, Baotian Hu, Min Zhang, Haizhou Li, Zhiquan Luo

arXiv:2607.20145v2cs.CLcs.AI

TL;DR

Trillion-parameter MoE post-training remains underexplored on non-GPU hardware, while Operations Research is under-represented in LLM post-training. SLAI T-Rex combines Ascend SuperPOD optimization with solver-grounded OR specialization, achieving 34.22% MFU and improved OR performance.

  • Problem

    Trillion-parameter MoE post-training on SIMD hardware and LLM integration with Operations Research remain comparatively underexplored.

  • Method

    SLAI T-Rex couples full-stack Ascend SuperPOD training optimization with solver-grounded CPT–SFT specialization for Operations Research.

  • Results

    34.22% MFU was achieved for trillion-parameter MoE training, while CPT initialization improved OR performance across the evaluated benchmarks.

  • Takeaways & Limitations

    The results demonstrate the practical feasibility of large-scale post-training on Ascend SuperPOD and effective OR workflow scale-up to DeepSeek-V4-Pro.

  • Takeaways & Limitations

    Evaluation remains bounded to the reported OR problem families and general-capability benchmarks, motivating broader future evaluation.

Abstract

from arXiv · show

Full-parameter post-training of trillion-parameter-scale MoE models introduces substantial system-level challenges for large-scale distributed training, including severe memory pressure, non-overlapped communication overhead, and inefficient kernel execution. While most large-scale LLM training systems are built around GPU-based clusters, this report presents an end-to-end optimization practice on the Ascend NPU SuperPOD. Using the DeepSeek-V4 model family as the target workload, we develop a hierarchical optimization framework spanning model-level parallelism, computation-communication orchestration, and low-level kernel execution. The resulting system achieves 34.22% Model FLOPs Utilization (MFU) with a 2.93x improvement over the open-source baseline recipe while maintaining training stability. Building on this optimized infrastructure, we further establish a CPT and SFT workflow for complex Operations Research (OR) tasks. We refer to the integrated framework as SLAI T-Rex. Using DeepSeek-V4-Flash, we develop OR-oriented CPT and SFT data pipelines that combine collected domain resources with solver-verified synthetic optimization documents. The resulting dataset contains 10K high-quality SFT samples spanning four task categories and three problem representations. The specialized model achieves the highest average zero-shot Pass@1 score among the evaluated models, reaching 71.81% and outperforming GPT-5.4-Mini and the base DeepSeek-V4-Flash model by 3.98 and 11.27 percentage points, respectively. Overall, this work demonstrates a full-stack pathway from efficient trillion-parameter model post-training on Ascend infra to domain-specialized Flash models for solver-grounded mathematical modeling, advancing frontier-model systems for complex reasoning.

1. Introduction

SLAI T-Rex combines Ascend-native system optimization for trillion-parameter DeepSeek-V4 post-training with solver-grounded CPT–SFT specialization for Operations Research. It improves both training efficiency and OR reasoning performance across DeepSeek-V4-Flash and DeepSeek-V4-Pro.

  • Motivation and scope: SLAI T-Rex addresses underexplored trillion-parameter MoE post-training on SIMD Ascend hardware, where prior work mainly targets CUDA- or TPU-oriented infrastructure.The framework uses DeepSeek-V4-Pro for system optimization and scale-up validation, and DeepSeek-V4-Flash for the OR workflow.
  • System optimization: 34.22% MFU raises DeepSeek-V4-Pro from 11.67% and delivers a 2.93× improvement over the open-source baseline recipe.The system jointly optimizes multi-dimensional parallelism, communication orchestration, memory management, and kernel execution on Ascend SuperPOD.
  • OR specialization: 71.81% average OR score makes SLAI T-Rex-Flash outperform GPT-5.4-Mini by 3.98 points and original DeepSeek-V4-Flash by 11.27 points.Applying the workflow to DeepSeek-V4-Pro also raises its average OR score from 70.16% to 77.33%.
  • CPT–SFT workflow: Under matched SFT settings, CPT initialization improves DeepSeek-V4-Flash from direct-SFT scores of 65.93% and 48.73% to 71.22% and 59.39% on B4O-Feasible and B4O-ORGEval.The workflow combines solver-grounded CPT data, self-distilled SFT generation, contract-aware cleaning, and benchmark evaluation; CPT adds OR-domain knowledge beyond task-specific SFT.
  • Kernel and hardware optimization: AuraKernel introduces solver-guided AscendC tiling optimization for dominant DeepSeek-V4 bottleneck operators, extending kernel optimization beyond manual and heuristic methods.Its OR solver and dedicated harness enable accurate tiling selection, while the broader analysis connects MoE, sparse attention, memory hierarchy, communication, and kernel execution.

2. Training Infrastructure on Ascend CloudMatrix384 SuperPOD

Ascend SuperPOD profiling identifies communication latency, architecture-specific sparse-attention kernels, and launch-fragmented data movement as the main training inefficiencies. The infrastructure addresses them through virtual pipeline parallelism, overlapped state swapping, and AuraKernel-driven AscendC optimization.

  • Communication orchestration: Profiling identifies TP/DP result collection and traditional 1F1B pipeline send-receive latency as major step-time bottlenecks, while MoE communication remains minimal on the SuperPOD interconnect.The optimization therefore prioritizes communication orchestration rather than treating MoE communication as the dominant overhead.
  • Kernel optimization: AuraKernel provides end-to-end AscendC optimization for memory- and control-bound kernels, complementing kernel-internal tuning with fusion or re-expression of fragmented operator chains.The approach targets both architecture-intrinsic compute-heavy kernels and PyTorch-level sequences of casts, views, tensor movement, and reductions.
  • Pipeline parallelism: Virtual pipeline parallelism suppresses dominant pipeline bubbles while keeping per-device memory within the hardware budget and preserving training stability.DualPipeV would impose a substantial memory penalty to accelerate a component contributing marginally to end-to-end latency.
  • Memory management: A double-buffered swap optimizer overlaps host-state transfers with fused AdamW computation through chunked asynchronous swap-in and swap-out streams.While one chunk is updated, the next is prefetched and the previous chunk is written back asynchronously.
  • Kernel bottlenecks: 206,503 compute-kernel launches and 39.84 s of device compute expose a heavy architecture-specific head alongside a launch-fragmented PyTorch-eager tail.Non-matmul eager operators comprise 91.3% of launches but 66.7% of Task Duration, while vector kernels spend about 15% utilization on computation versus approximately 68% and 40% on loads and stores.
  • Kernel bottlenecks: 16.86% of total kernel computation time comes from SparseAttnSharedkvGrad, making sparse-attention backward the largest individual device-compute contributor.Sparse-attention kernels collectively contribute roughly a quarter of non-communication kernel time, exceeding dense MatMulV3 and MoE GroupedMatmul combined.

3. OR-Oriented Post-Training Workflow

The OR-oriented workflow links original-checkpoint diagnosis, CPT–SFT design, and stage-wise evaluation to address protocol compliance, solver-facing implementation, mathematical formulation, and structural equivalence. Diagnosis shows that prompting can recover some behaviors, but stable one-pass generation requires targeted training and solver-verified data synthesis.

  • Original-checkpoint diagnosis: B4O-Feasible reaches 60.47% Pass@1 versus 34.26% for B4O ORGEval, despite 79.19% code/build pass rate, exposing weak alignment between executability and structural equivalence.The result identifies structural preservation, rather than program execution alone, as a central checkpoint bottleneck.
  • Stage-wise evaluation: 5-shot raises B4O-ORGEval from 34.26% to 71.57%, while Pass@16 improves several value-based benchmarks but remains below 5-shot on B4O ORGEval.These results show that contextual examples activate existing solver-facing protocols and formulation templates, whereas sampling alone does not reliably select canonical formulations.
  • CPT–SFT design: The training design assigns CPT to broaden OR-domain modeling priors and structural knowledge, while SFT stabilizes code-only generation, API usage, objective extraction, schema handling, and prompt-activated behaviors.Pass@16’s recoverable outputs motivate verifier-guided SFT and rejection-sampled distillation, while its remaining structural errors show sampling is insufficient.
  • Error analysis: Across 1,456 failed cases, structural equivalence is the largest error family, while Pass@1 is especially dominated by protocol/API/schema failures that decrease substantially with 5-shot prompting.The analysis groups failures across three inference settings and four benchmark cells, linking distribution shifts to promptable versus deeper modeling problems.
  • Error analysis: The taxonomy covers canonical structure mismatch, protocol and schema errors, discrete-variable semantics, conservation constraints, ratio and unit coupling, and solver-compatible nonlinear reformulation.Examples include incorrect variable or constraint families, missing integrality, distorted loss semantics, corrupted savings relations, and nonlinear objectives unsupported by the solver.
  • CPT data construction: The OR-CPT Data Engine uses solver-verified bidirectional synthesis to convert structured optimization instances into natural-language modeling documents targeting the diagnosed capability gaps.It addresses variable-domain ambiguity, weak parameter grounding, incomplete constraints, nonlinear or ratio-based modeling failures, and structural mismatch.

4. Experimental Results

Experiments show that SFT improves protocol-oriented OR tasks but can impair natural-language modeling, while contract-aware cleaning and CPT initialization recover and extend performance across the four benchmarks. The results support a reproducible 8-node Ascend workflow whose strongest gains come from validated, domain-specific supervision rather than scale alone.

  • Experimental workflow: The preprocessing pipeline, MindSpeed-LLM training script, DeepSeek-V4-Flash, and 8-node 910C environment form a reproducible full-parameter SFT workflow for subsequent data-scale and distillation experiments.The final SFT labels distinguish the uncleaned SFT-10K scale-analysis run from the cleaned CoT-enhanced checkpoint used in transfer and end-to-end comparisons.
  • SFT data scale: SFT-10K improves B4O-Feasible from 60.47 to 65.07 and B4O-ORGEval from 34.26 to 47.21, but lowers NL4OPT from 84.08 to 81.66 and OptiBench from 63.33 to 62.67.Scaling to SFT-50K does not resolve the natural-language degradation; OptiBench instead falls to 58.68.
  • Contract-aware cleaning: Clean-CoT improves over uncleaned SFT-10K by 5.27 percentage points on NL4OPT, 1.50 on OptiBench, 0.86 on B4O-Feasible, and 1.52 on B4O-ORGEval.The gains are strongest on tasks permitting explicit reasoning formats, where modeling checklists teach variable domains, objective direction, and constraint direction.
  • Error analysis: Cleaning reduces wrong_objective_or_model errors from 52 to 31 on NL4OPT and from 154 to 128 on OptiBench, but nonlinear_or_bad_- gurobi_form errors on OptiBench increase from 49 to 56.Thus, chain-of-thought enhancement must remain contract-aware and gated by reviewer and validator checks rather than simply lengthening reasoning.
  • CPT and SFT transfer: +10.66 pp on B4O-ORGEval is the largest CPT-over-SFT gain, alongside +2.59 pp on NL4OPT, +2.95 pp on OptiBench, and +5.29 pp on B4O-Feasible.CPT initialization improves all four benchmarks, strengthening both solver executability and structural equivalence.

5. Conclusion, Limitations, and Future Directions

SLAI T-Rex demonstrates stable trillion-parameter post-training on Ascend NPU SuperPOD while improving solver-grounded OR modeling across DeepSeek-V4-Flash and DeepSeek-V4-Pro. The report releases reproducibility artifacts and identifies more efficient full-parameter training and AgenticRL as future directions.

  • Conclusion: 90.00% AIME 2024, 86.67% AIME 2025, and 67.00% LiveCodeBench show improved general-capability retention, while HumanEval is unchanged and CMMLU decreases slightly to 93.12%.SLAI T-Rex-Pro improves the corresponding baselines of 83.33%, 70.00%, and 55.00%; the SFT-only checkpoint has the highest LiveCodeBench result.
  • Conclusion: 34.22% MFU validates the efficiency and practical feasibility of full-parameter trillion-parameter MoE post-training on Ascend NPU SuperPOD.The optimization spans system-level training orchestration and bottleneck-level kernel optimization on a non-GPU platform.
  • Conclusion: 77.33% average OR score is achieved by the complete SLAI T-Rex-Pro configuration, improving over 70.16% before post-training and 76.74% with SFT alone.The scale-up validation shows that the workflow developed on DeepSeek-V4-Flash remains effective for the larger Pro model.
  • Limitations: Targeted contract-aware cleaning and concise CoT enhancement improve 10K self-distilled SFT data more reliably than scaling it to 50K.Scaling from 10K to 50K is non-monotonic and may degrade natural-language modeling benchmarks, especially motivating repair of modeling errors and strict output contracts.
  • Future Directions: Future work will improve end-to-end full-parameter training, extend the workflow toward AgenticRL, and use operations-research modeling to explore multi-dimensional parallel configurations.Released artifacts include prompt templates, format-contract specifications, curriculum configurations, monitoring callbacks, and configurable Cleaner, Reviewer, and Diagnostic Resolver modules.
  • Conclusion: SLAI T-Rex supports stable full-parameter post-training of trillion-scale MoE and hybrid-attention models while producing measurable OR gains on DeepSeek-V4-Flash and DeepSeek-V4-Pro.The workflow aligns domain knowledge acquisition, solver-grounded data construction, and supervised adaptation for vertical-domain modeling.

Appendix · A Author List

The appendix lists the project leaders, core contributors, and additional contributors involved in the work.

  • A Author List: Core contributors include Dongfang Li, Xiaodong Luo, Ruoyu Sun, Xuhui Chen, and Linyuan Qiu.
  • A Author List: Core contributors also include Jian Meng, Zhengxuan Lu, Yiting Wang, Yucheng Xie, and Tao Guo.
  • A Author List: The core contributor list further includes Tianxiang Fang, Jing Li, Sihang Chen, Shihao Hong, and Chang Liu.
  • A Author List: Additional contributors include Yingmeng Gao, Jinhua Zhou, Taolue Chen, Chenwei Wu, and Dong Zhang.
  • A Author List: The additional contributor list also includes Wenlong Jin, Jinmin Xiang, Barkova Maria, Ushakov Anton, and Xianfei Jin.
  • A Author List: Project leaders are Baotian Hu, Min Zhang, Haizhou Li, and Zhiquan Luo.

B Post-training Recipe Details

This appendix details recipe-level settings for CPT, SFT, and agentic rollout, covering optimization, parallel execution, memory usage, inference behavior, and reproducibility. CPT uses long-context continued pretraining, while SFT trains instruction-response trajectories with loss masking on prompt tokens.

  • Overview: The appendix organizes post-training settings around optimization, parallel execution, memory usage, and reproducibility.
  • CPT recipe: CPT uses a long-context Ascend-based continued-pretraining recipe covering optimization, parallelism, and memory-saving mechanisms.Table 18 reports the configuration for domain-adaptive continued pretraining.
  • SFT recipe: SFT optimizes instruction-response trajectories and masks prompt-token loss so gradients apply only to target responses.Table 19 reports the recipe configuration supporting instruction following and solver-oriented modeling.
  • Agentic rollout configuration: Agentic rollout is evaluated in inference mode without parameter updates, so its configuration is separated from CPT and SFT recipes.Reported rollout parameters should describe inference-time behavior and future reinforcement-learning reproducibility.

C Solver-Verified OR-CPT Data Synthesis: Engine Design and Illustrative Cases

The method converts parameterized optimization instances into self-contained CPT documents through bidirectional language-model transformations, while generator contracts, static checks, invariant preservation, and independent solver execution filter mathematically incorrect samples.

  • Engine Design: The synthesis method preserves mathematical invariants across language-model transformations and prevents plausible but incorrect documents from entering the CPT corpus.It converts each parameterized optimization generator into a self-contained CPT document.
  • Engine Design: The pipeline transforms verified optimization instances into business problems and back into mathematical models and executable programs, with admissibility determined by generator contracts, static checks, and independent solver execution.Language-model fluency is explicitly not treated as evidence of mathematical correctness.

C.1 Role in the post-training workflow

The OR-CPT Engine constructs faithful, semantically covered, provenance-preserving documents upstream of the Ascend 910C training stack. Its continued-pretraining data and subsequent SFT data serve distinct objectives: exposing OR knowledge first, then eliciting it under task instructions and output contracts.

  • Role in the post-training workflow: The OR-CPT Engine is separate from the Ascend 910C training stack, constructing mathematically faithful, semantically covered, provenance-preserving documents for continued pre-training.The training stack instead tokenizes, packs, samples, and optimizes the resulting corpus; the engine exports self-contained documents rather than instruction–response conversations.
  • Role in the post-training workflow: CPT exposes OR terminology, formulation patterns, solver APIs, and verification habits, whereas SFT teaches their elicitation under task instructions and output contracts.The same OR taxonomy guides both stages, but their data representations and optimization objectives remain distinct.

C.2 End-to-end engine architecture … C.5 Controlled scenarios and CPT rendering

SLAI T-Rex uses independently verified optimization seeds, controlled bidirectional synthesis, and executable validation to produce solver-grounded CPT documents. The pipeline preserves mathematical structure while varying domain narratives and rendering complete business, modeling, code, solution, and validation content.

  • C.2 End-to-end engine architecture: The engine independently solves sampled optimization instances, filters invalid seeds, backtranslates verified seeds into business problems, and applies language-quality checks before reconstruction.Filtering targets infeasibility, degeneracy, numerical instability, duplication, numeric coverage, unit consistency, business grounding, and solution leakage.
  • C.3 Mathematical seeds and domain contracts: Each seed stores structured optimization data, executable or LP artifacts, and semantic metadata, with deterministic random seeds supporting regeneration and auditing.Stored mathematical content includes sets, coefficients, objectives, domains, bounds, and constraint families; metadata includes task family, concepts, difficulty, and formulation variant.
  • C.3 Mathematical seeds and domain contracts: Generator profiles define canonical mathematical signatures, while family contracts enforce shared invariants such as balance equations, linking rules, conservation laws, and required variable families.These layers are grounded in declared seed facts and restrict valid interpretations across related generators.
  • C.3 Mathematical seeds and domain contracts: Seed eligibility requires more than feasibility: the engine checks solution and constraint activity, objective recomputation, numerical precision, and duplicate signatures.This excludes solvable but uninformative instances with inactive decisions, fixed variables, or no genuine trade-off.
  • C.4 Prompt-controlled bidirectional synthesis: Backtranslation preserves coefficients, bounds, units, and indexed relations while hiding solver status, solutions, objective values, implementation artifacts, and code.It may vary industry, stakeholder perspective, planning trigger, horizon, naming style, and units without changing the source mathematics.
  • C.4 Prompt-controlled bidirectional synthesis: Forward modeling reconstructs explanations, formulations, and executable code using only declared indices and coefficients, returns a structured self-check, and must not hard-code the optimum.Static inspection and independent execution remain authoritative because generated prose can omit or misindex required constraints.
  • C.5 Controlled scenarios and CPT rendering: Scenario controls vary industry lenses, decision triggers, organization types, planning frames, naming styles, and units while preserving the underlying OR structure.Negative guidance prevents fluent narratives from silently changing model families, such as converting transportation into routing without vehicles and sequencing.
  • C.5 Controlled scenarios and CPT rendering: After executable verification, the renderer produces self-contained CPT documents containing business data, model explanations, formulations, solver code, interpreted solutions, and validation checks.Style and section order may vary, but mandatory mathematical and validation content remains fixed.

C.6 Illustrative case I: capacitated facility location

The capacitated facility-location example selects sites and allocates demand jointly under fixed activation costs, capacity limits, and demand satisfaction, yielding an objective value of 590. The case emphasizes that valid reconstruction must preserve these structural invariants and reproduce the reference objective within tolerance.

  • C.6 Illustrative case I: capacitated facility location: Opening B and C serves all demand, with B serving 30 units of Z1 and 35 of Z2 and C serving 10 of Z1 and 25 of Z3; B reaches capacity 65.The resulting objective is 95 + 80 + 6(30) + 8(10) + 3(35) + 2(25) = 590.
  • C.6 Illustrative case I: capacitated facility location: The fixed-charge trade-off requires jointly choosing opened sites and service allocations rather than selecting the cheapest coefficient independently for each demand zone.Site C has the lowest fixed cost and cheapest service to Z3, while B is cheapest for Z2; neither site can serve total demand alone because of capacity limits.
  • C.6 Illustrative case I: capacitated facility location: Valid reconstructions must preserve fixed activation, demand satisfaction, and capacity linking, while reproducing the reference objective within tolerance.The same formulation can represent clinic siting, regional repair depots, or computing-service hubs despite changes in entities and units.

C.7 Illustrative case II: executable but structurally wrong models

Executable optimization models can still be structurally unfaithful: an OPTIMAL status certifies only the constructed program, motivating validation gates and conservative repair rather than invented data.

  • Structural validation: The faithful formulation is distinguished from two executable structural errors using static term checks, family-level contracts, and independent objective comparison.Table 21 identifies which validation gate detects each reconstruction error.
  • Structural validation: Syntactically valid, executable programs solved to OPTIMAL status may still misrepresent the intended business problem.Optimality applies only to the program actually constructed, not to its faithfulness.
  • Safe repair: Missing coefficients or bounds should trigger regeneration, rejection, or authoritative-source recovery rather than invented values that merely bound the model or approach a reference objective.Repair must preserve the problem specification instead of optimizing toward a desired outcome.

D CPT–SFT–Deployment–Evaluation Provenance … D.4 Confidence levels

SLAI T-Rex introduces a lightweight provenance system linking CPT, SFT, artifact conversion, deployment, and evaluation through manifests and a centralized registry. Stage-specific records preserve model identity, lineage, reproducibility, auditability, and confidence of cross-stage links, including legacy runs.

  • D CPT–SFT–Deployment–Evaluation Provenance: The provenance system links CPT, SFT, conversion, deployment, and evaluation artifacts through per-stage manifests and a local file-based registry.This design makes benchmark improvements attributable to training configurations, checkpoints, deployed weights, and upstream outputs.
  • D.1 Design principles: Model identity is defined by the artifact manifest and upstream lineage rather than the served model name, IP address, or port.Multiple model versions can share an endpoint while retaining unambiguous weight identification.
  • D.1 Design principles: Each stage records actual paths, environment variables, and upstream relationships in a structured manifest immediately upon completion.Point-of-execution recording avoids fragile post-hoc reconstruction and preserves environmental details such as CANN version and parallelism configuration.
  • D.1 Design principles: Existing configuration snapshots, checkpoint directories, and converted artifacts can be registered post-hoc, preserving legacy experiments for cross-run comparison and trend analysis.
  • D.2 Provenance chain and registry layout: The five-stage chain stores manifests at output locations and in a centralized JSON registry organized by training runs, artifacts, deployments, and evaluation runs.A lightweight index maintains aggregate counts, while manifests remain co-located with artifacts.
  • D.3 Stage-level linkage mechanism: CPT-to-SFT lineage is established by matching the SFT checkpoint load path with the CPT checkpoint save path, recording path_match confidence or missing status.This linkage enables transfer gains to be traced to the specific CPT initialization checkpoint.
  • D.3 Stage-level linkage mechanism: Conversion and deployment manifests connect HuggingFace shards to training provenance and bind artifact versions to served endpoints, while evaluation can carry full lineage when given a deployment manifest reference.Without that reference, evaluation still runs, but scores cannot be automatically traced to specific weights or dashboard experiments.
  • D.4 Confidence levels: Each provenance link carries a confidence annotation distinguishing exact manifest-based linkage from path-based or heuristic matching.

D.5 Dashboard monitoring and retroactive registration

The provenance registry feeds a dashboard that visualizes experiment runs alongside training metrics, provenance status, and confidence annotations. A rebuild script incorporates newly completed runs, while legacy runs can be registered retroactively from existing artifacts.

  • Dashboard monitoring: The dashboard visualizes the full experiment graph, with each run showing identifiers, job type, iterations, key metrics, provenance status, and confidence annotations.Runs with complete manifest coverage receive exact_manifest or path_match confidence labels.
  • Dashboard monitoring: A build script scans the training archive and centralized registry, aggregates manifest contents into JSON, and refreshes the frontend so newly completed runs appear on the next rebuild.
  • Retroactive registration: Legacy runs can be registered retroactively by directing registration tooling to existing configuration files, checkpoint directories, or artifact outputs.

E Training Schedule and Monitoring Design

The training design balances OR adaptation with mathematics, code, and general-domain data, using a predefined curriculum and monitoring dashboard to preserve capabilities and track stability.

  • E Training Schedule and Monitoring Design: Overly concentrated OR-domain adaptation can weaken general capabilities, motivating a balanced mixture of OR, mathematics, code, and general-domain data.
  • E Training Schedule and Monitoring Design: The schedule uses a predefined curriculum to organize the balanced training-data mixture.
  • E Training Schedule and Monitoring Design: A monitoring dashboard tracks training stability and capability retention during training.

E.1 Static Curriculum Learning Design · E.2 Monitoring Dashboard

The training mixture follows a manually fixed three-stage curriculum that shifts emphasis from mathematical reasoning and general code toward Operations Research formats. A WandB monitoring dashboard tracks training health and capability retention for human inspection rather than automated control.

  • E.1 Static Curriculum Learning Design: The curriculum uses a pre-defined three-stage schedule with a step-dependent mixing vector over OR, mathematics, code, and general data.The mixture proportions, thresholds, and stage boundaries are fixed before training and are not dynamically modified.
  • E.1 Static Curriculum Learning Design: During the foundation stage covering the first 30% of steps, mathematics and clean general code each receive weight 0.35, general data 0.20, and OR data 0.10.This phase is intended to reinforce precise token generation and algorithmic logic.
  • E.1 Static Curriculum Learning Design: During the 30–70% domain ramp-up, OR weight increases linearly to 0.40 while code decreases gradually to 0.15, with OR formats introduced as DP, then DT, then DPS.The three formats are Data-in-Problem, Data-in-Table, and Data-Problem-Separate.
  • E.1 Static Curriculum Learning Design: The curriculum is a manually designed default trajectory whose configuration was selected from preliminary runs and the Section 4.4 data-mixture ablation.Related curriculum-like strategies are cited as Tzannetos et al., 2026; Zhao et al., 2026; Liang et al., 2025.
  • E.2 Monitoring Dashboard: The monitoring dashboard logs two metric groups every 500 training steps and serves as an observability layer for human developers, not a closed-loop controller.Training-health metrics include loss, gradient norm, NaN counts, and MFU; gradient spikes or NaN loss trigger manual investigation.
  • E.2 Monitoring Dashboard: Capability-retention metrics cover OR validation perplexity and proxy rewards, held-out Python syntax health, and general-domain perplexity.The OR proxies use NL4OPT and OptiBench, while syntax health evaluates 200 pure-Python algorithmic problems.
  • E.2 Monitoring Dashboard: Metrics are visualized in WandB for manual inspection, allowing engineers to pause OR-centric training, inspect data, or adjust the mixture when capability degradation persists.Thresholds and resulting decisions are made by human engineers rather than an automated state machine; the design is inspired by Qi et al., 2026.

F Representative Original-Checkpoint Error Cases

The original DeepSeek-V4-Flash checkpoint fails through structural, protocol, and domain-semantic errors that can produce executable but incorrect optimization models. These cases motivate template-focused CPT and SFT, including canonical formulations, schema-safe code, conservation patterns, and solver-aware reformulation.

  • Canonical LP-Graph Mismatch under ORGEval: Structural ORGEval failures occur when executable LPs use the wrong constraint family or graph structure, motivating CPT for domain templates and SFT for canonical formulations.Even matching variable and constraint counts may leave the WL graph incorrect.
  • Output-Contract and Schema Failures: Protocol and schema errors prevent reliable execution: markdown fences cause syntax failure, while string shift labels used as list indices cause a TypeError, motivating SFT.These failures occur despite the model having useful OR knowledge because Pass@1 behavior violates code-only contracts and schema-safe data access.
  • Discrete Semantics Error in NL4OPT: The NL4OPT case predicts 28125 versus the ground truth 28250 because continuous variables relax an integer workforce decision and yield a fractional optimum.The error is semantic: worker counts must be modeled as integer rather than continuous variables.
  • Loss/Yield-Driven Conservation Error: In the water-network case, Pass@1 predicts 265.5768398591736 versus the ground truth 531.1536797183472 because evaporation is incorrectly treated as both additive cost and conservation loss.The code reaches an optimum, but it represents the wrong physical system and requires stable loss, yield, shrinkage, and evaporation conservation templates.
  • Ratio, Unit, and Total-Quantity Coupling Error: The OptiBench case predicts 10000.0 in Pass@1 and Pass@16 versus the ground truth 18000.0 because investment, ratio, units, and total savings are coupled incorrectly.The failure requires explicit tracking of numerators, denominators, units, and scale variables rather than generic API knowledge.
  • Solver-Aware Reformulation: Additional failures show that meaningful mathematical expressions still require solver-aware reformulation, including auxiliary variables, legitimate nonconvex constraints, or analytic simplification before coding.Later inference settings can avoid an initial build error while still returning the wrong distance, so the limitation is not merely Gurobi API usage.
Loading 2607.20145v2…